Building and Running Zephyr with ACRN¶
Zephyr’s is capable of running as a guest under the x86 ACRN hypervisor (see https://projectacrn.org/). The process for getting this to work is somewhat involved, however.
Build your Zephyr App¶
First, build the Zephyr application you want to run in ACRN as you normally would, selecting an appropriate board:
west build -b acrn_ehl_crb samples/hello_world
Note the kconfig output in build/zephyr/.config, you will need to
reference that to configure ACRN later.
The Zephyr build artifact you will need is build/zephyr/zephyr.bin,
which is a raw memory image. Unlike other x86 targets, you do not
want to use zephyr.elf!
Configure and build ACRN¶
First you need the source code, clone from:
git clone https://github.com/projectacrn/acrn-hypervisor
Like Zephyr, ACRN favors build-time configuration management instead
of runtime probing or control. Unlike Zephyr, ACRN has single large
configuration files instead of small easily-merged configuration
elements like kconfig defconfig files or devicetree includes. You
have to edit a big XML file to match your Zephyr configuration.
Choose an ACRN host config that matches your hardware (“ehl-crb-b” in
this case). Then find the relavent file in
misc/config_tools/data/<platform>/hybrid.xml.
First, find the list of <vm> declarations. Each has an id=
attribute. For testing Zephyr, you will want to make sure that the
Zephyr image is ID zero. This allows you to launch ACRN with just one
VM image and avoids the need to needlessly copy large Linux blobs into
the boot filesystem. Under currently tested configurations, Zephyr
will always have a “vm_type” tag of “SAFETY_VM”.
Configure Zephyr Memory Layout¶
Next, locate the load address of the Zephyr image and its entry point address. These have to be configured manually in ACRN. Traditionally Zephyr distributes itself as an ELF image where these addresses can be automatically extracted, but ACRN does not know how to do that, it only knows how to load a single contiguous region of data into memory and jump to a specific address.
Find the “<vm id=”0”>…<os_config>” tag that will look something like this:
<os_config> <name>Zephyr</name> <kern_type>KERNEL_ZEPHYR</kern_type> <kern_mod>Zephyr_RawImage</kern_mod> <ramdisk_mod/> <bootargs></bootargs> <kern_load_addr>0x1000</kern_load_addr> <kern_entry_addr>0x1000</kern_entry_addr> </os_config>
The kern_load_addr tag must match the Zephyr LOCORE_BASE symbol
found in include/arch/x86/memory.ld. This is currently 0x1000 and
matches the default ACRN config.
The kern_entry_addr tag must match the entry point in the built
zephyr.elf file. You can find this with binutils, for example:
$ objdump -f build/zephyr/zephyr.elf build/zephyr/zephyr.elf: file format elf64-x86-64 architecture: i386:x86-64, flags 0x00000012: EXEC_P, HAS_SYMS start address 0x0000000000001000
By default this entry address is the same, at 0x1000. This has not always been true of all configurations, however, and will likely change in the future.
Configure Zephyr CPUs¶
Now you need to configure the CPU environment ACRN presents to the
guest. By default Zephyr builds in SMP mode, but ACRN’s default
configuration gives it only one CPU. Find the value of
CONFIG_MP_NUM_CPUS in the Zephyr .config file give the guest that
many CPUs in the <cpu_affinity> tag. For example:
<cpu_affinity> <pcpu_id>0</pcpu_id> <pcpu_id>1</pcpu_id> </cpu_affinity>
Note that these indexes are physical CPUs on the host. When configuring multiple guests, you probably don’t want to overlap these assignments with other guests. But for testing Zephyr simply using CPUs 0 and 1 works fine. (Note that ehl-crb-b has four physical CPUs, so configuring all of 0-3 will work fine too, but leave no space for other guests to have dedicated CPUs).
Build ACRN¶
Once configuration is complete, ACRN builds fairly cleanly:
$ make -j BOARD=ehl-crb-b SCENARIO=hybrid
The only build artifact you need is the ACRN multiboot image in
build/hypervisor/acrn.bin
Assemble EFI Boot Media¶
ACRN will boot on the hardware via the GNU GRUB bootloader, which is itself launched from the EFI firmware. These need to be configured correctly.
Locate GRUB¶
First, you will need a GRUB EFI binary that corresponds to your hardware. In many cases, a simple upstream build from source or a copy from a friendly Linux distribution will work. In some cases it will not, however, and GRUB will need to be specially patched for specific hardware. Contact your hardware support team (pause for laughter) for clear instructions for how to build a working GRUB. In practice you may just need to ask around and copy a binary from the last test that worked for someone.
Create EFI Boot Filesystem¶
Now attach your boot media (e.g. a USB stick on /dev/sdb, your hardware may differ!) to a Linux system and create an EFI boot partition (type code 0xEF) large enough to store your boot artifacts. This command feeds the relevant commands to fdisk directly, but you can type them yourself if you like:
# for i in n p 1 "" "" t ef w; do echo $i; done | fdisk /dev/sdb ... <lots of fdisk output>
Now create a FAT filesystem in the new partition and mount it:
# mkfs.vfat -n ACRN_ZEPHYR /dev/sdb1 # mkdir -p /mnt/acrn # mount /dev/sdb1 /mnt/acrn
Copy Images and Configure GRUB¶
ACRN does not have access to a runtime filesystem of its own. It receives its guest VMs (i.e. zephyr.bin) as GRUB “multiboot” modules. This means that we must rely on GRUB’s filesystem driver. The three files (GRUB, ACRN and Zephyr) all need to be copied into the “/efi/boot” directory of the boot media. Note that GRUB must be named “bootx64.efi” for the firmware to recognize it as the bootloader:
# mkdir -p /mnt/acrn/efi/boot # cp $PATH_TO_GRUB_BINARY /mnt/acrn/efi/boot/bootx64.efi # cp $ZEPHYR_BASE/build/zephyr/zephyr.bin /mnt/acrn/efi/boot/ # cp $PATH_TO_ACRN/build/hypervisor/acrn.bin /mnt/acrn/efi/boot/
At boot, GRUB will load a “efi/boot/grub.cfg” file for its runtime
configuration instructions (a feature, ironically, that both ACRN and
Zephyr lack!). This needs to load acrn.bin as the boot target and
pass it the zephyr.bin file as its first module (because Zephyr was
configured as <vm id="0"> above). This minimal configuration will
work fine for all but the weirdest hardware (i.e. “hd0” is virtually
always the boot filesystem from which grub loaded), no need to fiddle
with GRUB plugins or menus or timeouts:
# cat > /mnt/acrn/efi/boot/grub.cfg<<EOF set root='hd0,msdos1' multiboot2 /efi/boot/acrn.bin module2 /efi/boot/zephyr.bin Zephyr_RawImage boot EOF
Now the filesystem should be complete
# umount /dev/sdb1 # sync
Boot ACRN¶
If all goes well, booting your EFI media on the hardware will result in a running ACRN, a running Zephyr (because by default Zephyr is configured as a “prelaunched” VM), and a working ACRN command line on the console.
You can see the Zephyr (vm 0) console output with the “vm_console” command:
ACRN:\>vm_console 0 ----- Entering VM 0 Shell ----- *** Booting Zephyr OS build v2.6.0-rc1-324-g1a03783861ad *** Hello World! acrn