Preparing and running QEMU ARM for Cubieboard (Part 1)

Preparing and running QEMU ARM for Cubieboard (Part 1)

This is part 1 of QEMU Board Emulation series

In this post I will cover the following things

At the end of this post we will have a working QEMU, and starting point for U-Boot and Linux kernel which can be used for further work. A simple init ramdsik will be used instead of root filesystem, so the Linux kernel does not panic on boot. In the next post a real root filesystem will be used.

The Cubieboard will be used as the target board.

Before we start with the development, prerequisites need to be installed. I am using Ubuntu 22.04 as the host system, so if different system is used it is possible that some additional packages from prerequisites need to be installed.

# Installing prerequisites
sudo apt -y install git libglib2.0-dev libfdt-dev libpixman-1-dev \
    zlib1g-dev libnfs-dev libiscsi-dev git-email libaio-dev \
    libbluetooth-dev libbrlapi-dev libbz2-dev libcap-dev \
    libcap-ng-dev libcurl4-gnutls-dev libgtk-3-dev libibverbs-dev \
    libjpeg8-dev libncurses5-dev libnuma-dev librbd-dev \
    librdmacm-dev libsasl2-dev libsdl2-dev libseccomp-dev \
    libsnappy-dev libssh2-1-dev libvde-dev libvdeplug-dev \
    libxen-dev liblzo2-dev valgrind xfslibs-dev kpartx libssl-dev \
    net-tools python3-sphinx python3-sphinx-rtd-theme libsdl2-image-dev \
    flex bison libgmp3-dev libmpc-dev device-tree-compiler u-boot-tools \
    bc git libncurses5-dev lzop make tftpd-hpa uml-utilities \
    nfs-kernel-server swig ninja-build libusb-1.0-0-dev python3-venv \
    python3-setuptools python3-dev fdisk

The development will be done in the $HOME/work directory, and that directory will be refered to as $PROJ_DIR.


Building QEMU

The latest stable version at the time of writing is 8.2.2.

Downloading QEMU source code

The QEMU emulator source code can be obtained as an archive from here or as a git repository from github.

If archive is used, then QEMU can be downloaded and prepared using

# archive
wget -c https://download.qemu.org/qemu-8.2.2.tar.xz
tar xf qemu-8.2.2.tar.xz && mv qemu-8.2.2 qemu
cd qemu

If github is used, then QEMU can be downloaded and prepared using

# github
git clone https://github.com/qemu/qemu.git
cd qemu
git checkout v8.2.2 -b devel
git submodule init
git submodule update --recursive

Configuring and building QEMU

Before building QEMU, it needs to be configured. In this process various options can be selected. For these posts following configuration command will be used

# configuration
mkdir -p build && cd build
../configure --target-list=arm-softmmu \
             --enable-sdl \
             --enable-tools \
             --enable-fdt \
             --enable-libnfs

SDL is selected as GUI backend, QEMU tools for handling image and network will be compiled, device tree support and NFS support will also be included.


After the configuration step is done, code can be compiled using

# building
make -j

The output files will be in ./arm-softmmu directory, where the most import one is qemu-system-arm. The tools, like qemu-img, will be in the ./ directory.

In order to keep all of the details in one place, all relevant paths and exports will be saved to a file called env.sh in the root of the project.

So, after compiling the QEMU the $PROJ_DIR/env.sh should look like

# Environment file
PROJ_DIR=$HOME/work

export PATH=$PROJ_DIR/qemu/build/arm-softmmu:$PROJ_DIR/qemu/build:$PATH

Running QEMU

QEMU has many options which can be selected at runtime. Some of them are shown in the following table

switch description example value
-M select machine that will be emulated cubieboard, sabrelite
-m set amount of RAM memory 512M, 1G
-kernel executable file that will be loaded u-boot or kernel ELF file
-drive specify storage drive to be used file=sd.img,format=raw,if=sd
-device specify device to be allocated loader,file=zImage,addr=0x62000000,force-raw
-net ethernet network nic, tap,ifname=tap0,script=no
-nographic disable display window N/A
-serial set how serial interface is connected stdio, pty

The list of all switches can be obtained with

qemu-system-arm --help

and available values for a specific switch using

qemu-system-arm <switch> ?

Before using QEMU we need an executable file to run, so we will proceed to obtaining toolchain and building U-Boot.

Getting toolchain

The cross-compilation toolchain for ARM architecture can obtained in various ways: get a prebuilt from ARM/Linaro or Bootlin, or build a custom toolchain using crosstool-NG.

In this post we will be using a prebuilt toolchain from ARM. It can be downloaded using

# Download toolchain
wget -c https://developer.arm.com/-/media/Files/downloads/gnu/13.2.rel1/binrel/arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-linux-gnueabihf.tar.xz
mkdir toolchain
tar xf arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-linux-gnueabihf.tar.xz -C toolchain --strip-components=1

Update the $PROJ_DIR/env.sh file so it looks like

# Environment file
PROJ_DIR=$HOME/work

export PATH=$PROJ_DIR/toolchain/bin:$PROJ_DIR/qemu/build/arm-softmmu:$PROJ_DIR/qemu/build:$PATH

This way, before cross-compiling any part of the code it is enough just to source the $PROJ_DIR/env.sh script.

Building U-Boot

Even though QEMU can be used without the bootloader, where Linux kernel image, device tree blob and kernel command line are passed, in this blog series the goal is to emulate also the boot process. We will use U-Boot as bootloader, but Barebox can also be used.

Downloading U-Boot source code

As with QEMU, the source code can be obtained in the form of an archive or from a github repository.

If archive is used, then U-Boot source code can be downloaded and prepared using

# archive
wget -c https://ftp.denx.de/pub/u-boot/u-boot-2024.01.tar.bz2
tar xf u-boot-2024.01.tar.bz2 && mv u-boot-2024.01 u-boot
cd u-boot

If github is used, then U-Boot source code can be downloaded and prepared using

# github
git clone https://github.com/u-boot/u-boot.git
cd u-boot
git checkout v2024.01 -b devel

Configuring and building U-Boot

Before compiling U-Boot the environment script that was created needs to be sourced in order to add toolchain executables to the $PATH

# sourcing environment script
source $PROJ_DIR/env.sh

The configuration for Cubieboard is done using the following command

# Configure U-Boot
make CROSS_COMPILE=arm-none-linux-gnueabihf- O=build_cubieboard Cubieboard_defconfig

If additional adjustment needs to be made, it can be done using the menuconfig command as

# Configure U-Boot
make CROSS_COMPILE=arm-none-linux-gnueabihf- O=build_cubieboard menuconfig

In our case, since we are using just raw U-Boot, we need to embed the device tree file with the U-Boot image. The device tree file holds hardware description information, so without it U-Boot will not start.

In order to do it, CONFIG_OF_EMBED setting needs to be applied

Device Tree Control -> Provider of DTB for DT control -> Embedded DTB for DT control

Once configuration is done, the build is started using

# Build U-Boot
make CROSS_COMPILE=arm-none-linux-gnueabihf- O=build_cubieboard -j

After the build is completed, in the $PROJ_DIR/u-boot/build_cubieboard directory there will be a file called u-boot which will be run inside QEMU. For simpler handling, the path to this file can be added into the environment file, so next time it is sourced we will be able to access u-boot executable from anywhere.

After exporting this path the $PROJ_DIR/env.sh should look like

# Environment file
PROJ_DIR=$HOME/work

export PATH=$PROJ_DIR/toolchain/bin:$PROJ_DIR/qemu/build/arm-softmmu:$PROJ_DIR/qemu/build:$PATH

export UBOOT=$PROJ_DIR/u-boot/build_cubieboard/u-boot

Running U-Boot inside QEMU

In order to run U-Boot in QEMU the u-boot ELF file needs to be passed with the -kernel switch. Since at this moment only U-Boot is ready, we will be able to enter U-Boot and look around the provided console interface.

The command that can be used to run U-Boot inside QEMU is (do not forget to source the $PROJ_DIR/env.sh file beforehand)

# Run U-Boot in QEMU
qemu-system-arm -M cubieboard -kernel $UBOOT -serial mon:stdio
U-Boot 2024.01 (Mar 23 2024 - 20:40:51 +0100) Allwinner Technology

CPU:   Allwinner A10 (SUN4I)
Model: Cubietech Cubieboard
DRAM:  1 GiB
Core:  74 devices, 24 uclasses, devicetree: embed
WDT:   Not starting watchdog@1c20c90
MMC:   mmc@1c0f000: 0
Loading Environment from FAT... Card did not respond to voltage select! : -110
** Bad device specification mmc 0 **
Unknown monitor
Unknown monitor
In:    serial,usbkbd
Out:   serial,vidconsole
Err:   serial,vidconsole
Net:
Error: ethernet@1c0b000 No valid MAC address found.
No ethernet found.

starting USB...
Bus usb@1c14000: USB EHCI 0.00
Bus usb@1c14400: USB OHCI 0.0
Bus usb@1c1c000: USB EHCI 0.00
Bus usb@1c1c400: USB OHCI 0.0
scanning bus usb@1c14000 for devices... 1 USB Device(s) found
scanning bus usb@1c14400 for devices... 1 USB Device(s) found
scanning bus usb@1c1c000 for devices... 1 USB Device(s) found
scanning bus usb@1c1c400 for devices... 1 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot:  0

After testing, exit QEMU with Ctrl+a,x.

Building Linux kernel

Latest stable kernel at the time of writing is 6.8.1

Downloading source code

The code can be obtained from git server or from an archive.

If archive is used, then Linux source code can be downloaded and prepared using

# archive
wget -c https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.8.1.tar.xz
tar xf linux-6.8.1.tar.xz && mv linux-6.8.1 linux
cd linux

If git server is used, then Linux source code can be downloaded and prepared using

# git server
git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
cd linux
git checkout v6.8.1 -b devel

Configuring and building

Before compiling Linux, the environment script that was created needs to be sourced in order to add toolchain executables to the $PATH

# sourcing environment script
source $PROJ_DIR/env.sh

The configuration for Cubieboard is done using the following command

# Configure Linux kernel - multi_v7
make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabihf- O=build_cubieboard multi_v7_defconfig

The multi_v7_defconfig is a universal configuration for many ARMv7 based boards, where actual configuration is done based on the Device Tree.

If additional adjustment needs to be made, it can be done using the menuconfig command as

# Configure Linux - manual
make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabihf- O=build_cubieboard menuconfig

Once configuration is done, the build is started using

# Build Linux kernel
make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabihf- O=build_cubieboard -j

After the build is completed, two files will be needed for running in the QEMU:

  • file zImage in the $PROJ_DIR/linux/build_cubieboard/arch/arm/boot directory - compressed Linux kernel image file
  • file sun4i-a10-cubieboard.dtb in the $PROJ_DIR/linux/build_cubieboard/arch/arm/boot/dts/allwinner directory - compiled device tree file with hardware description used by Linux kernel to set up hardware.

For simpler handling, the path to these file can be added into the environment file. After exporting these paths the $PROJ_DIR/env.sh should look like

# Environment file
PROJ_DIR=/home/user/qemu_devel

export PATH=$PROJ_DIR/toolchain/bin:$PROJ_DIR/qemu/build/arm-softmmu:$PROJ_DIR/qemu/build:$PATH

export UBOOT=$PROJ_DIR/u-boot/build_cubieboard/u-boot

export ZIMAGE=$PROJ_DIR/linux/build_cubieboard/arch/arm/boot/zImage

export DTB=$PROJ_DIR/linux/build_cubieboard/arch/arm/boot/dts/allwinner/sun4i-a10-cubieboard.dtb

Running U-Boot and Linux inside QEMU

The idea is to use U-Boot to start the Linux kernel, the same way it would have been done on the real board. Since we will not handle flash, SD card interface or network intferace in this post, we will use the ‘loader’ feature of the QEMU to place the Linux kernel image and Device Tree file at the appropriate addresses in memory, as if the U-Boot code had already copied them from some of the possible bootable locations. In some of the other posts other methods will be covered.

The command that can be used to run U-Boot inside QEMU, with loading Linux kernel image and device tree blob at the appropriate addresses is (do not forget to source the $PROJ_DIR/env.sh file beforehand)

# Run U-Boot and Linux kernel in QEMU
qemu-system-arm -M cubieboard -m 1G -kernel $UBOOT -nographic \
                -device loader,file=$ZIMAGE,addr=0x42000000,force-raw=on \
                -device loader,file=$DTB,addr=0x43000000,force-raw=on
U-Boot 2024.01 (Mar 23 2024 - 20:40:51 +0100) Allwinner Technology

CPU:   Allwinner A10 (SUN4I)
Model: Cubietech Cubieboard
DRAM:  1 GiB
Core:  74 devices, 24 uclasses, devicetree: embed
WDT:   Not starting watchdog@1c20c90
MMC:   mmc@1c0f000: 0
Loading Environment from FAT... Card did not respond to voltage select! : -110
** Bad device specification mmc 0 **
Unknown monitor
Unknown monitor
In:    serial,usbkbd
Out:   serial,vidconsole
Err:   serial,vidconsole
Net:
Error: ethernet@1c0b000 No valid MAC address found.
No ethernet found.

starting USB...
Bus usb@1c14000: USB EHCI 0.00
Bus usb@1c14400: USB OHCI 0.0
Bus usb@1c1c000: USB EHCI 0.00
Bus usb@1c1c400: USB OHCI 0.0
scanning bus usb@1c14000 for devices... 1 USB Device(s) found
scanning bus usb@1c14400 for devices... 1 USB Device(s) found
scanning bus usb@1c1c000 for devices... 1 USB Device(s) found
scanning bus usb@1c1c400 for devices... 1 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot:  0

U-Boot needs to be stopped and then following commands need to be entered so Linux kernel is started with the device tree.

# Run Linux kernel from U-Boot
bootz 42000000 - 43000000
Kernel image @ 0x42000000 [ 0x000000 - 0xa7b200 ]
## Flattened Device Tree blob at 43000000
   Booting using the fdt blob at 0x43000000
Working FDT set to 43000000
   Loading Device Tree to 49ff7000, end 49fffb4d ... OK
Working FDT set to 49ff7000
DE is present but not probed

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 6.8.1 (straxy@DESKTOP-CTRN0PE) (arm-none-linux-gnueabihf-gcc (Arm GNU Toolchain 13.2.rel1 (Build arm-13.7)) 13.2.1 20231009, GNU ld (Arm GNU Toolchain 13.2.rel1 (Build arm-13.7)) 2.41.0.20231009) #1 SMP Sat Mar 23 21:21:00 CET 2024

[snip]

[    1.709098] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

The first command sets boot arguments that are passed to the Linux kernel. In this case, the only thing that needs to be configured is the device that is used for serial console, and that is the ttyAMA0, or the UART0 port.

The second command start the boot of the Linux kernel, where parameters that are passed are address of zImage kernel image, address of init ramdisk (in this case - since it is not used) and the address where device tree blob is placed.

After executing these commands, the Linux kernel will panic since there is no root filesystem, which is expected.

Simple “ramdisk”

What is actually a root filesystem? It is a set of files and directories organized in a certain way. Those files and directories can be on a physical medium (HDD, SD card, eMMC, Flash memory), a remote location (NFS boot), but also can be executed from RAM in case the init ramdisk/ramfs is used.

In all cases, the kernel is looking for an ‘init’ file which is the first one that is executed. So, by supplying an ‘init’ file we can give the kernel a reason not to panic.

The simplest way to create this ‘init’ file, without building a full-blown root filesystem, is to create a ‘Hello, world!’ application and link it statically. The application should write directly to registers (we will use UART peripheral so we can get some messages) and will be used only for the demonstration.

Hello, world

A classic ‘Hello, world!’ C application can be used:

/* hello.c */
#include <stdio.h>

void main()
{
    printf("Hello, world!\n");
    while(1);
}

The code can be compiled into a static binary using:

# Compile 'Hello, world!'
arm-none-linux-gnueabihf-gcc -static hello.c -o hello

The init ramdisk that can be used with U-Boot can be created using:

# Create ramdisk
echo hello | cpio -o -H newc > initrd
gzip initrd
mkimage -A arm -O linux -T ramdisk -d initrd.gz uRamdisk

The command that can be used to run U-Boot inside QEMU, with loading Linux kernel image, device tree blob and uRamdisk at the appropriate addresses is (do not forget to source the $PROJ_DIR/env.sh file beforehand)

# Run U-Boot and Linux kernel in QEMU with uRamdisk
qemu-system-arm -M cubieboard -m 1G -kernel $UBOOT -nographic \
                -device loader,file=$ZIMAGE,addr=0x42000000,force-raw=on \
                -device loader,file=$DTB,addr=0x43000000,force-raw=on \
                -device loader,file=uRamdisk,addr=0x43400000,force-raw=on
U-Boot 2024.01 (Mar 23 2024 - 20:40:51 +0100) Allwinner Technology

CPU:   Allwinner A10 (SUN4I)
Model: Cubietech Cubieboard
DRAM:  1 GiB
Core:  74 devices, 24 uclasses, devicetree: embed
WDT:   Not starting watchdog@1c20c90
MMC:   mmc@1c0f000: 0
Loading Environment from FAT... Card did not respond to voltage select! : -110
** Bad device specification mmc 0 **
Unknown monitor
Unknown monitor
In:    serial,usbkbd
Out:   serial,vidconsole
Err:   serial,vidconsole
Net:
Error: ethernet@1c0b000 No valid MAC address found.
No ethernet found.

starting USB...
Bus usb@1c14000: USB EHCI 0.00
Bus usb@1c14400: USB OHCI 0.0
Bus usb@1c1c000: USB EHCI 0.00
Bus usb@1c1c400: USB OHCI 0.0
scanning bus usb@1c14000 for devices... 1 USB Device(s) found
scanning bus usb@1c14400 for devices... 1 USB Device(s) found
scanning bus usb@1c1c000 for devices... 1 USB Device(s) found
scanning bus usb@1c1c400 for devices... 1 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot:  0

U-Boot needs to be stopped and then following commands need to be entered so Linux kernel is started with the device tree and uRamdisk.

# Run Linux kernel from U-Boot with ramdisk
setenv bootargs "root=/dev/ram rdinit=/hello console=$console"
bootz 42000000 43400000 43000000
Kernel image @ 0x42000000 [ 0x000000 - 0xa7b200 ]
## Loading init Ramdisk from Legacy Image at 43400000 ...
   Image Name:
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    1109384 Bytes = 1.1 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 43000000
   Booting using the fdt blob at 0x43000000
Working FDT set to 43000000
   Loading Ramdisk to 49ef1000, end 49fffd88 ... OK
   Loading Device Tree to 49ee8000, end 49ef0b4d ... OK
Working FDT set to 49ee8000
DE is present but not probed

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 6.8.1 (straxy@DESKTOP-CTRN0PE) (arm-none-linux-gnueabihf-gcc (Arm GNU Toolchain 13.2.rel1 (Build arm-13.7)) 13.2.1 20231009, GNU ld (Arm GNU Toolchain 13.2.rel1 (Build arm-13.7)) 2.41.0.20231009) #1 SMP Sat Mar 23 21:21:00 CET 2024

[snip]

[    1.737288] Run /hello as init process
Hello, world!

In kernel arguments there are now two additional

  • root=/dev/ram, indicating that root filesystem will be in RAM memory, where init ramdisk is extracted,
  • rdinit=/hello, overriding the default init program that is used from ramdisk so our hello is used.

Once kernel boots, it will run the hello application and print "Hello, world!".

Summary

In this post the basic steps building U-Boot and Linux kernel were covered. This is still far from the actual use-case for an embedded Linux system, since root filesystem is missing.

The root filesystem will be covered in the next post, together with steps for running bootloader and Linux kernel from different mediums, which will be similar to the way and embedded Linux is used on real development boards.

Subscribe
If you would like to get information as soon as new content is published, please subscribe to the "MistraSolutions newsletter".
    Built with Kit

    Similar Posts

    Yocto for Cubieboard in QEMU - Step-by-step

    Yocto for Cubieboard in QEMU - Step-by-step

    Creating Custom QEMU Memory Mapped Device

    Creating Custom QEMU Memory Mapped Device