Creating Custom QEMU Memory Mapped Device

This is part 1 of the Linux device driver development post series.

In this post we will cover the following things

• Designing custom memory-mapped device in QEMU
  • Register map
  • Fitting into Cubieboard memory map
• QEMU implementation
  • Register mapping
  • QEMU timers
  • QEMU interrupt handling
• Integrating with board file and build system
• Testing developed peripheral
• Summary

Designing custom memory-mapped device in QEMU

QEMU can emulate many different peripherals. More importantly, it is possible to create new peripherals that are emulated.

That can be particularly useful when someone wants to learn Linux device driver development, since that peripheral will be unique, and it would be impossible to find an already existing driver.

In this post we will develop a new, custom memory-mapped peripheral for QEMU.

The peripheral will have several 32-bit registers and ability to generate interrupts.

It will function as an 4-digit BCD counter. It can be enabled or disabled, and generate interrupts on each count. It will also have a register for initial count value and possibility to select the counting frequency.

In next posts we will develop Linux device driver that will enable access to this peripheral and userspace application that will use the interfaces provided by the driver to interact with the peripheral.

Register map

Register map for the custom peripheral is shown in the following table

Bit values of CONFIG register are shown in the following table

Bit values of STATUS register are shown in the following table

Bit values of INITVAL register are shown in the following table

Bit values of DATA register are shown in the following table

Fitting into Cubieboard memory map

In order to instantiate and use developed memory-mapped component, we need to integrate it into the Cubieboard memory map and connect it to interrupt controller.

Looking at the memory map for Allwinner-A10, which is found on Cubieboard, there are several regions that are unused, where we can place our custom component. In this example, base address 0x01c2a800 is chosen.

The interrupt line from the custom component has to be connected to the interrupt controller. According to the Allwinner-A10 User Manual, heading “11.2 Interrupt source”, there are several “reserved” interrupt lines which can be used. In this example, interrupt line 36 will be used.

QEMU implementation

Details of implementation of a new memory-mapped device in QEMU will be shown in this section. Main points will be displayed, so someone can use it as instructions for creating a new device.

The device will be described by it’s registers and added to the Allwinner-A10 memory map.

This device is added to the Allwinner-A10 memory map and not to the Cubieboard, since it is an internal memory-mapped peripheral.

Register mapping

QEMU has a very simple and verbose way of describing registers of a component.

For each register, it’s offset from the base address is specified, as well as individual bit-fields that make up the register. Additionally, the access permissions can be specified for each bit field, as well as callback functions that are executed before or after accessing the register.

Register description for our custom component would look like

REG32(CTRL, 0x00)
    FIELD(CTRL,     EN,     0,  1)      /* component enable */
    FIELD(CTRL,     IEN,    1,  1)      /* interrupt enable */
    FIELD(CTRL,     FREQ,   2,  1)      /* sampling frequency setting */

REG32(STATUS, 0x04)
    FIELD(STATUS,   INITW,  0,  1)      /* initial value warning */
    FIELD(STATUS,   IFG,    1,  1)      /* interrupt flag */

REG32(INITVAL, 0x08)
    FIELD(INITVAL,  VALUE,  0,  16)     /* initial counter value */

REG32(DATA, 0x0C)
    FIELD(DATA,     SAMPLE, 0,  16)     /* current value */

The previous code excerpt defines that we have four 32-bit registers called CTRL (offset 0x0 from base address), STATUS (offset 0x4), INITVAL (offset 0x8), and DATA (offset 0xC).

If we look at the register CTRL, it has three bit-fields that are used: EN (bit position 0, size 1 bit), IEN (bit position 1, size 1 bit), FREQ (bit position 2, size 1 bit). Similar goes for the other registers.

The details of the REG32 and FIELD macros can be found in hw/registerfields.h include file. Thing to keep in mind is that these macros create additional macros (mask, shift, etc.) which we can use later for accessing individual registers and bit-fields.

In order to specify actions that are performed when someone tries to access these registers, the read and write functions need to be implemented. Those functions receive offset from the base address as one of the parameters, and based on that offset the corresponding register functionality is implemented.

For most of the registers, the only functionality that is needed is returning the stored value, or writing a new value to the storage. However, for some registers there will be side-effects that will modify the value to be written (think like “write 1 to clear”, or “read-only” bit-values) and/or affect the state of other registers as well (e.g. setting a bit in status register based on setting in control register).

The following code excerpt represents the read function for our custom peripheral. Since none of the read actions have side-effects, the value stored in the register is returned.

static uint64_t mm_sens_read(void *opaque, hwaddr offset, unsigned size) {
    const MMSensorState *s = MM_SENS(opaque);
    const uint32_t idx = REG_INDEX(offset);

    switch (offset) {
    case A_CTRL:
    case A_STATUS:
    case A_INITVAL:
    case A_DATA:
        break;
    case 0x010 ... MM_SENS_IOSIZE:
        qemu_log_mask(LOG_GUEST_ERROR, "%s: out-of-bounds offset 0x%04x\n",
                      __func__, (uint32_t)offset);
        return 0;
    default:
        qemu_log_mask(LOG_UNIMP, "%s: unimplemented read offset 0x%04x\n",
                      __func__, (uint32_t)offset);
        return 0;
    }

    return s->regs[idx];
}

Unlike reading, writing to registers has side-effects, and for those the pre_write functions are used.

static void mm_sens_write(void *opaque, hwaddr offset,
                          uint64_t val, unsigned size) {
    MMSensorState *s = MM_SENS(opaque);
    const uint32_t idx = REG_INDEX(offset);

    switch (offset) {
    case A_CTRL:
        r_ctrl_pre_write(s, val);
        break;
    case A_STATUS:
        trace_mm_sens_status_post_write(val);
        mm_sens_update_irq(s);
        /* STATUS.INITW should not be affected by written value */
        val = val & (~R_STATUS_INITW_MASK);
        val |= s->regs[R_STATUS] & R_STATUS_INITW_MASK;
        break;
    case A_INITVAL:
        r_initval_pre_write(s, val);
        break;
    case A_DATA:
        /* Data is read-only register. */
        return;
    case 0x010 ... MM_SENS_IOSIZE:
        qemu_log_mask(LOG_GUEST_ERROR, "%s: out-of-bounds offset 0x%04x\n",
                      __func__, (uint32_t)offset);
        break;
    default:
        qemu_log_mask(LOG_UNIMP, "%s: unimplemented write offset 0x%04x\n",
                      __func__, (uint32_t)offset);
        break;
    }

    s->regs[idx] = (uint32_t)val;
}

The pre_write functions in the previous code block need to perform certain actions based on the values that are written to the registers.

For instance, after CTRL register bit EN bit is set to 1, the DATA values should start incrementing. Or, after bit FREQ has changed, the frequency of incrementing DATA register should change.

Before we go into details of pre_write functions, it would be useful to first explain how is DATA register periodically incremented, as well as how are interrupts implemented in QEMU code.

QEMU timers

QEMU uses timers to enable periodical execution. Timers have a simple API (described in hw/ptimer.h) which we will go over in this section.

The goal is to increment value of DATA register at two different frequencies: normal (1 Hz) and fast (2 Hz). The value should be incremented only when bit EN in CTRL register is set to 1. The value should also be incremented as BCD value, so each nibble should have values only in the range of 0-9.

Based on the requirements, following ptimer functions are of interest

• ptimer_init - create timer object and define callback that is executed when timer period expires
• ptimer_set_freq - set timer reload frequency in Hz
• ptimer_run - start timer and select whether continuous or one-shot mode is used
• ptimer_stop - stop timer

Since CTRL register has several bit-fields with different functionalities, the post_write function for CTRL has two parts.

In the first part the bit-field FREQ is handled so after every write to the CTRL register a check is made if the value of FREQ bit has changed, and if it has, timer frequency is updated.

// first part, FREQ handling
...
    new_sfreq = (s->regs[R_CTRL] & R_CTRL_FREQ_MASK) >> R_CTRL_FREQ_SHIFT;

    if (new_sfreq != s->sampling_frequency) {
        s->sampling_frequency = new_sfreq;
        switch (s->sampling_frequency) {
            case FREQ_NORMAL:
                ptimer_set_freq(s->timer, DATA_UPDATE_NORMAL_FREQ);
                break;
            case FREQ_FAST:
                ptimer_set_freq(s->timer, DATA_UPDATE_FAST_FREQ);
                break;
            default:
                trace_mm_sens_unknown_frequency(s->sampling_frequency);
                break;
        }
    }
...

In the second part bit EN is handled and timer is started/stopped if EN bit is set/reset. Also, if the initial value written to the INITVAL register is valid, it will be loaded to the DATA register when EN is set.

Additionally, if timer is enabled and IEN bit is also set, then evaluation of interrupt generation condition is made (more on interrupts in next subsection).

// second part, EN/IEN handling
...
    if (val & R_CTRL_EN_MASK) {
        /* start timer if not started. */
        if (!(s->regs[R_CTRL] & R_CTRL_EN_MASK)) {
            ptimer_run(s->timer, 0);

            /* load initial value to DATA register. */
            if (s->regs[R_STATUS] & R_STATUS_INITW_MASK) {
                s->regs[R_DATA] = 0;
            } else {
                s->regs[R_DATA] = s->regs[R_INITVAL];
            }
        }

        if (val & R_CTRL_IEN_MASK) {
            /* check if alarm should be triggered */
            mm_sens_update_irq(s);
        }
    } else {
        /* stop timer */
        ptimer_stop(s->timer);
    }
...

Increments of DATA register are implemented in the timer callback in the following manner

// DATA incrementing
static void mm_sens_update_data(void *opaque)
{
    MMSensor *s = MM_SENS(opaque);

    s->regs[R_DATA] = s->regs[R_DATA] + 1;
    if ((s->regs[R_DATA] & 0x000fu) > 0x0009u) {
        s->regs[R_DATA] += 0x0006u;
        if ((s->regs[R_DATA] & 0x00f0u) > 0x0090u) {
            s->regs[R_DATA] += 0x0060u;
            if ((s->regs[R_DATA] & 0x0f00u) > 0x0900u) {
                s->regs[R_DATA] += 0x0600u;
                if ((s->regs[R_DATA] & 0xf000u) > 0x9000u) {
                    s->regs[R_DATA] += 0x6000u;
                }
            }
        }
    }

    s->regs[R_STATUS] |= R_STATUS_IFG_MASK;

    mm_sens_update_irq(s);
}

This way the BCD requirement is met and counting will look like in the following diagram

Finally, the update of the initial value is handled so check is made on every write if the value is BCD, meaning every nibble is in the range 0-9.

static void r_initval_pre_write(MMSensorState *s, uint64_t val)
{
    uint8_t is_wrong = 0;
    uint32_t bcd_val = val;

    while (bcd_val > 0) {
        if ((bcd_val & 0x0000000fu) > 9) {
            is_wrong = 1;
            break;
        }
        bcd_val >>= 4;
    }

    trace_mm_sens_initval(val, is_wrong);

    s->regs[R_STATUS] &= ~R_STATUS_INITW_MASK;
    s->regs[R_STATUS] |= is_wrong << R_STATUS_INITW_SHIFT;
}

QEMU interrupt handling

Interrupt handling in peripheral in QEMU is performed using the qemu_set_irq function. The function receives an additional parameter which indicates whether interrupt is pending or not. If interrupt is pending (and is not masked in the interrupt controller) it will be raised to the CPU.

In the case of our peripheral, interrupt is pending if both bit IFG in STATUS register and bit IEN in CTRL register are set. This condition has to be checked every time a change happens to any of these two registers, so there is a function that can be reused.

// IRQ handling
static void mm_sens_update_irq(MMSensor *s)
{
    bool pending = s->regs[R_CTRL] & s->regs[R_STATUS] & R_CTRL_IEN_MASK;

    qemu_set_irq(s->irq, pending);
}

Integrating with board file and build system

In order to use the custom memory mapped peripheral, it must be ‘placed’ in the memory map of the emulated board. Since we are using Cubieboard with Allwinner-A10, then it’s description must be updated.

Luckily, this is done with the following lines, where we can see chosen base address (0x01c2a800) and interrupt number (36).

// Addition in the static void aw_a10_init(Object *obj)
object_initialize_child(obj, "mmsens", &s->mmsens, TYPE_MM_SENS);

// Addition in the static void aw_a10_realize(DeviceState *dev, Error **errp)
sysbus_realize(SYS_BUS_DEVICE(&s->mmsens), &error_fatal);
sysbus_mmio_map(SYS_BUS_DEVICE(&s->mmsens), 0, 0x01c2a800);
sysbus_connect_irq(SYS_BUS_DEVICE(&s->mmsens), 0, qdev_get_gpio_in(dev, 36));

Since build system uses meson and ninja, the new component file is added to the build system in the following manner

system_ss.add(files('mmsens.c'))

The patch file with implementation of the custom component is available in github. The main details of our custom component were explained in previous sections. However, there are standard QEMU structures that also must be used in order to describe VMState, as well as initialization and those can be reused from the patch.

Patch is applied to the QEMU source tree with following command

cd $QEMU_SRC
patch -p1 < BCD-memory-mapped.patch

After the patch is applied, QEMU must be rebuilt.

Testing developed peripheral

Testing peripheral without appropriate Linux device driver is a bit harder, but not impossible.

We can use the trace messages from the memory-mapped component. In order to have trace prints shown, QEMU has to be started with extra option --trace mm_sens*. This way all debug prints from the component will be visible.

U-Boot has integrated commands for reading and writing to memory addresses, so we can use it to try to enable the component, interrupt generation and read current data value.

After QEMU is started with

# Run QEMU with SD card and networking
$ qemu-system-arm -M cubieboard -m 1G \
                  -drive file=sd.img,format=raw,if=sd \
                  -net nic -net tap,ifname=qemu-tap0,script=no \
                  -nographic \
                  --trace mm_sens*

U-Boot prompt should be reached by pressing a key.

Following commands are available

• md <addr> - read data from address addr
• mw <addr> <val> - write val to address addr

We can first try reading all of the registers

# Read values from registers
md 0x01c2a800 4

01c2a800: 00000000 00000000 00000000 00000000  ................

In order to enable peripheral, so DATA value starts incrementing, we can write 1 to EN bit in CTRL register. If we read DATA register afterwards, we can see that the values are changing.

# Enable component
mw 0x01c2a800 1

mm_sens_ctrl_post_write Wrote 0x1 to CTRL

The message above is trace log printed from QEMU.

md 0x01c2a80c 1

01c2a80c: 00000001                             ....

md 0x01c2a80c 1

01c2a80c: 00000002                             ....

We can also check that the IFG in STATUS register is set. However, interrupt is not triggered since handling of this interrupt is not implemented in U-Boot, which is expected. We will implement interrupt handling in the next blog post, when we develop the Linux device driver for this component.

Summary

In this blog post the process of developing a custom memory-mapped peripheral for QEMU is shown. Main details are described and the complete patch is available with full details of the component.

Using this process many different components can be implemented.

In the next blog post we will go through the process of development of Linux device driver for this component.