Linux Device Driver for QEMU custom memory-mapped device

This is part 2 of the Linux device driver development post series.

In the previous post the steps for creating a custom memory-mapped peripheral in QEMU were presented.

In this post following items will be covered

• Developing character device driver for the designed memory-mapped device
  • Platform driver
    • Device Tree description
    • Platform driver for memory-mapped peripheral
  • Chardev operations
  • Sysfs attributes
• Building driver out-of-tree
• Testing driver
  • Data incrementing
  • Frequency change
  • Interrupt generation
• Summary

Developing character device driver for the designed memory-mapped device

The driver for the custom memory-mapped device should provide interface for user space applications to use the custom-memory mapped peripheral. This means that all bit-fields can be read and modified by a user-space application.

The device first needs to be ‘recognized’ by the system for which we will use Device-tree mapping and platform driver structures.

After device is recognized by the system, we need to have some methods to access and modify registers of our custom memory-mapped peripheral, and we will use character device driver structures with sysfs attributes.

Platform driver

Peripherals can be connected to the processor directly (via ‘platform’ bus, like our memory-mapped peripheral) and also via external buses: I2C, SPI, UART, USB, PCI, etc. Some of these buses support dynamic enumeration of devices (USB, PCI), but for others there needs to be a way to let the system know what is present in those buses.

In standard PCs BIOS is in charge of preparing and providing information about present devices to the operating system on boot.

ARM systems do not have BIOS, so the information must be provided in some other way. Initially, it was done by hardcoding details for each board in the Linux architecture specific code, so when board runs it has all of the information about peripherals that it needs.

However, that also meant that two boards with just minor differences could not use the same Linux kernel image.

To avoid that, Device Tree specification is now used. In Device Tree specifics of a device are described: memory regions, interrupt line numbers, DMA channels, as well as key for binding the compatible driver with that device.

This way the device driver gets all relevant information about the way the device is integrated into the system from the Device Tree.

Device Tree description

Device Tree is a tree-like hierarchical description of system devices and buses they are connected to. It is used for describing peripherals that cannot be automatically enumerated.

Device Tree is written in textual form (.dts and .dtsi files, but also C header files can be preprocessed) and need to be translated into binary form (binary blob, .dtb) before they can be used on a board. At boot time Linux kernel (also recent versions of U-Boot) parse the Device Tree blob and try to match corresponding device drivers in order to initialize the system.

Without going into too much details (there are always good materials on Bootlin website), the Device Tree excerpt for our memory-mapped peripheral should look like

  mmsens: mmsens@1c2a800 {
    compatible = "mistra,mmsens";
    reg = <0x01c2a800 0x400>;
    interrupts = <36>;
  };

The compatible string is used by the device driver when probing to match with this device.

Field reg is used to describe memory regions used by the device, in our case it is offset 0x01c2a800 in the memory map. Memory range is 1kB (0x400) long.

The interrupt line that is used is noted in the interrupts field.

Platform driver for memory-mapped peripheral

The platform driver provides callbacks that are called when Device Tree is parsed and device is probed, which are then used to get information about the device. The platform_driver structure has following important fields

• .driver - initialized with driver name and table of compatible strings used for matching driver with the device from the Device Tree
• .probe callback - called when Device Tree is parsed, to try to register device with the driver
• .remove callback - called when device is removed (not particularly interesting for platform drivers) or when driver is removed from the system

In the case of our memory-mapped device, platform_driver structure should look like

static struct platform_driver mmsens_driver = {
  .driver = {
    .name = DRIVER_NAME,
    .of_match_table = mmsens_of_match,
  },
  .probe = mmsens_probe,
  .remove = mmsens_remove,
};

The of_ prefix comes from ‘Open Firmware’, since full name of Device Tree is Open Firmware Device Tree.

Match table should contain the compatible string, which (from Device Tree excerpt above) is selected to be "mistra,mmsens"

static const struct of_device_id mmsens_of_match[] = {
  { .compatible = "mistra,mmsens", },
  { /* end of table */ }
};
MODULE_DEVICE_TABLE(of, mmsens_of_match);

The .probe callback should do the following things:

1. Try to match device from Device Tree with the driver based on the ‘compatible’ string (of_match_node)
2. Try to extract memory regions information from Device Tree (platform_get_resource) and remap memory so it is accessible by the driver (devm_ioremap_resource)
3. Try to extract interrupt lines information from Device Tree (platform_get_irq) and register handler function for that interrupt (devm_request_irq)

Interrupt handler in our case should only clear the IFG flag

static irqreturn_t mmsens_isr(int irq, void *data)
{
  struct mmsens *dev = data;

  pr_info("Interrupt received\n");

  iowrite32(0, dev->base_addr + MMSENS_STATUS_OFFSET);

  return IRQ_HANDLED;
}

In the simplest scenario the .remove callback does not need to do anything, but once we add the character device operations it will change.

Chardev operations

So far, the platform driver only allows us to match the driver with the device when Device Tree description is parsed.

In order to be able to interact with the device and read/write some data to it, we need add another layer, which is character device. The use of character device allows us later on to add more operations, like IOCTL or sysfs attributes, to have even more ways to interact with the device.

Character device operations are executed when character device file (usually under /dev) is accessed, so we need to make sure that information obtained from platform driver framework (base address of the remapped region) can be used within the character device operations. For that purpose, we will create a custom structure which will be stored as private_data and shared between these two frameworks

/**
 * struct mmsens - mmsens device private data structure
 * @base_addr: base address of the device
 * @irq: interrupt for the device
 * @dev: struct device pointer
 * @parent: parent pointer
 * @cdev: struct cdev
 * @devt: dev_t member
 */
struct mmsens {
  void __iomem *base_addr;
  int irq;
  struct device *dev;
  struct device *parent;
  struct cdev cdev;
  dev_t devt;
};

The character device operations structure defines several callbacks

• .open - used to prepare driver structures for accessing the device
• .release - cleanup of operations done in .open
• .read - read operation from the device, usually raw data that is copied to the user space
• .write - write operation to the device, usually raw data that is copied from the user space

In the case of our memory-mapped device, it should look like

static struct file_operations mmsensdev_fops = {
  .owner = THIS_MODULE,
  .open = mmsens_open,
  .release = mmsens_release,
  .read = mmsens_read,
  .write = mmsens_write,
};

The details of the implementation are available in the github repository.

Sysfs attributes

The character device operations allow only reading or writing of raw data to the device. However, since our device has several registers and supports different operations, we need an additional interface to be able to control it.

That can be achieved using IOCTL callback, or by using sysfs attributes (we will use the latter).

The sysfs attributes are created in the /sys directory when the device is created as separate files. Each file can be used according to the way they are specified (read-only, write-only, read-write) and they can be used to access individual bits/registers, or perform specific operations.

In order to be able to use the sysfs attributes, the character device class must be created, and all devices of that class will appear under that directory.

The attributes can be read and/or written. The <operation>_show callback is used when attribute file is read, <operation>_store callback is used when attribute file is written, and attribute is initialized using static DEVICE_ATTR_RW(<operation>); (there are also the _RO and _WO variants).

In the case of our device, following attributes are supported

• DEVICE_ATTR_RO(interrupt) - used to obtain interrupt status, and can be used to poll from user space application (more on that in next post)
• DEVICE_ATTR_RO(initval_warn) - used to read the flag indicating if the initial value is invalid
• DEVICE_ATTR_RW(enable_interrupt) - used to enable/disable interrupt generation
• DEVICE_ATTR_RW(enable) - used to enable/disable device
• DEVICE_ATTR_RW(frequency) - used to select desired sampling frequency
• DEVICE_ATTR_RO(available_frequencies) - used to show available sampling frequencies
• DEVICE_ATTR_RW(initval) - used to access the INITVAL register, to update the initial value
• DEVICE_ATTR_RO(data) - used to show data in BCD format

If we take a look at the data attribute for instance, we can see that inside it reads the register value and returns string with formatted value

static ssize_t data_show(struct device *child, struct device_attribute *attr, char *buf)
{
  struct mmsens *dev = dev_get_drvdata(child);

  u32 data = ioread32(dev->base_addr + MMSENS_DATA_OFFSET);
  data &= DATA_MASK;

  return sprintf(buf, "%04X\n", data);
}

The details of the implementation are available in github repository.

Building driver out-of-tree

The kernel driver can be provided in two ways: as part of the kernel source code, or as an out-of-tree entity. In first case, using the kernel configuration (menuconfig for instance) it can be selected whether driver will be built into the kernel, or as a separate kernel module (.ko extension). In the out-of-tree build, kernel header files are needed and driver can be built as a kernel module file.

In this case, we will be using the out-of-tree approach. If we have the kernel source code in the $KERNEL_PATH, the Makefile for building the kernel module would look like

obj-m := mmsensdrv.o

SRC := $(shell pwd)

all:
  $(MAKE) ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabihf- -C $(KERNEL_PATH) M=$(SRC)

After executing make command, the mmsensdrv.ko file would be available and can be transferred to the SD card and tested.

Testing driver

Once Linux kernel is started inside QEMU, the module can be loaded. After mmsensdrv.ko is copied to the SD card manually, it can me loaded into the kernel with

insmod mmsensdrv.ko

First we can check that mmsensX file exists in the /dev directory

ls /dev/mmsens*

/dev/mmsens0

Reading that file should return value 0 since device must be started manually, by setting the EN bit in CTRL register.

cat /dev/mmsens0

0

Next, we can check that appropriate entries exist in the sysfs

ls /sys/class/mmsens/mmsens0/

available_frequencies  device            frequency     interrupt  uevent
data                   enable            initval       power
dev                    enable_interrupt  initval_warn  subsystem

Finally, we can do the proper testing.

Data incrementing

If we enable the device and try to read data attribute, as well as /dev/mmsens0, we should see that values are changing.

echo 1 > /sys/class/mmsens/mmsens0/enable

mm_sens_ctrl_post_write Wrote 0x1 to CTRL

sleep 10
cat /sys/class/mmsens/mmsens0/data && cat /dev/mmsens0

0010
16

After enabling the device, the data attribute returns the BCD formatted value, while /dev/mmsens0 returns the raw (integer) value, as expected.

Frequency change

The list of available frequencies can be obtained from the available_frequencies attribute and current frequency selection from frequency attribute.

cat /sys/class/mmsens/mmsens0/available_frequencies

normal fast

cat /sys/class/mmsens/mmsens0/frequency

normal

Per specification, normal frequency means that value changes once per second, while fast frequency means that value changes twice per second (every 0.5 seconds).

If we change sampling frequency from normal to fast, we should see that values are changing twice as often.

# Before
pushd /sys/class/mmsens/mmsens0
cat data && sleep 1 && cat data

0081
0082

# Change
echo fast > frequency
# After
cat data && sleep 1 && cat data

0085
0087

# Cleanup
popd

Interrupt generation

Finally, we should check that interrupts are generated properly. However, since we do not have userspace application that would do something useful with those interrupts (that will be done in the next post), we can use output from /proc/interrupts to validate that the interrupt has been generated

# Before enabling interrupt generation
cat /proc/interrupts | grep mmsens

110:          0  sun4i_irq  36 Edge      1c2a800.mmsens

We can enable interrupt by writing 1 to enable_interrupt and number of occurrences (column 1) increases. It is also visible from QEMU debug code that once interrupt is generated, interrupt handler in device driver clears STATUS register, thus acknowledging interrupt.

# After enabling interrupt generation
echo 1 > /sys/class/mmsens/mmsens0/enable_interrupt

mm_sens_update_irq Interrupt pending 1
[  257.548025] Interrupt received
mm_sens_status_post_write Wrote 0x0 to STATUS
mm_sens_update_irq Interrupt pending 0

cat /proc/interrupts | grep mmsens

110:          1  sun4i_irq  36 Edge      1c2a800.mmsens

Summary

In this blog post a simple character device platform driver is presented. The driver allows initialization and bit-field access of memory mapped peripheral.

Device Tree description for the memory-mapped device is also shown, which allows device driver to obtain information about the device that is present.

The driver itself handles interrupt, but we have not gone into processing that event, and if someone would want to use it as it is (for instance with bash script), they could only do a polling approach to handle data.

Next step is to develop a user space application which will be able to initialize device using the driver, as well as receive information about received interrupt and process updated data. This will be done in next blog post in this series.