A minimal system for Kernel testing with QEmu

When I went to Kernel Recipes earlier this year, I watched a very interesting presentation on using QEmu for Kernel development.

A few hours ago, working on Eudyptula challenge, I was getting a bit annoyed :

The nvidia module was not working on my custom kernel, I needed to do a mkinitcpio each time I compiled a kernel based on the Archlinux config, and occasionally my development module would crash the system. Thus I went on a quest to get QEmu working.

The subject is fairly easy, but unfortunately, documentation is sparse, so here’s a Howto that will allow you to get running in just a few minutes.

If you really are in a hurry, jump to the end of this article. I provide the finished scripts to build the system.

Principles

What people usually do when building a custom system is re-use a system, like debootstrap (it builds a debian system inside a directory), or openembedded, buildroot, ptxdist (mostly used in the embedded world). In our case however, we want to do a really really small system, so even those build systems are to much. We want something completely bare.

So what we are going to create is a build script that will :

  • Compile your Kernel and modules
  • Build an initramfs/initrd image, of about 3-4MB containing :
    • busybox
    • your kernel modules
    • a custom init script

Building your kernel

I am not going to explain how to configure your kernel. If you don’t want to worry too much, just reuse your distributions “.config” file. It should be located in /proc/config.gz.

Compiling is easy :

ROOT_DIR=$(pwd)

# adjust accordingly
KERNEL_DIR=$ROOT_DIR/linux

# Compile your kernel
cd $KERNEL_DIR
make
make modules
cd $ROOT_DIR

Building an Initramfs

First a little reminder : what is an initramdisk?

An initramdisk is a (small) CPIO archive that is loaded in ram as a / filesystem.

It is usually used in distros to do a 2-stage boot. The first boot will load the vital modules, mount the real file system (probably from a disk) and start the second stage from the disk.

This allows distros to provide a precompiled linux kernel with everything as modules for everyone (including motherboard drivers), and generate on installation an initrd that provides the modules your specific system will need to do a basic boot (ie: your “/” filesystem, essential motherboard drivers, …).

In our case, we don’t care about that second stage, we just want to boot into a shell.

Installing kernel modules

Make modules_install will install your modules in /lib/modules/<kernel_name>/…

You can just set INSTALL_MOD_PATH, and the modules will be installed in $INSTALL_MOD_PATH/lib/modules/<kernel_name>/… instead

INITRAMFS_DIR=$ROOT_DIR/initramfs

# remove old modules
rm -fr $INITRAMFS_DIR/lib/modules
# install new modules
cd $KERNEL_DIR
make INSTALL_MOD_PATH=$INITRAMFS_DIR modules_install
cd $ROOT_DIR

Installing Busybox

You can just install Busybox on your host system (provided you want to emulate the same architecture), and run

busybox --install $INITRAMFS_DIR/bin/

This will install all binaries that busybox can emulate into /bin of your initramfs. You only need to do this once, so no need to include it in your build script.

If you want, you can compile Busybox with specific options, to tell it you want some special command available. I find the one provided by Archlinux sufficient.

Your Init script

Here things are starting to be interesting. After the kernel boots, it will launch “/init” as process 0. On your distribution, this is probably systemd.

We don’t need something that huge, so we are just going to write our own as a shell script (remember busybox: it provides us with /bin/sh !)

What do you want to do in your init?

Initializing the system

We want to mount our /proc and /sys (add debugfs if you do kernel debugging), and populate /dev a bit (create /dev/zero, ttys, and other required devices)

#!/bin/sh
# Don't forget the shebang on the first line

/bin/mount -t proc none /proc
/bin/mount -t sysfs sysfs /sys

echo "> Populating /dev/"
/bin/mdev -s

Loading modules

Well obviously, modules are not going to load themselves right? Use /bin/modprobe (provided by Busybox)

# ehci = qemu usb 2.0
# uhci = qemu usb 1.1
# add other modules as you see fit
MODULES="ehci-pci ehci-hcd uhci-hcd"

for module in $MODULES; do
    echo "> Loading module $module"
    /bin/modprobe $module
done

Give you control

Let’s not forget the most important part, right? If the init script exited your system would end.

/bin/sh

Generating your Initramfs

We need the cpio command

cd $INITRAMFS_DIR
find . -print0 | cpio --null -ov --format=newc | gzip - > $ROOT_DIR/initramfs.img
cd $ROOT_DIR

This will list all files in your ramfs directory, send them to cpio, and compress the image generated by cpio with gzip.

Launching QEmu

qemu-system-x86_64 \
-m 1024 \
-kernel linux/arch/x86_64/boot/bzImage \
-initrd initramfs.img \
-append 'console=ttyS0' \
-nographic \
-usb \
-device usb-ehci,id=ehci \
-device usb-host,bus=usb-bus.0,vendorid=0x046d,productid=0xc52b \
-device usb-tablet

Let’s inspect each line :

  • -m 1024 will give 1GB to the system. This is definitely not required, ignore it or adapt it as you want
  • -kernel and -initrd are ovious : you want to provide the generated initramfs and kernel
  • -append ‘console=ttyS0’ and -nographic makes QEmu not create a new window, and redirect all the output to the terminal you used to launch it. If you want to access the QEmu console, use the shortcut <Ctl-a c>
  • -usb enables usb. It will create ” usb-bus” hub to connect your USB 1.1 devices to.
  • -device usb-ehci,id=ehci will create a “ehci” hub to connect your USB 2.0 devices to.
  • -device usb-tablet creates a QEmu special “tablet” pointer. It will connect to your ehci hub automatically
  • -device usb-host,bus=usb-bus.0,vendorid=0x046d,productid=0xc52b will pass the control from an usb device connected to your host, to the virtual machine. We provide the vendorid and productid of the device (as returned by lsusb), and tell it to connect to the first port of the usb-bus (USB 1.1) hub.

If your computer is recent enough and provides virtualization instructions, you could add the -enable-kvm option. Without it, the system takes about 4 seconds to boot on my computer.

TL;DR

initrd build script and init

Leave a comment

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: