Qemu Networking Investigation - On The Shoulders Of Giants

If you create a virtual disk, install your favourite distribution on it, and want to run it with qemu without having to hard-wire a static IP inside the image itself then this post provides you with some things to consider.

In previous posts I looked at how The Yocto Project is able to accomplish these goals. It does so by being able to supply a kernel and a kernel append, separate from the filesystem image itself. An IP address can be provided in the kernel append, which allows you to specify a static IP... in a flexible way :-)

When you create a virtual machine of, say, your favourite distribution, the installation puts the VM's kernel inside the filesystem, which means it's not available outside the image for you to use the kernel append trick to specify an IP address.

A flexible way for a virtual machine to handle networking is to use dhcp. This requires a dhcp server to be available somewhere on the network which can assign an IP to whoever asks. dnsmasq provides dhcp server capabilities and provides lots of configurability. As I noted in previous blog posts, since we already have a mechanism for running up and down networking-related scripts before and after running a qemu image, there is no reason why we can't start and stop an appropriately configured dhcp server to satisfy our virtual machine. We just want to make sure the dhcp server is only listening on the virtual interface and not interfering with any other interface, or with any other dhcp server running on the network.

Building on the steps the scripts from The Yocto Project use, we can simply add a line to startup dnsmasq and configure it so that it only listens to the virtual tap device which has been setup (by the rest of the up script) for the virtual machine we are bringing up:

dnsmasq \
        --strict-order \
        --except-interface=lo \
        --interface=$TAPDEV \
        --listen-address=$HOSTIP \
        --bind-interfaces \
        -d \
        -q \
        --dhcp-range=${TAPDEV},${IPBASE}5,${IPBASE}20,255.255.255.0,${IPBASE}255 \
        --conf-file="" \
        > dnsmasq.log 2>&1 &

The really hard part was the --dhcp-range option. Originally I had only specified

--dhcp-range=${TAPDEV},${IPBASE}5,${IPBASE}20

which kept leading to the following error message:

dnsmasq-dhcp no address range available for dhcp request via tap0

Wow, how frustrating was that?! As it turns out, if you don't explicitly specify the mask, it assumes 255.255.255.255, leaving absolutely no space whatsoever from which to generate an address!

This is really the only tweak to The Yocto Project's qemu scripts you need in order to run an image in qemu without having to hard-code an IP address and allowing you to access services running on the target from the host.

Before your image is started:

• a virtual tap interface is created
• various ifconfig-fu is used to setup the host's side of this interface and to setup the correct routing
• iptables is used to enable NAT on the VM's network interface
• dnsmasq is run which can provide an IP of your choosing to the VM (provided your image is configured to use dhcp to obtain its IP address)

You can then run multiple images on various virtual tap interfaces without the worry of an IP collision (provided you change the dhcp server range for each invocation) and you never need to make any changes to the image itself in order to accomplish this goal.