psplash Improvements for OpenEmbedded/Yocto

The psplash project uses fbdev graphics to show a logo on a screen during bootup and shutdown. It was started in 2006 and is meant for embedded systems. It is integrated into OpenEmbedded/Yocto quite well; simply add "splash" [NOTE: not "psplash"] to IMAGE_FEATURES and you're on your way.

IMAGE_FEATURES += "splash"

Of course there are caveats. For example the psplash program is run by the init system. On most embedded systems this means that it only runs after the bootloader, and only once the kernel gets to the point of starting userspace. Another caveat is that your kernel config needs to include (y) the appropriate fbdev driver into the kernel (and not as a module) otherwise by the time the module is loaded, psplash will be done its work.

By default, an oecore build will have an OpenEmbedded logo; a poky build uses a Yocto Project logo. Layers are free to override the image with whatever logo they want. meta-raspberrypi provides a good example of this.

While I was playing around with psplash the other day, I noticed something peculiar. With qemu images everything works fine, if the init system is systemd everything is fine again. But on my raspberrypi build with sysvinit as the init system, psplash would fail to run on the very first boot. For the first shutdown, and every bootup and shutdown thereafter it works just fine. So why not on the first boot? Note also that currently sysvinit is still the default init system of an OE/Yocto build.

On a real system with real hardware, on the very first boot the filesystem is initially mounted read-only. Only later is the filesystem remounted read-write (and is forevermore read-write). One of the functions of the psplash program is to display a progress bar to give users a vague idea of the progress of the bootup. In order to communicate the progress to the psplash program, when it starts up the psplash program attempts to use, or create if it doesn't exist, a fifo. Processes that want to update the progress send text messages to the psplash program over its fifo.

Due to the fact that the psplash program is one of the very first things started by the init system, at the point where psplash is started on the very first boot, the filesystem is still mounted read-only. The psplash program would try to use a fifo if one existed, but prior to my investigation it isn't easy to place a fifo into a image that is being created. Since there is no pre-existing fifo the psplash program tries to create one. Since the filesystem is currently read-only, it fails at doing so and the psplash program terminates with error.

At some point in the first bootup the filesystem is re-mounted read-write, so all subsequent shutdowns and boots succeed at running psplash.

One of the unfortunate things about missing out on the first boot's psplash is due to the fact that the first boot of most images can often take longer than subsequent boots. Often times an image needs to do some housekeeping chores on the very first boot, but not on any others. For example: unique keys might need to be generated, post-install scripts might need to be run, etc. So having psplash work for all boots is a worthy goal.

My first thought was to get psplash to create its fifo in a part of the filesystem that is read-write from the start. It turns out that the root of the filesystem (i.e. /) is read-write from the start. Reviewers of that patch, however, weren't keen on the idea of messing up the root of the filesystem.

My next approach was to consider what would be involved in trying to add a fifo to an image that is being created. On the one hand, it's not hard at all: simply DEPEND on coreutils-native and call mkfifo to create the fifo in the staging area. In practice, however, I ran into a snag. The build ends up hanging forever until you kill it explicitly.

Turns out, when you perform a build with OpenEmbedded/Yocto, one of the great benefits is that the build includes a number of post-build steps that are run to check over various parts of the system for sanity. These checks and their logic are part of the insane.bbclass. As it turns out, one such check is to look for a shebang in the first line of all the files included in the image. The point of this check is to make sure that anything that is being shebanged (e.g. sh, bash, perl, python) is included in the runtime on-target image. Therefore the first line of every object in the image is checked. Unfortunately if the object happens to be a fifo, reading from it will hang forever waiting for data to appear.

Looking through the code of that bbclass, it wasn't all that hard to find the one and only case where a sanity check reads through the contents of all files. Adding an extra check to make sure that any object whose contents will be examined is not a fifo is all that was required.

Now that it was possible to create an image with an already-existing fifo, it was a simple matter of updating a recipe's install process to add a fifo to the image and point psplash to it.

Turns out others have noticed that adding fifos to an image isn't easily possible, so a bug was filed. Thankfully Richard Purdie remembered seeing the bug and made me aware that in my quest to get psplash working better, I was also helping to close a bug for the project! (yeah!!) Randy MacLeod had suggested that, as part of fixing the bug, an automated unit test should be added to make sure we don't accidentally lose the ability to add fifos to an image. A little bit of back-and-forth with Richard over IRC pointed me in the right direction of how to run the existing tests, and where to look to add a new fifo test.

While I was working on this project I had the opportunity to look through the psplash code itself. One thing I noticed is that in addition to sending messages via the fifo to update the progress, there is another message that can be sent to specify a text string to print immediately above the progress bar. Turns out there's an invisible text field just above the progress bar one can use to show messages to the user. I thought it would be a good addition to have the name of the current boot script displayed above the progress bar. In this way a user can see exactly what is running.

When a system is booting, the progress bar doesn't proceed smoothly; instead it updates in spurts and jumps. Sometimes the progress bar seemingly stops for a noticeable period of time. Showing the user what's currently running lets them know the reason for any pauses and provides for, in my opinion, a better user experience. Therefore, as part of my updates, I also added the plumbing necessary for the sysvinit system to provide not only a progress bar indication of the progress, but also a textual description of which startup/shutdown script is currently running.

Since, up until this point, showing a textual description of the progress hadn't been used, the feedback I was provided suggested that this new feature be added with a knob to turn it on and off, and that it should be off by default. Therefore should you want to enable it in your builds you'll need to:

PACKAGECONFIG_pn-sysvinit = "psplash-text-updates"

I've tried to capture the changes in the following video. Unfortunately the camera struggles to focus, so it's not the best video you'll see. Hopefully it conveys the gist.

The video shows the very first boot of a rasberrypi system. After the system finishes booting up, I then reboot the device. Notice how the sshd module takes a noticeable amount of time to run on the first boot, but is finished almost instantly the second time around.