DoodleTs: userland graphics with OpenEmbedded/Yocto

When the RaspberryPi was first released (Apr 2012), support for its GPU was provided by a binary blob that was accessible via a library called "userland". Originally this glue library was supplied in binary format, but on Oct 24, 2012 the sources for the userland glue library were made available. The userland library exposes support for a number of GPU APIs including: EGL, GLESv2, OpenVG, and others. Although many parts of this graphics stack are open-source, the core GPU code remains closed. Developers can call GLESv2 functions, for example, but aren't able to manage any of the GPU's resources.

Note that this graphics stack (binary blob + userland) is separate from the fully open-source support that has been added to Mesa since release 10.3.

There exist, therefore, two providers of GLES on a RaspberryPi system: the binary blob + userland, and full Mesa with support for vc4.

When using OpenEmbedded/Yocto to put together an image targetting a RaspberryPi device, if the intent is to run GLES applications, one must choose between these two graphics stacks. Otherwise it would be confusing for the build system to know which graphics stack was intended to provide GLES when linking a GLES application (e.g. glmark2).

An application using GLES usually sits atop a large number of libraries such as various X11 libraries, xcb, drm, EGL, GL/GLES, etc. which allow such an application to run in its own window as part of a windowing system. With the blob+userland option, a number of the libraries lower down in the stack are replaced by the dispmanx library. The dispmanx library doesn't sit on top of a window system; it prefers to take over the entire screen. As a result, running a graphics application with the blob+userland requires far fewer packages, doesn't require any x11/weston, and doesn't require any windowing environment.

The original RaspberryPis ran on 32-bit hardware. In an effort to prioritize backwards compatibility, even when the RaspberryPi migrated to 64-bit architectures, the official images continued to be built for 32-bit. As a result, most of the blob+userland code was written for, and assumes, a 32-bit environment.

There isn't very much information available regarding the dispmanx and userland libraries. The userland code comes with a bunch of sample applications that can be optionally built when building the library. A person named Andrew Duncan wrote a bunch of sample dispmanx applications which were published on github in a repository named raspidmx. Also, the popular glmark2 code has an experimental fork with the changes necessary to run on top of dispmanx.

The best-known BSP layer with support for RaspberryPi includes the necessary knobs and buttons to allow you to create an image and select which graphics stack you would like to use. If you choose the blob+userland stack, support is also available for building the optional userland applications, for building raspidmx, as well as a version of glmark2 that runs on dispmanx.

Assuming you're already familiar with most of the basics of setting up an OpenEmbedded/Yocto build, for this test (at a minimum) you'll need:

bitbake
openembedded-core
meta-openembedded
meta-raspberrypi

Once you've cloned those repositories and setup your build environment, in order to use the blob+userland graphics stack as well as build all the sample applications mentioned above, my conf/local.conf has the following changes and additions:

MACHINE = "raspberrypi3"
DISABLE_VC4GRAPHICS = "1"

DISTRO_FEATURES += "x11 dispmanx"
CORE_IMAGE_EXTRA_INSTALL += "glmark2 raspidmx"
PACKAGECONFIG_append_pn-userland = " allapps"
PACKAGECONFIG_pn-glmark2 = "dispmanx"

ENABLE_UART = "1"
GPU_MEM = "512"

NOTE: this is not my entire conf/local.conf, but rather just the things that I added or changed.

I plan to run this image on a Raspberry Pi 3 Model B V1.2, which has a 64-bit SoC, but I'm setting the MACHINE to "raspberrypi3" (i.e. the 32-bit machine)
On line 2 I'm explicitly disabling VC4GRAPHICS (which refers to the fully open-sourced Mesa support), thereby enabling support for the blob+userland graphics stack
Although I'm not building or including x11 support, when building glmark2 the code refers to some X11 headers (which might be an oversight on the part of the glmark2 code), in any case we need to add x11 to the DISTRO_FEATURES
On line 5 I'm adding the glmark2 (GLES) and raspidmx (dispmanx) sample applications to the image
On line 6 I'm enabling the optional build of sample dispmanx samples when building the userland library
On line 7 I'm making sure glmark2 is build with support for dispmanx
I enable the console UART on the Pi since I prefer working with the device over the console rather than plugging in a keyboard and mouse
On line 10 I increase the memory available to the GPU since one of the glmark2 tests fails with the default setting

With my build properly configured, I proceed to build "core-image-minimal" as follows:

Build Configuration:
BB_VERSION           = "1.49.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "opensuseleap-15.2"
TARGET_SYS           = "arm-oe-linux-gnueabi"
MACHINE              = "raspberrypi3"
DISTRO               = "nodistro"
DISTRO_VERSION       = "nodistro.0"
TUNE_FEATURES        = "arm vfp cortexa7 neon vfpv4 thumb callconvention-hard"
TARGET_FPU           = "hard"
meta-raspberrypi     = "master:e4f5c32925fec90ff688e51197cb052fe12af82e"
meta                 = "master:a55b01a3a1faf9a52d7edad074c76327f637aaa2"
meta-oe              = "master:936f2380bb5112721eec2db46eb35b5600ac28de"

Note that bitbake is at checkout: 71aaac9efa69abbf6c27d174e0862644cbf674ef

When my build is done the only packages in my image are:

base-files_3.0.14-r89_raspberrypi3.ipk
base-passwd_3.5.29-r0_cortexa7t2hf-neon-vfpv4.ipk
bash_5.0-r0_cortexa7t2hf-neon-vfpv4.ipk
busybox_1.32.0-r0_cortexa7t2hf-neon-vfpv4.ipk
busybox-syslog_1.32.0-r0_cortexa7t2hf-neon-vfpv4.ipk
busybox-udhcpc_1.32.0-r0_cortexa7t2hf-neon-vfpv4.ipk
eudev_3.2.9-r0_cortexa7t2hf-neon-vfpv4.ipk
glmark2_20201114+0+784aca755a-r0_cortexa7t2hf-neon-vfpv4.ipk
init-ifupdown_1.0-r7_raspberrypi3.ipk
initscripts-functions_1.0-r155_cortexa7t2hf-neon-vfpv4.ipk
initscripts_1.0-r155_cortexa7t2hf-neon-vfpv4.ipk
init-system-helpers-service_1.58-r0_cortexa7t2hf-neon-vfpv4.ipk
kbd_2.3.0-r0_cortexa7t2hf-neon-vfpv4.ipk
keymaps_1.0-r31_raspberrypi3.ipk
ldconfig_2.32-r0_cortexa7t2hf-neon-vfpv4.ipk
libblkid1_2.36-r0_cortexa7t2hf-neon-vfpv4.ipk
libc6_2.32-r0_cortexa7t2hf-neon-vfpv4.ipk
libgcc1_10.2.0-r0_cortexa7t2hf-neon-vfpv4.ipk
libjpeg62_1:2.0.6-r0_cortexa7t2hf-neon-vfpv4.ipk
libkmod2_27-r0_cortexa7t2hf-neon-vfpv4.ipk
libpng16-16_1.6.37-r0_cortexa7t2hf-neon-vfpv4.ipk
libstdc++6_10.2.0-r0_cortexa7t2hf-neon-vfpv4.ipk
libtinfo5_6.2-r0_cortexa7t2hf-neon-vfpv4.ipk
libz1_1.2.11-r0_cortexa7t2hf-neon-vfpv4.ipk
modutils-initscripts_1.0-r7_cortexa7t2hf-neon-vfpv4.ipk
ncurses-terminfo-base_6.2-r0_cortexa7t2hf-neon-vfpv4.ipk
netbase_1:6.2-r0_all.ipk
packagegroup-core-boot_1.0-r17_raspberrypi3.ipk
raspidmx_0.0+git0+e2ee6faa0d-r0_cortexa7t2hf-neon-vfpv4.ipk
run-postinsts_1.0-r10_all.ipk
sysvinit-inittab_2.88dsf-r10_raspberrypi3.ipk
sysvinit-pidof_2.97-r0_cortexa7t2hf-neon-vfpv4.ipk
sysvinit_2.97-r0_cortexa7t2hf-neon-vfpv4.ipk
update-alternatives-opkg_0.4.3-r0_cortexa7t2hf-neon-vfpv4.ipk
update-rc.d_0.8-r0_all.ipk
userland_20201027-r0_cortexa7t2hf-neon-vfpv4.ipk

My entire image size is ~28MB:

-rw-r--r-- 2 trevor users 28M Dec 15 14:23 core-image-minimal-raspberrypi3-20201215192252.rootfs.ext3

Flashing to a µSD card and booting, from the serial console I am able to run the userland sample applications (not all of the sample apps are shown running here, but they all do run):