5 Oct 2020

Pimoroni Automation-HAT Meets OpenEmbedded

Preamble

I found this device, the Automation-HAT by Pimoroni, which I thought might be fun for some HVAC experimenting I'd like to explore in the future. The creators of this Automation-HAT even provide a python library. Since I'm a fan of Yocto/OpenEmbedded (OE) I wanted to explore the steps in getting the Automation-HAT working with my own image instead of using, say, Raspberry Pi OS.

Master or Release?

Choosing between using master or basing on a release isn't easy when writing a blog post. On the one hand using a release can often be "safer", meaning a person stumbling into this post might have a greater chance of following along if everything is based on a "stable" release. But in my experience all software ages badly in blog posts, and working off of master is more fun for me. So master it is!

Getting Setup and First Build

There are lots of "Getting Started" guides for working with The Yocto Project or OpenEmbedded so if the following instructions skip too many steps, feel free to work through some of those other resources in order to get going.

Whenever I work on a technical project, I've always liked the approach of getting something working, and building from there. As such my first goal is to get a very basic image building and working on the device. Then we can start adding to what we know works, in order to build up to our goal.

For this Yocto build I'm going to start with poky. Since I'll be attaching this device to a Raspberry Pi (a Raspberry Pi 3, to be exact) I'll also need the OE BSP layer for that hardware. Create a fresh directory somewhere on your system to work and:

1
2
3
4
5
$ mkdir layers
$ pushd layers
$ git clone git://git.yoctoproject.org/poky
$ git clone git://git.yoctoproject.org/meta-raspberrypi.git
$ popd

In my specific case the HEADs of these repositories are set at:

1
2
poky: 7cad26d585f67fa6bf873b8be361c6335a7db376
meta-raspberrypi: 6f85611576b7ccbfb6012631f741bd1daeffc9c9

Initialize your shell environment to get ready to build:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
$ . layers/poky/oe-init-build-env 
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.

You had no conf/bblayers.conf file. This configuration file has therefore been
created for you with some default values. To add additional metadata layers
into your configuration please add entries to conf/bblayers.conf.

The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
    http://yoctoproject.org/documentation

For more information about OpenEmbedded see their website:
    http://www.openembedded.org/


### Shell environment set up for builds. ###

You can now run 'bitbake <target>'

Common targets are:
    core-image-minimal
    core-image-sato
    meta-toolchain
    meta-ide-support

You can also run generated qemu images with a command like 'runqemu qemux86'

Other commonly useful commands are:
 - 'devtool' and 'recipetool' handle common recipe tasks
 - 'bitbake-layers' handles common layer tasks
 - 'oe-pkgdata-util' handles common target package tasks

Make your build aware of the BSP layer:

1
2
$ bitbake-layers add-layer ../layers/meta-raspberrypi/
NOTE: Starting bitbake server...

When you initialized your shell earlier, it created a basic conf/local.conf template for you. You need to edit this file. The basic config file is filled with comments that can be quite useful. Stripping out the comments and adding/tweaking the build for my purposes I ended up with the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
MACHINE = "raspberrypi3-64"
DL_DIR = "/opt/Downloads"
DISTRO = "poky"

CORE_IMAGE_EXTRA_INSTALL += " \
        ${MACHINE_EXTRA_RRECOMMENDS} \
        wpa-supplicant \
        "

PACKAGECONFIG_append_pn-gdb = " tui"
PACKAGECONFIG_append_pn-gdb-cross-canadian-arm = " tui"

IMAGE_FSTYPES_append = " wic"
IMAGE_FSTYPES_remove = "wic.bz2 tar.bz2"
WARN_QA_append = " version-going-backwards"
ERROR_QA_remove = "version-going-backwards"
BB_DANGLINGAPPENDS_WARNONLY = "yes"
INHERIT += "buildhistory image-buildinfo buildstats-summary"
BUILDHISTORY_COMMIT = "1"

PACKAGE_CLASSES ?= "package_ipk"
SDKMACHINE = "x86_64"
EXTRA_IMAGE_FEATURES ?= "debug-tweaks"
USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS ??= "\
    STOPTASKS,${TMPDIR},1G,100K \
    STOPTASKS,${DL_DIR},1G,100K \
    STOPTASKS,${SSTATE_DIR},1G,100K \
    STOPTASKS,/tmp,100M,100K \
    ABORT,${TMPDIR},100M,1K \
    ABORT,${DL_DIR},100M,1K \
    ABORT,${SSTATE_DIR},100M,1K \
    ABORT,/tmp,10M,1K"
PACKAGECONFIG_append_pn-qemu-system-native = " sdl"
CONF_VERSION = "1"

  • Line 1: The MACHINE is set to "raspberrypi3-64" since that is the target I want from the set of machines that are defined in the meta-raspberrypi BSP.
  • Line 2: The DL_DIR points to a folder on my host machine which will be where all downloads for this build will be placed. You'll need to specify a location on your machine to which you have write access and which is large enough to store these downloads.
  • Line 3: I've chosen the poky DISTRO.
  • Lines 5-8: I want to add a couple things to my image so I include them here (the fact MACHINE_EXTRA_RRECOMMENDS isn't getting added by default to core-image-full-cmdline, I believe, is a bug, but we can work around it like this)
  • Lines 10-11: Whenever I used gdb I like to use its curses-based GUI mode. Therefore I've set some PACKAGECONFIGs to say that whenever gdb is built, please configure it for "tui" mode.
  • Line 13: Flashing wic images are much faster than flashing a non-wic image. Therefore I like to configure my µSD card builds to use wic.
  • Line 14: Since I'm not transferring these images over a network, or saving them for eternity on a drive somewhere, I don't care for them to be compressed; the compression stage at the end of a build just wastes time for me in this case.
  • Lines 15-16: Sometimes when building different revisions of git code, one SHA will be alphanumerically "lower" than a previous (in time) SHA. The build will flag this and error out thinking that you're downgrading a given piece of software. I like to add these lines to my builds to switch this from an error to a warning.
  • Line 17: Sometimes a layer includes metadata that appends to a recipe that doesn't exist anywhere in your build. This line switches this situation from an error to a warning.
  • Lines 18-19: I like using the buildhistory feature, as well as enabling other build metrics.
  • Line 21: I've set the PACKAGE_CLASS to ipk from poky's default of rpm
  • Line 22: I've set the SDKMACHINE to x86_64
  • Lines 23-36: Are taken from the template that is created when the build area is initialized by the oe-init-build-env script.

Now we build:
[NOTE: depending on the characteristics of your host machine and the speed of your connection to the internet, your initial build could take anywhere from 20 minutes to an hour or so]

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
$ bitbake core-image-full-cmdline
Loading cache: 100% |                                                                                                                                                                          | ETA:  --:--:--
Loaded 0 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################################################################| Time: 0:00:09
Parsing of 814 .bb files complete (0 cached, 814 parsed). 1383 targets, 65 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.47.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "opensuseleap-15.1"
TARGET_SYS           = "aarch64-poky-linux"
MACHINE              = "raspberrypi3-64"
DISTRO               = "poky"
DISTRO_VERSION       = "3.1+snapshot-20201017"
TUNE_FEATURES        = "aarch64 armv8a crc cortexa53"
TARGET_FPU           = ""
meta                 
meta-poky            
meta-yocto-bsp       = "master:7cad26d585f67fa6bf873b8be361c6335a7db376"
meta-raspberrypi     = "master:6f85611576b7ccbfb6012631f741bd1daeffc9c9"

Initialising tasks: 100% |######################################################################################################################################################################| Time: 0:00:03
Sstate summary: Wanted 1522 Found 0 Missed 1522 Current 0 (0% match, 0% complete)
NOTE: Executing Tasks
WARNING: linux-raspberrypi-1_5.4.69+gitAUTOINC+5d52d9eea9_31d364af25-r0 do_kernel_configcheck: [kernel config]: specified values did not make it into the kernel's final configuration:

    [NOTE]: 'CONFIG_I2C_BCM2835' last val (m) and .config val (y) do not match
    [INFO]: CONFIG_I2C_BCM2835 : y ## .config: 2783 :configs///defconfig (m) 
    [INFO]: raw config text:

        config I2C_BCM2835
                tristate "Broadcom BCM2835 I2C controller"
                depends on (ARCH_BCM2835 || ARCH_BRCMSTB) && HAS_IOMEM && I2C
                help
                  If you say yes to this option, support will be included for the
                  BCM2835 I2C controller.
          
                  If you don't know what to do here, say N.
          
                  This support is also available as a module.  If so, the module
                  will be called i2c-bcm2835.

        Config 'I2C_BCM2835' has the following Direct dependencies (I2C_BCM2835=y):
                ARCH_BCM2835(=y) || ARCH_BRCMSTB(=n) (=y) && HAS_IOMEM(=y) && I2C(=y)
        Parent dependencies are:
             HAS_IOMEM [y] ARCH_BRCMSTB [n] I2C [y] ARCH_BCM2835 [y]
    [NOTE]: 'CONFIG_DRM' last val (m) and .config val (y) do not match
    [INFO]: CONFIG_DRM : y ## .config: 4017 :configs///defconfig (m) 
    [INFO]: raw config text:

        menuconfig DRM
                tristate "Direct Rendering Manager (XFree86 4.1.0 and higher DRI support)"
                select DRM_PANEL_ORIENTATION_QUIRKS
                select HDMI
                select FB_CMDLINE
                select I2C
               select I2C_ALGOBIT
                select DMA_SHARED_BUFFER
                select SYNC_FILE
                depends on (AGP || AGP = n) && !EMULATED_CMPXCHG && HAS_DMA && HAS_IOMEM
                help
                  Kernel-level support for the Direct Rendering Infrastructure (DRI)
                  introduced in XFree86 4.0. If you say Y here, you need to select
                  the module that's right for your graphics card from the list below.
                  These modules provide support for synchronization, security, and
                  DMA transfers. Please see <http://dri.sourceforge.net/> for more
                  details.  You should also select and configure AGP
                  (/dev/agpgart) support if it is available for your platform.

        Config 'DRM' has the following Direct dependencies (DRM=y):
                AGP(=n) || AGP(=n) = n (=y) && !EMULATED_CMPXCHG(undefined/n) (=y) && HAS_DMA(=y) && HAS_IOMEM(=y)
        Parent dependencies are:
             AGP [n] EMULATED_CMPXCHG [EMULATED_CMPXCHG] HAS_IOMEM [y] HAS_DMA [y]

NOTE: Tasks Summary: Attempted 3929 tasks of which 1 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 189 seconds
NOTE: Build completion summary:
NOTE:   do_populate_sysroot: 0.0% sstate reuse(0 setscene, 282 scratch)
NOTE:   do_deploy_source_date_epoch: 0.0% sstate reuse(0 setscene, 283 scratch)
NOTE:   do_package_qa: 0.0% sstate reuse(0 setscene, 166 scratch)
NOTE:   do_package: 0.0% sstate reuse(0 setscene, 166 scratch)
NOTE:   do_packagedata: 0.0% sstate reuse(0 setscene, 166 scratch)
NOTE:   do_package_write_ipk: 0.0% sstate reuse(0 setscene, 166 scratch)
NOTE:   do_populate_lic: 0.0% sstate reuse(0 setscene, 285 scratch)

Summary: There was 1 WARNING message shown.

Now we write the resulting image to a µSD card. As I said earlier, I like to flash a "wic" image whenever possible. But before we can do that, we need to build the wic-flashing tool (bmap) so we can use it to flash the wic image to the µSD card:
[NOTE: this step only ever needs to be done once in a given build area]

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
$ bitbake bmap-tools-native -caddto_recipe_sysroot
Loading cache: 100% |###########################################################################################################################################################################| Time: 0:00:00
Loaded 1383 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.47.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "aarch64-poky-linux"
MACHINE              = "raspberrypi3-64"
DISTRO               = "poky"
DISTRO_VERSION       = "3.1+snapshot-20201017"
TUNE_FEATURES        = "aarch64 armv8a crc cortexa53"
TARGET_FPU           = ""
meta                 
meta-poky            
meta-yocto-bsp       = "master:7cad26d585f67fa6bf873b8be361c6335a7db376"
meta-raspberrypi     = "master:6f85611576b7ccbfb6012631f741bd1daeffc9c9"

Initialising tasks: 100% |######################################################################################################################################################################| Time: 0:00:00
Sstate summary: Wanted 0 Found 0 Missed 0 Current 56 (0% match, 100% complete)
NOTE: Executing Tasks
NOTE: Tasks Summary: Attempted 253 tasks of which 252 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 1 seconds

Insert the µSD card into your host computer. You'll need to figure out where it appears in your computer so you can specify its location. The method I use is to check the system log. If I've just inserted the device, there will be an entry somewhere at the bottom of the system log indicating the fact a USB device was just inserted and it will specify where it shows up. In my case, when I insert a µSD device into my host via a USB reader, it shows up as /dev/sdk. In the examples below you'll have to replace where I write /dev/sdk with whatever path is valid for you.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
$ oe-run-native bmap-tools-native bmaptool copy tmp/deploy/images/raspberrypi3-64/core-image-full-cmdline-raspberrypi3-64.wic /dev/sdk
Running bitbake -e bmap-tools-native
bmaptool: info: discovered bmap file 'tmp/deploy/images/raspberrypi3-64/core-image-full-cmdline-raspberrypi3-64.wic.bmap'
bmaptool: info: block map format version 2.0
bmaptool: info: 87962 blocks of size 4096 (343.6 MiB), mapped 50393 blocks (196.8 MiB or 57.3%)
bmaptool: info: copying image 'core-image-full-cmdline-raspberrypi3-64.wic' to block device '/dev/sdk' using bmap file 'core-image-full-cmdline-raspberrypi3-64.wic.bmap'
bmaptool: WARNING: failed to disable excessive buffering, expect worse system responsiveness (reason: cannot set max. I/O ratio to 1: [Errno 13] Permission denied: '/sys/dev/block/8:160/bdi/max_ratio')
bmaptool: info: 100% copied
bmaptool: info: synchronizing '/dev/sdk'
bmaptool: info: copying time: 22.6s, copying speed 8.7 MiB/sec

Turning to the target device (i.e. the raspberrypi3) we fit the Automation-HAT to the Pi, wire up the serial console cable, and insert the µSD card.

Here are some pics of my setup:




On the host we watch the system log (again) to see where the serial console cable shows up (e.g. /dev/ttyUSB0) and start a terminal on that device (e.g. using screen, minicom, etc). Now we apply power to the target and… nothing.

As it turns out, we have to explicitly tell the raspberrypi that we want a serial console. This gets configured in the conf/local.conf as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
MACHINE = "raspberrypi3-64"
DL_DIR = "/opt/Downloads"
DISTRO = "poky"

ENABLE_UART = "1"

CORE_IMAGE_EXTRA_INSTALL += " \
        ${MACHINE_EXTRA_RRECOMMENDS} \
        wpa-supplicant \
        "

PACKAGECONFIG_append_pn-gdb = " tui"
PACKAGECONFIG_append_pn-gdb-cross-canadian-arm = " tui"

IMAGE_FSTYPES_append = " wic"
IMAGE_FSTYPES_remove = "wic.bz2 tar.bz2"
WARN_QA_append = " version-going-backwards"
ERROR_QA_remove = "version-going-backwards"
BB_DANGLINGAPPENDS_WARNONLY = "yes"
INHERIT += "buildhistory image-buildinfo buildstats-summary"
BUILDHISTORY_COMMIT = "1"

PACKAGE_CLASSES ?= "package_ipk"
SDKMACHINE = "x86_64"
EXTRA_IMAGE_FEATURES ?= "debug-tweaks"
USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS ??= "\
    STOPTASKS,${TMPDIR},1G,100K \
    STOPTASKS,${DL_DIR},1G,100K \
    STOPTASKS,${SSTATE_DIR},1G,100K \
    STOPTASKS,/tmp,100M,100K \
    ABORT,${TMPDIR},100M,1K \
    ABORT,${DL_DIR},100M,1K \
    ABORT,${SSTATE_DIR},100M,1K \
    ABORT,/tmp,10M,1K"
PACKAGECONFIG_append_pn-qemu-system-native = " sdl"
CONF_VERSION = "1"

Edit your conf/local.conf, rebuild the image, re-flash the image to your µSD card, insert the µSD card into the target, apply power and…

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
[    0.000000] Machine model: Raspberry Pi 3 Model B Rev 1.2
[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] efi: UEFI not found.
[    0.000000] Reserved memory: created CMA memory pool at 0x000000001ec00000, size 256 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] percpu: Embedded 31 pages/cpu s88024 r8192 d30760 u126976
[    0.000000] Detected VIPT I-cache on CPU0
[    0.000000] CPU features: detected: ARM erratum 845719
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[    0.000000] CPU features: detected: ARM erratum 843419
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 238896
[    0.000000] Kernel command line: coherent_pool=1M 8250.nr_uarts=1 snd_bcm2835.enable_compat_alsa=0 snd_bcm2835.enable_hdmi=1 snd_bcm2835.enable_headphones=1 video=Composite-1:720x480@60i vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  dwc_otg.lpm_enable=0 console=ttyS0,115200 root=/dev/mmcblk0p2 rootfstyt
[    0.000000] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.000000] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 669904K/970752K available (9724K kernel code, 1048K rwdata, 3320K rodata, 3072K init, 954K bss, 38704K reserved, 262144K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] ftrace: allocating 30859 entries in 121 pages
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000]  Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] random: get_random_bytes called from start_kernel+0x334/0x4c4 with crng_init=0
[    0.000000] arch_timer: cp15 timer(s) running at 19.20MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x46d987e47, max_idle_ns: 440795202767 ns
[    0.000007] sched_clock: 56 bits at 19MHz, resolution 52ns, wraps every 4398046511078ns
[    0.000254] Console: colour dummy device 80x25
[    0.000323] Calibrating delay loop (skipped), value calculated using timer frequency.. 38.40 BogoMIPS (lpj=19200)
[    0.000350] pid_max: default: 32768 minimum: 301
[    0.000557] LSM: Security Framework initializing
[    0.000827] Mount-cache hash table entries: 2048 (order: 2, 16384 bytes, linear)
[    0.000863] Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes, linear)
[    0.002356] Disabling memory control group subsystem
[    0.004396] ASID allocator initialised with 32768 entries
[    0.004678] rcu: Hierarchical SRCU implementation.
[    0.005371] EFI services will not be available.
[    0.005935] smp: Bringing up secondary CPUs ...
[    0.007351] Detected VIPT I-cache on CPU1
[    0.007419] CPU1: Booted secondary processor 0x0000000001 [0x410fd034]
[    0.009359] Detected VIPT I-cache on CPU2
[    0.009410] CPU2: Booted secondary processor 0x0000000002 [0x410fd034]
[    0.011221] Detected VIPT I-cache on CPU3
[    0.011268] CPU3: Booted secondary processor 0x0000000003 [0x410fd034]
[    0.011885] smp: Brought up 1 node, 4 CPUs
[    0.011937] SMP: Total of 4 processors activated.
[    0.011957] CPU features: detected: 32-bit EL0 Support
[    0.011978] CPU features: detected: CRC32 instructions
[    0.037684] CPU: All CPU(s) started at EL2
[    0.037762] alternatives: patching kernel code
[    0.039583] devtmpfs: initialized
[    0.056561] Enabled cp15_barrier support
[    0.056610] Enabled setend support
[    0.057401] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[    0.057438] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.070451] pinctrl core: initialized pinctrl subsystem
[    0.070834] DMI not present or invalid.
[    0.071361] NET: Registered protocol family 16
[    0.080371] DMA: preallocated 1024 KiB pool for atomic allocations
[    0.080464] audit: initializing netlink subsys (disabled)
[    0.080858] audit: type=2000 audit(0.079:1): state=initialized audit_enabled=0 res=1
[    0.081662] cpuidle: using governor menu
[    0.081950] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.082313] Serial: AMBA PL011 UART driver
[    0.085113] bcm2835-mbox 3f00b880.mailbox: mailbox enabled
[    0.095260] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-10-02 16:09, variant start
[    0.096274] raspberrypi-firmware soc:firmware: Firmware hash is 4672d0274057d726f3a327e2b3fe76f831b811bb
[    0.140930] bcm2835-dma 3f007000.dma: DMA legacy API manager, dmachans=0x1
[    0.143534] SCSI subsystem initialized
[    0.143880] usbcore: registered new interface driver usbfs
[    0.143957] usbcore: registered new interface driver hub
[    0.144141] usbcore: registered new device driver usb
[    0.145929] clocksource: Switched to clocksource arch_sys_counter
[    1.429264] VFS: Disk quotas dquot_6.6.0
[    1.429409] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    1.429643] FS-Cache: Loaded
[    1.430056] CacheFiles: Loaded
[    1.444000] thermal_sys: Registered thermal governor 'step_wise'
[    1.444462] NET: Registered protocol family 2
[    1.445522] tcp_listen_portaddr_hash hash table entries: 512 (order: 1, 8192 bytes, linear)
[    1.445576] TCP established hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    1.445702] TCP bind hash table entries: 8192 (order: 5, 131072 bytes, linear)
[    1.445964] TCP: Hash tables configured (established 8192 bind 8192)
[    1.446214] UDP hash table entries: 512 (order: 2, 16384 bytes, linear)
[    1.446276] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes, linear)
[    1.446645] NET: Registered protocol family 1
[    1.447835] RPC: Registered named UNIX socket transport module.
[    1.447855] RPC: Registered udp transport module.
[    1.447872] RPC: Registered tcp transport module.
[    1.447889] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    1.449642] hw perfevents: enabled with armv8_cortex_a53 PMU driver, 7 counters available
[    1.452049] Initialise system trusted keyrings
[    1.452478] workingset: timestamp_bits=46 max_order=18 bucket_order=0
[    1.466411] FS-Cache: Netfs 'nfs' registered for caching
[    1.467485] NFS: Registering the id_resolver key type
[    1.467593] Key type id_resolver registered
[    1.467612] Key type id_legacy registered
[    1.467643] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    1.469406] Key type asymmetric registered
[    1.469429] Asymmetric key parser 'x509' registered
[    1.469494] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 249)
[    1.469517] io scheduler mq-deadline registered
[    1.469536] io scheduler kyber registered
[    1.478032] Serial: 8250/16550 driver, 1 ports, IRQ sharing enabled
[    1.481110] bcm2835-rng 3f104000.rng: hwrng registered
[    1.481721] vc-mem: phys_addr:0x00000000 mem_base=0x3ec00000 mem_size:0x40000000(1024 MiB)
[    1.482825] gpiomem-bcm2835 3f200000.gpiomem: Initialised: Registers at 0x3f200000
[    1.483358] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    1.499607] brd: module loaded
[    1.517239] loop: module loaded
[    1.518966] Loading iSCSI transport class v2.0-870.
[    1.520136] libphy: Fixed MDIO Bus: probed
[    1.520290] usbcore: registered new interface driver lan78xx
[    1.520369] usbcore: registered new interface driver smsc95xx
[    1.520399] dwc_otg: version 3.00a 10-AUG-2012 (platform bus)
[    2.249221] Core Release: 2.80a
[    2.249243] Setting default values for core params
[    2.249280] Finished setting default values for core params
[    2.449973] Using Buffer DMA mode
[    2.449993] Periodic Transfer Interrupt Enhancement - disabled
[    2.450010] Multiprocessor Interrupt Enhancement - disabled
[    2.450029] OTG VER PARAM: 0, OTG VER FLAG: 0
[    2.450052] Dedicated Tx FIFOs mode
[    2.451562] WARN::dwc_otg_hcd_init:1072: FIQ DMA bounce buffers: virt = ffffffc010251000 dma = 0x00000000ded00000 len=9024
[    2.451597] FIQ FSM acceleration enabled for :
[    2.451597] Non-periodic Split Transactions
[    2.451597] Periodic Split Transactions
[    2.451597] High-Speed Isochronous Endpoints
[    2.451597] Interrupt/Control Split Transaction hack enabled
[    2.451660] WARN::hcd_init_fiq:496: MPHI regs_base at ffffffc01005d000
[    2.451711] dwc_otg 3f980000.usb: DWC OTG Controller
[    2.451755] dwc_otg 3f980000.usb: new USB bus registered, assigned bus number 1
[    2.451830] dwc_otg 3f980000.usb: irq 9, io mem 0x00000000
[    2.451886] Init: Port Power? op_state=1
[    2.451933] Init: Power Port (0)
[    2.452382] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.04
[    2.452407] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    2.452429] usb usb1: Product: DWC OTG Controller
[    2.452451] usb usb1: Manufacturer: Linux 5.4.69-v8 dwc_otg_hcd
[    2.452472] usb usb1: SerialNumber: 3f980000.usb
[    2.453459] hub 1-0:1.0: USB hub found
[    2.453542] hub 1-0:1.0: 1 port detected
[    2.455546] usbcore: registered new interface driver usb-storage
[    2.457384] bcm2835-wdt bcm2835-wdt: Broadcom BCM2835 watchdog timer
[    2.461090] sdhci: Secure Digital Host Controller Interface driver
[    2.461110] sdhci: Copyright(c) Pierre Ossman
[    2.461765] mmc-bcm2835 3f300000.mmcnr: could not get clk, deferring probe
[    2.462539] sdhost-bcm2835 3f202000.mmc: could not get clk, deferring probe
[    2.462837] sdhci-pltfm: SDHCI platform and OF driver helper
[    2.464815] ledtrig-cpu: registered to indicate activity on CPUs
[    2.465298] hidraw: raw HID events driver (C) Jiri Kosina
[    2.465494] usbcore: registered new interface driver usbhid
[    2.465512] usbhid: USB HID core driver
[    2.466076] ashmem: initialized
[    2.466719] Initializing XFRM netlink socket
[    2.466774] NET: Registered protocol family 17
[    2.466984] Key type dns_resolver registered
[    2.467565] registered taskstats version 1
[    2.467596] Loading compiled-in X.509 certificates
[    2.468032] Key type ._fscrypt registered
[    2.468052] Key type .fscrypt registered
[    2.484212] uart-pl011 3f201000.serial: cts_event_workaround enabled
[    2.484349] 3f201000.serial: ttyAMA0 at MMIO 0x3f201000 (irq = 66, base_baud = 0) is a PL011 rev2
[    2.487448] printk: console [ttyS0] disabled
[    2.487542] 3f215040.serial: ttyS0 at MMIO 0x0 (irq = 61, base_baud = 50000000) is a 16550
[    3.473145] printk: console [ttyS0] enabled
[    3.478603] bcm2835-power bcm2835-power: Broadcom BCM2835 power domains driver
[    3.487779] mmc-bcm2835 3f300000.mmcnr: mmc_debug:0 mmc_debug2:0
[    3.493937] mmc-bcm2835 3f300000.mmcnr: DMA channel allocated
[    3.526207] sdhost: log_buf @ (____ptrval____) (f8e51000)
[    3.551613] mmc1: queuing unknown CIS tuple 0x80 (2 bytes)
[    3.558881] mmc1: queuing unknown CIS tuple 0x80 (3 bytes)
[    3.566201] mmc1: queuing unknown CIS tuple 0x80 (3 bytes)
[    3.574850] mmc1: queuing unknown CIS tuple 0x80 (7 bytes)
[    3.585510] Indeed it is in host mode hprt0 = 00021501
[    3.592974] mmc0: sdhost-bcm2835 loaded - DMA enabled (>1)
[    3.602767] of_cfs_init
[    3.605452] of_cfs_init: OK
[    3.610371] Waiting for root device /dev/mmcblk0p2...
[    3.664973] random: fast init done
[    3.729776] mmc1: new high speed SDIO card at address 0001
[    3.747467] mmc0: host does not support reading read-only switch, assuming write-enable
[    3.762580] mmc0: new high speed SDHC card at address aaaa
[    3.769519] mmcblk0: mmc0:aaaa SL16G 14.8 GiB
[    3.774047] usb 1-1: new high-speed USB device number 2 using dwc_otg
[    3.780896] Indeed it is in host mode hprt0 = 00001101
[    3.797189]  mmcblk0: p1 p2
[    3.815991] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
[    3.824333] VFS: Mounted root (ext4 filesystem) readonly on device 179:2.
[    3.835841] devtmpfs: mounted
[    3.848102] Freeing unused kernel memory: 3072K
[    3.852800] Run /sbin/init as init process
INIT: version 2.97 booting
[    3.978789] usb 1-1: New USB device found, idVendor=0424, idProduct=9514, bcdDevice= 2.00
[    3.987214] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    3.995769] hub 1-1:1.0: USB hub found
[    3.999985] hub 1-1:1.0: 5 ports detected
Framebuffer /dev/fb0 not detected
Boot splashscreen disabled
[    4.293048] usb 1-1.1: new high-speed USB device number 3 using dwc_otg
[    4.388594] usb 1-1.1: New USB device found, idVendor=0424, idProduct=ec00, bcdDevice= 2.00
[    4.397199] usb 1-1.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    4.408145] smsc95xx v1.0.6
[    4.460223] smsc95xx 1-1.1:1.0 eth0: register 'smsc95xx' at usb-3f980000.usb-1.1, smsc95xx USB 2.0 Ethernet, b8:27:eb:fd:e3:45
Starting udev
[    4.701746] udevd[119]: starting version 3.2.9
[    4.738668] random: udevd: uninitialized urandom read (16 bytes read)
[    4.746220] random: udevd: uninitialized urandom read (16 bytes read)
[    4.758069] random: udevd: uninitialized urandom read (16 bytes read)
[    4.808414] udevd[120]: starting eudev-3.2.9
[    4.911884] random: crng init done
[    4.915359] random: 3 urandom warning(s) missed due to ratelimiting
[    4.970283] vchiq: module is from the staging directory, the quality is unknown, you have been warned.
[    4.998866] vchiq: vchiq_init_state: slot_zero = 00000000ae5c87c4
[    5.197187] vc_sm_cma: module is from the staging directory, the quality is unknown, you have been warned.
[    5.206689] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[    5.216350] bcm2835_vc_sm_cma_probe: Videocore shared memory driver
[    5.222796] [vc_sm_connected_init]: start
[    5.228112] [vc_sm_connected_init]: installed successfully
[    5.234222] snd_bcm2835: module is from the staging directory, the quality is unknown, you have been warned.
[    5.251418] bcm2835_audio bcm2835_audio: card created with 8 channels
[    5.270497] mc: Linux media interface: v0.10
[    5.330355] videodev: Linux video capture interface: v2.00
[    5.342758] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
[    5.352409] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[    5.361291] cfg80211: failed to load regulatory.db
[    5.374974] vc4-drm soc:gpu: bound 3f600000.firmwarekms (ops vc4_fkms_ops [vc4])
[    5.382958] vc4-drm soc:gpu: bound 3fc00000.v3d (ops vc4_v3d_ops [vc4])
[    5.389812] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[    5.396598] [drm] No driver support for vblank timestamp query.
[    5.402747] [drm] Setting vblank_disable_immediate to false because get_vblank_timestamp == NULL
[    5.413321] [drm] Initialized vc4 0.0.0 20140616 for soc:gpu on minor 0
[    5.478261] bcm2835_mmal_vchiq: module is from the staging directory, the quality is unknown, you have been warned.
[    5.478633] bcm2835_mmal_vchiq: module is from the staging directory, the quality is unknown, you have been warned.
[    5.479494] bcm2835_mmal_vchiq: module is from the staging directory, the quality is unknown, you have been warned.
[    5.481108] Console: switching to colour frame buffer device 90x30
[    5.496212] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43430-sdio for chip BCM43430/1
[    5.496585] usbcore: registered new interface driver brcmfmac
[    5.550829] vc4-drm soc:gpu: fb0: vc4drmfb frame buffer device
[    5.561091] bcm2835_v4l2: module is from the staging directory, the quality is unknown, you have been warned.
[    5.598697] bcm2835_codec: module is from the staging directory, the quality is unknown, you have been warned.
[    5.609303] bcm2835_isp: module is from the staging directory, the quality is unknown, you have been warned.
[    5.624981] bcm2835-codec bcm2835-codec: Device registered as /dev/video10
[    5.625067] bcm2835-codec bcm2835-codec: Loaded V4L2 decode
[    5.639147] bcm2835-isp bcm2835-isp: Device node output[0] registered as /dev/video13
[    5.647793] bcm2835-isp bcm2835-isp: Device node capture[0] registered as /dev/video14
[    5.656481] bcm2835-isp bcm2835-isp: Device node capture[1] registered as /dev/video15
[    5.656588] bcm2835-codec bcm2835-codec: Device registered as /dev/video11
[    5.665123] bcm2835-isp bcm2835-isp: Device node stats[2] registered as /dev/video16
[    5.671659] bcm2835-codec bcm2835-codec: Loaded V4L2 encode
[    5.679485] bcm2835-isp bcm2835-isp: Register output node 0 with media controller
[    5.679507] bcm2835-isp bcm2835-isp: Register capture node 1 with media controller
[    5.679526] bcm2835-isp bcm2835-isp: Register capture node 2 with media controller
[    5.708379] bcm2835-isp bcm2835-isp: Register capture node 3 with media controller
[    5.716471] bcm2835-isp bcm2835-isp: Loaded V4L2 bcm2835-isp
[    5.723339] bcm2835-codec bcm2835-codec: Device registered as /dev/video12
[    5.730762] bcm2835-codec bcm2835-codec: Loaded V4L2 isp
[    5.837048] EXT4-fs (mmcblk0p2): re-mounted. Opts: (null)
[    5.860586] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43430-sdio for chip BCM43430/1
[    5.869718] brcmfmac: brcmf_c_process_clm_blob: no clm_blob available (err=-2), device may have limited channels available
[    5.882177] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM43430/1 wl0: Oct 22 2019 01:59:28 version 7.45.98.94 (r723000 CY) FWID 01-3b33decd
Fri Mar  9 12:34:56 UTC 2018
chmod: cannot access '/var/log/wtmp': No such file or directory
Failed to set mode -0664- for -/var/log/wtmp-.
Configuring packages on first boot....
 (This may take several minutes. Please do not power off the machine.)
Running postinst /etc/ipk-postinsts/000-sysvinit-inittab...
update-rc.d: /etc/init.d/run-postinsts exists during rc.d purge (continuing)
 Removing any system startup links for run-postinsts ...
  /etc/rcS.d/S99run-postinsts
INIT: Entering runlevel: 5
Configuring network interfaces... [    7.341201] smsc95xx 1-1.1:1.0 eth0: hardware isn't capable of remote wakeup
udhcpc: started, v1.32.0
udhcpc: sending discover
udhcpc: sending discover
udhcpc: sending discover
udhcpc: no lease, forking to background
done.
Starting system message bus: dbus.
Starting random number generator daemon.
Starting OpenBSD Secure Shell server: sshd
  generating ssh RSA host key...
  generating ssh ECDSA host key...
  generating ssh ED25519 host key...
[   21.472551] NET: Registered protocol family 10
[   21.486542] Segment Routing with IPv6
done.
Starting rpcbind daemon...done.
starting statd: done
Starting atd: OK
[   21.992880] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
starting 8 nfsd kernel threads: [   23.151164] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
[   23.163714] NFSD: Using legacy client tracking operations.
[   23.169445] NFSD: starting 90-second grace period (net f00004c1)
done
starting mountd: done
Starting system log daemon...0
Starting crond: OK
umount: /mnt/.psplash: no mount point specified.

Poky (Yocto Project Reference Distro) 3.1+snapshot raspberrypi3-64 ttyS0

raspberrypi3-64 login:

Good! Now we have a solid base on which to work with our device!

Adding Software

As I mentioned earlier, the Pimoroni Automation-HAT comes with a python library to help you get up and running with your device. But how do you get that library onto the device? In theory you could install pip (and whatever it requires) into your image, run that image on the device, connect the device to the internet, and install the library by hand. But this isn't what we want. We don't want to generate a partial image that then needs manual intervention in the production stage. What if we were preparing 20 such devices? The moment human intervention is involved, the chances for errors goes up… significantly! What happens when all our devices are provisioned and we then discover we need to make a fix or add more steps?

What we want is a way to build a fully complete master image on the host that has all the software we need with everything configured and ready to go. Flash a µSD card, insert it into the device, add power, and it's up and running. Not: flash the µSD card, insert it into the device, log into the device, make sure the network is running, run a bunch of comands by hand…

The way we add things to an OpenEmbedded image is to create a recipe. OE layers are full of recipes. There are thousands and thousands of recipes for all sorts of software and situations. The first thing we do is to check to see if someone else had already written a recipe for the software we want to add. To do this we check the layer index: http://layers.openembedded.org/. Select Recipes, enter a string, and click search. Unfortunately there doesn't seem to be any pre-existing recipes for us.

We could just write a recipe from scratch. But before resorting to that scenario, we could use devtool. Devtool is a tool that is part of OpenEmbedded that helps users to create and maintain recipes. Before asking devtool to create the recipe for us, we have to understand what it is we want to build. We start off by visiting the code's website: https://github.com/pimoroni/automation-hat


Along the right-hand side, we see that this project has produced releases, but that the last release was about 5 months ago. There have been 8 commits since the last release. Therefore I think it would be best to work from the master branch's HEAD commit instead of the release.

Continuing to work in the same shell that was setup earlier, the same one we've been using to bitbake all our builds so far:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
$ devtool add --srcrev a41084cb4db02b76fc536235321a613d1c88ba52 automation-hat https://github.com/pimoroni/automation-hat.git
NOTE: Starting bitbake server...
INFO: Creating workspace layer in /z/build-master/6951/automation-hat/poky/build/workspace
NOTE: Starting bitbake server...
INFO: Fetching git://github.com/pimoroni/automation-hat.git;protocol=https;nobranch=1...
Loading cache: 100% |                                                                                                                                                                          | ETA:  --:--:--
Loaded 0 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################################################################| Time: 0:00:06
Parsing of 815 .bb files complete (0 cached, 815 parsed). 1384 targets, 65 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.47.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "aarch64-poky-linux"
MACHINE              = "raspberrypi3-64"
DISTRO               = "poky"
DISTRO_VERSION       = "3.1+snapshot-20201018"
TUNE_FEATURES        = "aarch64 armv8a crc cortexa53"
TARGET_FPU           = ""
meta                 
meta-poky            
meta-yocto-bsp       = "master:7cad26d585f67fa6bf873b8be361c6335a7db376"
meta-raspberrypi     = "master:6f85611576b7ccbfb6012631f741bd1daeffc9c9"
workspace            = "<unknown>:<unknown>"

Initialising tasks: 100% |######################################################################################################################################################################| Time: 0:00:00
Sstate summary: Wanted 0 Found 0 Missed 0 Current 0 (0% match, 0% complete)
NOTE: No setscene tasks
NOTE: Executing Tasks
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 0 seconds
INFO: Using default source tree path /z/build-master/6951/automation-hat/poky/build/workspace/sources/automation-hat
NOTE: Starting bitbake server...
INFO: Using source tree as build directory since that would be the default for this recipe
INFO: Recipe /z/build-master/6951/automation-hat/poky/build/workspace/recipes/automation-hat/automation-hat_git.bb has been automatically created; further editing may be required to make it fully functional

As the comment says, we should take a look at what devtool created to see if it needs any tweaking. But where do we find the recipe devtool just created? Technically devtool created a workspace for you and put the recipe there. But we don't even need to know that detail since we can just let devtool do all the work:

1
$ devtool edit-recipe automation-hat

Devtool will locate the recipe for you and open it in your $EDITOR. As we can see devtool created some parts of the recipe, but it only did a so-so job. It doesn't appear to know how to handle this python library particularly well:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
# Recipe created by recipetool
# This is the basis of a recipe and may need further editing in order to be fully functional.
# (Feel free to remove these comments when editing.)

# WARNING: the following LICENSE and LIC_FILES_CHKSUM values are best guesses - it is
# your responsibility to verify that the values are complete and correct.
#
# The following license files were not able to be identified and are
# represented as "Unknown" below, you will need to check them yourself:
#   packaging/debian/copyright
#
# NOTE: multiple licenses have been detected; they have been separated with &
# in the LICENSE value for now since it is a reasonable assumption that all
# of the licenses apply. If instead there is a choice between the multiple
# licenses then you should change the value to separate the licenses with |
# instead of &. If there is any doubt, check the accompanying documentation
# to determine which situation is applicable.
LICENSE = "MIT & Unknown"
LIC_FILES_CHKSUM = "file://LICENSE;md5=1be647ba1f2b40ff716e997a3e4a0d47 \
                    file://library/LICENSE.txt;md5=1be647ba1f2b40ff716e997a3e4a0d47 \
                    file://packaging/debian/copyright;md5=711b00d4d30e94965c8a959ae574dcc2"

SRC_URI = "git://github.com/pimoroni/automation-hat.git;protocol=https"

# Modify these as desired
PV = "1.0+git${SRCPV}"
SRCREV = "a41084cb4db02b76fc536235321a613d1c88ba52"

S = "${WORKDIR}/git"

# NOTE: this is a Makefile-only piece of software, so we cannot generate much of the
# recipe automatically - you will need to examine the Makefile yourself and ensure
# that the appropriate arguments are passed in.

do_configure () {
        # Specify any needed configure commands here
        :
}

do_compile () {
        # You will almost certainly need to add additional arguments here
        oe_runmake
}

do_install () {
        # This is a guess; additional arguments may be required
        oe_runmake install
}

Good recipes should have a SUMMARY, a DESCRIPTION, and a HOMEPAGE tag (among others). Also, experience dictates that OpenEmbedded includes many recipes for dealing with python libraries. Chances are good there's already a bbclass that takes care of this situation. A little investigation leads to the setuptools3 class:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
SUMMARY = "Pimoroni Automation-HAT python library"
DESCRIPTION = "Automation HAT is a home monitoring and automation controller \
featuring relays, analog channels, powered outputs, and buffered inputs \
(all 24V tolerant)."
HOMEPAGE = "https://shop.pimoroni.com/products/automation-hat"
SECTION = "iot/python"
LICENSE = "MIT & Unknown"
LIC_FILES_CHKSUM = "file://LICENSE;md5=1be647ba1f2b40ff716e997a3e4a0d47 \
                    file://library/LICENSE.txt;md5=1be647ba1f2b40ff716e997a3e4a0d47 \
                    file://packaging/debian/copyright;md5=711b00d4d30e94965c8a959ae574dcc2"

SRC_URI = "git://github.com/pimoroni/automation-hat.git;protocol=https"

PV = "1.0+git${SRCPV}"
SRCREV = "a41084cb4db02b76fc536235321a613d1c88ba52"

S = "${WORKDIR}/git"

inherit setuptools3

Now we try building:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
$ devtool build automation-hat
NOTE: Starting bitbake server...
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)...
Loading cache: 100% |###########################################################################################################################################################################| Time: 0:00:00
Loaded 1384 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################################################################| Time: 0:00:00
Parsing of 815 .bb files complete (814 cached, 1 parsed). 1384 targets, 65 skipped, 0 masked, 0 errors.
Loading cache: 100% |###########################################################################################################################################################################| Time: 0:00:01
Loaded 1384 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################################################################| Time: 0:00:00
Parsing of 815 .bb files complete (814 cached, 1 parsed). 1384 targets, 65 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.47.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "aarch64-poky-linux"
MACHINE              = "raspberrypi3-64"
DISTRO               = "poky"
DISTRO_VERSION       = "3.1+snapshot-20201018"
TUNE_FEATURES        = "aarch64 armv8a crc cortexa53"
TARGET_FPU           = ""
meta                 
meta-poky            
meta-yocto-bsp       = "master:7cad26d585f67fa6bf873b8be361c6335a7db376"
meta-raspberrypi     = "master:6f85611576b7ccbfb6012631f741bd1daeffc9c9"
workspace            = "master:b1a0414a6df77674a860c365825a4500e6cd698b"

Initialising tasks: 100% |######################################################################################################################################################################| Time: 0:00:01
Sstate summary: Wanted 0 Found 0 Missed 0 Current 190 (0% match, 100% complete)
NOTE: Executing Tasks
NOTE: automation-hat: compiling from external source tree /z/build-master/6951/automation-hat/poky/build/workspace/sources/automation-hat
ERROR: automation-hat-1.0+git999-r0 do_compile: 'python3 setup.py build ' execution failed.
ERROR: automation-hat-1.0+git999-r0 do_compile: Execution of '/z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069' failed with exit code 1:
/z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/recipe-sysroot-native/usr/bin/python3-native/python3: can't open file '/z/build-master/6951/automation-hat/poky/build/workspace/sources/automation-hat/setup.py': [Errno 2] No such file or directory
WARNING: /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069:175 exit 1 from 'exit 1'
WARNING: Backtrace (BB generated script): 
        #1: bbfatal_log, /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069, line 175
        #2: distutils3_do_compile, /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069, line 165
        #3: do_compile, /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069, line 154
        #4: main, /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069, line 179

Backtrace (metadata-relative locations):
        #1: bbfatal_log, /opt/oe/configs/z/build-master/6951/automation-hat/poky/layers/poky/meta/classes/logging.bbclass, line 72
        #2: distutils3_do_compile, /opt/oe/configs/z/build-master/6951/automation-hat/poky/layers/poky/meta/classes/distutils3.bbclass, line 26
        #3: do_compile, autogenerated, line 2
ERROR: Logfile of failure stored in: /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/log.do_compile.16069
Log data follows:
| DEBUG: Executing python function externalsrc_compile_prefunc
| NOTE: automation-hat: compiling from external source tree /z/build-master/6951/automation-hat/poky/build/workspace/sources/automation-hat
| DEBUG: Python function externalsrc_compile_prefunc finished
| DEBUG: Executing shell function do_compile
| /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/recipe-sysroot-native/usr/bin/python3-native/python3: can't open file '/z/build-master/6951/automation-hat/poky/build/workspace/sources/automation-hat/setup.py': [Errno 2] No such file or directory
| ERROR: 'python3 setup.py build ' execution failed.
| WARNING: /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069:175 exit 1 from 'exit 1'
| WARNING: Backtrace (BB generated script):
|       #1: bbfatal_log, /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069, line 175
|       #2: distutils3_do_compile, /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069, line 165
|       #3: do_compile, /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069, line 154
|       #4: main, /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069, line 179
| ERROR: Execution of '/z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069' failed with exit code 1:
| /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/recipe-sysroot-native/usr/bin/python3-native/python3: can't open file '/z/build-master/6951/automation-hat/poky/build/workspace/sources/automation-hat/setup.py': [Errno 2] No such file or directory
| WARNING: /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069:175 exit 1 from 'exit 1'
| WARNING: Backtrace (BB generated script):
|       #1: bbfatal_log, /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069, line 175
|       #2: distutils3_do_compile, /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069, line 165
|       #3: do_compile, /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069, line 154
|       #4: main, /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp/run.do_compile.16069, line 179
| 
| Backtrace (metadata-relative locations):
|       #1: bbfatal_log, /opt/oe/configs/z/build-master/6951/automation-hat/poky/layers/poky/meta/classes/logging.bbclass, line 72
|       #2: distutils3_do_compile, /opt/oe/configs/z/build-master/6951/automation-hat/poky/layers/poky/meta/classes/distutils3.bbclass, line 26
|       #3: do_compile, autogenerated, line 2
ERROR: Task (/z/build-master/6951/automation-hat/poky/build/workspace/recipes/automation-hat/automation-hat_git.bb:do_compile) failed with exit code '1'
NOTE: Tasks Summary: Attempted 723 tasks of which 719 didn't need to be rerun and 1 failed.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 1 seconds
NOTE: Build completion summary:
NOTE:   do_deploy_source_date_epoch: 0.0% sstate reuse(0 setscene, 1 scratch)

Summary: 1 task failed:
  /z/build-master/6951/automation-hat/poky/build/workspace/recipes/automation-hat/automation-hat_git.bb:do_compile
Summary: There were 2 ERROR messages shown, returning a non-zero exit code.

Looking through the error message we can see that the setuptools3 bbclass is not able to find the setup.py file from the library. Looking through the sources we see that the setup.py file is found in the library subdirectory. Therefore we edit the recipe again and tweak:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
SUMMARY = "Pimoroni Automation-HAT python library"
DESCRIPTION = "Automation HAT is a home monitoring and automation controller \
featuring relays, analog channels, powered outputs, and buffered inputs \
(all 24V tolerant)."
HOMEPAGE = "https://shop.pimoroni.com/products/automation-hat"
SECTION = "iot/python"
LICENSE = "MIT & Unknown"
LIC_FILES_CHKSUM = "file://LICENSE;md5=1be647ba1f2b40ff716e997a3e4a0d47 \
                    file://library/LICENSE.txt;md5=1be647ba1f2b40ff716e997a3e4a0d47 \
                    file://packaging/debian/copyright;md5=711b00d4d30e94965c8a959ae574dcc2"

SRC_URI = "git://github.com/pimoroni/automation-hat.git;protocol=https"

PV = "1.0+git${SRCPV}"
SRCREV = "a41084cb4db02b76fc536235321a613d1c88ba52"

S = "${WORKDIR}/git/library"

inherit setuptools3

We build again… and we get the same error? Now we need to dig a little deeper. If we've modified the S variable explicitly, why is the build still looking in the old location? Now we ask bitbake to explain itself: "bitbake what do you know about the S variable?":

1
2
$ bitbake automation-hat -e | grep "^S="
S="/z/build-master/6951/automation-hat/poky/build/workspace/sources/automation-hat"

This is very perplexing. We're setting the S variable explicitly in our recipe, but something is overriding it. If we look deeper into the workspace directory that devtool creates for us, we find:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
$ tree workspace
workspace
├── README
├── appends
│   └── automation-hat_git.bbappend
├── conf
│   └── layer.conf
├── recipes
│   └── automation-hat
│       └── automation-hat_git.bb
└── sources
    └── automation-hat
        ├── LICENSE
        ├── Makefile
        ├── README.md
        ├── autohat_360.png
        ├── documentation
        │   └── REFERENCE.md
        ├── examples
        │   ├── hat
        │   │   ├── analog.py
        │   │   ├── input.py
        │   │   ├── lights.py
        │   │   ├── output.py
        │   │   └── relay.py
        │   └── hat-mini
        │       ├── analog.py
        │       ├── images
        │       │   ├── analog-inputs-blank.jpg
        │       │   ├── inputs-blank.jpg
        │       │   └── outputs-blank.jpg
        │       ├── input.py
        │       └── output.py
        ├── library
        │   ├── CHANGELOG.txt
        │   ├── LICENSE.txt
        │   ├── MANIFEST.in
        │   ├── README.rst
        │   ├── automationhat
        │   │   ├── __init__.py
        │   │   ├── ads1015.py
        │   │   └── pins.py
        │   ├── setup.py
        │   ├── test.py
        │   ├── tests
        │   │   └── test_setup.py
        │   └── tox.ini
        ├── oe-logs -> /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0/temp
        ├── oe-workdir -> /z/build-master/6951/automation-hat/poky/build/tmp/work/cortexa53-poky-linux/automation-hat/1.0+git999-r0
        ├── packaging
        │   ├── CHANGELOG
        │   ├── debian
        │   │   ├── README
        │   │   ├── changelog
        │   │   ├── clean
        │   │   ├── compat
        │   │   ├── control
        │   │   ├── copyright
        │   │   ├── docs
        │   │   ├── rules
        │   │   └── source
        │   │       ├── format
        │   │       └── options
        │   ├── makeall.sh
        │   ├── makedeb.sh
        │   └── makelog.sh
        └── terminal.jpg

19 directories, 46 files

Snooping around in the workspace I'm curious about the bbappend:

1
2
3
4
5
6
$ cat workspace/appends/automation-hat_git.bbappend
inherit externalsrc
EXTERNALSRC = "/z/build-master/6951/automation-hat/poky/build/workspace/sources/automation-hat"
EXTERNALSRC_BUILD = "/z/build-master/6951/automation-hat/poky/build/workspace/sources/automation-hat"

# initial_rev: a41084cb4db02b76fc536235321a613d1c88ba52

It's pulling in the externalsrc bbclass. I wonder what the externalsrc bbclass does and whether or not it plays with the S variable? Turns out it does! Looking at the top part of the externalsrc bbclass:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
# Copyright (C) 2012 Linux Foundation
# Author: Richard Purdie
# Some code and influence taken from srctree.bbclass:
# Copyright (C) 2009 Chris Larson <clarson@kergoth.com>
# Released under the MIT license (see COPYING.MIT for the terms)
#
# externalsrc.bbclass enables use of an existing source tree, usually external to
# the build system to build a piece of software rather than the usual fetch/unpack/patch
# process.
#
# To use, add externalsrc to the global inherit and set EXTERNALSRC to point at the
# directory you want to use containing the sources e.g. from local.conf for a recipe
# called "myrecipe" you would do:
#
# INHERIT += "externalsrc"
# EXTERNALSRC_pn-myrecipe = "/path/to/my/source/tree"
#
# In order to make this class work for both target and native versions (or with
# multilibs/cross or other BBCLASSEXTEND variants), B is set to point to a separate
# directory under the work directory (split source and build directories). This is
# the default, but the build directory can be set to the source directory if
# circumstances dictate by setting EXTERNALSRC_BUILD to the same value, e.g.:
#
# EXTERNALSRC_BUILD_pn-myrecipe = "/path/to/my/source/tree"
#

SRCTREECOVEREDTASKS ?= "do_patch do_unpack do_fetch"
EXTERNALSRC_SYMLINKS ?= "oe-workdir:${WORKDIR} oe-logs:${T}"

python () {
    externalsrc = d.getVar('EXTERNALSRC')
    externalsrcbuild = d.getVar('EXTERNALSRC_BUILD')

    if externalsrc and not externalsrc.startswith("/"):
        bb.error("EXTERNALSRC must be an absolute path")
    if externalsrcbuild and not externalsrcbuild.startswith("/"):
        bb.error("EXTERNALSRC_BUILD must be an absolute path")

    # If this is the base recipe and EXTERNALSRC is set for it or any of its
    # derivatives, then enable BB_DONT_CACHE to force the recipe to always be
    # re-parsed so that the file-checksums function for do_compile is run every
    # time.
    bpn = d.getVar('BPN')
    classextend = (d.getVar('BBCLASSEXTEND') or '').split()
    if bpn == d.getVar('PN') or not classextend:
        if (externalsrc or
                ('native' in classextend and 
                 d.getVar('EXTERNALSRC_pn-%s-native' % bpn)) or
                ('nativesdk' in classextend and 
                 d.getVar('EXTERNALSRC_pn-nativesdk-%s' % bpn)) or
                ('cross' in classextend and 
                 d.getVar('EXTERNALSRC_pn-%s-cross' % bpn))):
            d.setVar('BB_DONT_CACHE', '1')

    if externalsrc:
        import oe.recipeutils
        import oe.path

        d.setVar('S', externalsrc)
        if externalsrcbuild:
            d.setVar('B', externalsrcbuild)
        else:
            d.setVar('B', '${WORKDIR}/${BPN}-${PV}/')

…

We can see on line 59 that the S variable is being set to externalsrc, which is being set on line 31 to the value of EXTERNALSRC which it finds in the automation-hat.bbappend file. Therefore if we tweak the automation-hat.bbappend file it should be able to find our setup.py code:

1
2
3
4
5
6
$ cat workspace/appends/automation-hat_git.bbappend 
inherit externalsrc
EXTERNALSRC = "/z/build-master/6951/automation-hat/poky/build/workspace/sources/automation-hat/library"
EXTERNALSRC_BUILD = "/z/build-master/6951/automation-hat/poky/build/workspace/sources/automation-hat/library"

# initial_rev: a41084cb4db02b76fc536235321a613d1c88ba52

and build:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
$ devtool build automation-hat
NOTE: Starting bitbake server...
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)...
Loading cache: 100% |###########################################################################################################################################################################| Time: 0:00:00
Loaded 1384 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################################################################| Time: 0:00:00
Parsing of 815 .bb files complete (814 cached, 1 parsed). 1384 targets, 65 skipped, 0 masked, 0 errors.
Loading cache: 100% |###########################################################################################################################################################################| Time: 0:00:01
Loaded 1384 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################################################################| Time: 0:00:00
Parsing of 815 .bb files complete (814 cached, 1 parsed). 1384 targets, 65 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.47.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "aarch64-poky-linux"
MACHINE              = "raspberrypi3-64"
DISTRO               = "poky"
DISTRO_VERSION       = "3.1+snapshot-20201018"
TUNE_FEATURES        = "aarch64 armv8a crc cortexa53"
TARGET_FPU           = ""
meta                 
meta-poky            
meta-yocto-bsp       = "master:7cad26d585f67fa6bf873b8be361c6335a7db376"
meta-raspberrypi     = "master:6f85611576b7ccbfb6012631f741bd1daeffc9c9"
workspace            = "master:b1a0414a6df77674a860c365825a4500e6cd698b"

Initialising tasks: 100% |######################################################################################################################################################################| Time: 0:00:01
Sstate summary: Wanted 0 Found 0 Missed 0 Current 190 (0% match, 100% complete)
NOTE: Executing Tasks
NOTE: automation-hat: compiling from external source tree /z/build-master/6951/automation-hat/poky/build/workspace/sources/automation-hat/library
NOTE: Tasks Summary: Attempted 727 tasks of which 721 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 0 seconds
NOTE: Build completion summary:
NOTE:   do_populate_sysroot: 0.0% sstate reuse(0 setscene, 1 scratch)
NOTE:   do_package: 0.0% sstate reuse(0 setscene, 1 scratch)
NOTE:   do_packagedata: 0.0% sstate reuse(0 setscene, 1 scratch)

Phew!

However, this isn't the "correct" solution. Turns out devtool has an option for this very situation (i.e. the setup.py not being in the top-level directory of a repository). If we look at the options available for devtool add we find:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
$ devtool add
NOTE: Starting bitbake server...
devtool add: error: At least one of recipename, srctree, fetchuri or -f/--fetch must be specified
usage: devtool add [-h] [--same-dir | --no-same-dir] [--fetch URI] [--npm-dev]
                   [--version VERSION] [--no-git]
                   [--srcrev SRCREV | --autorev] [--srcbranch SRCBRANCH]
                   [--binary] [--also-native] [--src-subdir SUBDIR]
                   [--mirrors] [--provides PROVIDES]
                   [recipename] [srctree] [fetchuri]

Adds a new recipe to the workspace to build a specified source tree. Can
optionally fetch a remote URI and unpack it to create the source tree.

arguments:
  recipename            Name for new recipe to add (just name - no version,
                        path or extension). If not specified, will attempt to
                        auto-detect it.
  srctree               Path to external source tree. If not specified, a
                        subdirectory of /z/build-master/6951/automation-
                        hat/build/workspace/sources will be used.
  fetchuri              Fetch the specified URI and extract it to create the
                        source tree

options:
  -h, --help            show this help message and exit
  --same-dir, -s        Build in same directory as source
  --no-same-dir         Force build in a separate build directory
  --fetch URI, -f URI   Fetch the specified URI and extract it to create the
                        source tree (deprecated - pass as positional argument
                        instead)
  --npm-dev             For npm, also fetch devDependencies
  --version VERSION, -V VERSION
                        Version to use within recipe (PV)
  --no-git, -g          If fetching source, do not set up source tree as a git
                        repository
  --srcrev SRCREV, -S SRCREV
                        Source revision to fetch if fetching from an SCM such
                        as git (default latest)
  --autorev, -a         When fetching from a git repository, set SRCREV in the
                        recipe to a floating revision instead of fixed
  --srcbranch SRCBRANCH, -B SRCBRANCH
                        Branch in source repository if fetching from an SCM
                        such as git (default master)
  --binary, -b          Treat the source tree as something that should be
                        installed verbatim (no compilation, same directory
                        structure). Useful with binary packages e.g. RPMs.
  --also-native         Also add native variant (i.e. support building recipe
                        for the build host as well as the target machine)
  --src-subdir SUBDIR   Specify subdirectory within source tree to use
  --mirrors             Enable PREMIRRORS and MIRRORS for source tree fetching
                        (disable by default).
  --provides PROVIDES, -p PROVIDES
                        Specify an alias for the item provided by the recipe.
                        E.g. virtual/libgl

So let's try creating the recipe again with the --src-subdir option:

1
2
3
$ devtool add --srcrev a41084cb4db02b76fc536235321a613d1c88ba52 --src-subdir library automation-hat https://github.com/pimoroni/automation-hat.git
NOTE: Starting bitbake server...
ERROR: recipe automation-hat is already in your workspace

We have to clean out the workspace before we can try again (if we want to use the same recipe name):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
$ devtool reset automation-hat
NOTE: Starting bitbake server...
INFO: Cleaning sysroot for recipe automation-hat...
WARNING: File automation-hat_git.bb modified since it was written, preserving in /z/build-master/6951/automation-hat/poky/build/workspace/attic/automation-hat
WARNING: File automation-hat_git.bbappend modified since it was written, preserving in /z/build-master/6951/automation-hat/poky/build/workspace/attic/automation-hat
INFO: Leaving source tree /z/build-master/6951/automation-hat/poky/build/workspace/sources/automation-hat/library as-is; if you no longer need it then please delete it manually

$ rm -fr workspace/sources/automation-hat/

$ devtool add --srcrev a41084cb4db02b76fc536235321a613d1c88ba52 --src-subdir library automation-hat https://github.com/pimoroni/automation-hat.git
NOTE: Starting bitbake server...
NOTE: Starting bitbake server...
INFO: Fetching git://github.com/pimoroni/automation-hat.git;protocol=https;nobranch=1...
Loading cache: 100% |###########################################################################################################################################################################| Time: 0:00:01
Loaded 1384 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################################################################| Time: 0:00:00
Parsing of 815 .bb files complete (814 cached, 1 parsed). 1384 targets, 65 skipped, 0 masked, 0 errors.
Removing 1 recipes from the cortexa53 sysroot: 100% |###########################################################################################################################################| Time: 0:00:00
Removing 1 recipes from the raspberrypi3_64 sysroot: 100% |#####################################################################################################################################| Time: 0:00:00
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.47.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "aarch64-poky-linux"
MACHINE              = "raspberrypi3-64"
DISTRO               = "poky"
DISTRO_VERSION       = "3.1+snapshot-20201018"
TUNE_FEATURES        = "aarch64 armv8a crc cortexa53"
TARGET_FPU           = ""
meta                 
meta-poky            
meta-yocto-bsp       = "master:7cad26d585f67fa6bf873b8be361c6335a7db376"
meta-raspberrypi     = "master:6f85611576b7ccbfb6012631f741bd1daeffc9c9"
workspace            = "master:b1a0414a6df77674a860c365825a4500e6cd698b"

Initialising tasks: 100% |######################################################################################################################################################################| Time: 0:00:00
Sstate summary: Wanted 0 Found 0 Missed 0 Current 0 (0% match, 0% complete)
NOTE: No setscene tasks
NOTE: Executing Tasks
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 1 seconds
INFO: Scanning paths for packages & dependencies: automationhat
INFO: Using default source tree path /z/build-master/6951/automation-hat/poky/build/workspace/sources/automation-hat
WARNING: A modified recipe from a previous invocation exists in /z/build-master/6951/automation-hat/poky/build/workspace/attic/automation-hat/automation-hat_git.bb - you may wish to move this over the top of the new recipe if you had changes in it that you want to continue with
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)...
NOTE: Reconnecting to bitbake server...
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)...
NOTE: Retrying server connection (#1)...
NOTE: Starting bitbake server...
INFO: Recipe /z/build-master/6951/automation-hat/poky/build/workspace/recipes/automation-hat/automation-hat_git.bb has been automatically created; further editing may be required to make it fully functional

When we look at the recipe devtool generated for us we find a lot of improvements:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# Recipe created by recipetool
# This is the basis of a recipe and may need further editing in order to be fully functional.
# (Feel free to remove these comments when editing.)

SUMMARY = "Automation HAT Driver"
HOMEPAGE = "http://www.pimoroni.com"
# WARNING: the following LICENSE and LIC_FILES_CHKSUM values are best guesses - it is
# your responsibility to verify that the values are complete and correct.
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=1be647ba1f2b40ff716e997a3e4a0d47"

SRC_URI = "git://github.com/pimoroni/automation-hat.git;protocol=https"

# Modify these as desired
PV = "0.2.2+git${SRCPV}"
SRCREV = "a41084cb4db02b76fc536235321a613d1c88ba52"

S = "${WORKDIR}/git/library"

inherit setuptools3

# WARNING: the following rdepends are from setuptools install_requires. These
# upstream names may not correspond exactly to bitbake package names.
RDEPENDS_${PN} += "python3-rpi-gpio python3-st7735 python3-sn3218"

# WARNING: the following rdepends are determined through basic analysis of the
# python sources, and might not be 100% accurate.
RDEPENDS_${PN} += "python3-core"

# WARNING: We were unable to map the following python package/module
# dependencies to the bitbake packages which include them:
#    smbus

  • line 5: devtool found a summary (although I'd prefer to change it to "Automation-HAT Python Library")
  • line 6: devtool found a homepage (although I'd prefer to point it to the specific product, not just the company's website)
  • lines 9-10: it's clearer about the license
  • line 15: it found a version (although the latest version is 0.2.3, not 0.2.2)
  • line 18: it set the S variable correctly
  • line 20: devtool discovered that it could use setuptools3 to build
  • lines 24 and 28: devtool found a couple runtime dependencies

This is great! This is a much nicer recipe to use as a starting point. Good job devtool.

After a little cleaning up, my recipe now looks like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
SUMMARY = "Automation-HAT Python Library"
DESCRIPTION = "Automation HAT is a home monitoring and automation controller \
featuring relays, analog channels, powered outputs, and buffered inputs (all \
24V tolerant)."
HOMEPAGE = "https://shop.pimoroni.com/products/automation-hat"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=1be647ba1f2b40ff716e997a3e4a0d47"

SRC_URI = "git://github.com/pimoroni/automation-hat.git;protocol=https"

PV = "0.2.3+git${SRCPV}"
SRCREV = "a41084cb4db02b76fc536235321a613d1c88ba52"

S = "${WORKDIR}/git/library"

inherit setuptools3

RDEPENDS_${PN} += "rpi-gpio python3-smbus python3-sn3218 python3-core"

  • line 1: I've updated the summary
  • line 2: I've added a description
  • line 5: I've updated the homepage
  • I've verified the license and therefore removed the comment warning me to check it by hand
  • line 11: I've updated the version
  • I've removed the comments around the runtime dependencies
  • line 18: I've changed the name of the python3-rpi-gpio runtime dependency to rpi-gpio and I've removed the python3-st7735 runtime dependency since I don't have the ST7735 TFT display attached to my setup. I've consolidated the two RDEPENDS lines, and figured out that the recipe needs a runtime dependency on python3-smbus

Searching the OpenEmbedded layer index for the names of the runtime dependencies that devtool has identified in creating this recipe, I found that a recipe already exists for rpi-gpio (it's in the meta-raspberrypi BSP layer, which we're already using), so I updated the name to match. A recipe for the other runtime dependency, python3-sn3218, isn't available and this is probably because this repository is also from Pimoroni's github account.

There's no point trying to build the automation-hat recipe until we've attempted to satisfy all of its (runtime) dependencies so we invoke devtool again:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
$ devtool add --version 1.2.7 --srcrev d497c6e9762c6d31bd9a7f9da9ccef0318b8e31c --src-subdir library python3-sn3218 https://github.com/pimoroni/sn3218
NOTE: Starting bitbake server...
NOTE: Starting bitbake server...
INFO: Fetching git://github.com/pimoroni/sn3218;protocol=https;nobranch=1...
Loading cache: 100% |###########################################################################################################################################################################| Time: 0:00:01
Loaded 1385 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################################################################| Time: 0:00:00
Parsing of 816 .bb files complete (814 cached, 2 parsed). 1385 targets, 65 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.47.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "aarch64-poky-linux"
MACHINE              = "raspberrypi3-64"
DISTRO               = "poky"
DISTRO_VERSION       = "3.1+snapshot-20201018"
TUNE_FEATURES        = "aarch64 armv8a crc cortexa53"
TARGET_FPU           = ""
meta                 
meta-poky            
meta-yocto-bsp       = "master:7cad26d585f67fa6bf873b8be361c6335a7db376"
meta-raspberrypi     = "master:6f85611576b7ccbfb6012631f741bd1daeffc9c9"
workspace            = "master:b1a0414a6df77674a860c365825a4500e6cd698b"

Initialising tasks: 100% |######################################################################################################################################################################| Time: 0:00:00
Sstate summary: Wanted 0 Found 0 Missed 0 Current 0 (0% match, 0% complete)
NOTE: No setscene tasks
NOTE: Executing Tasks
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 1 seconds
INFO: Scanning paths for packages & dependencies: sn3218.py
INFO: Using default source tree path /z/build-master/6951/automation-hat/poky/build/workspace/sources/python3-sn3218
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)...
NOTE: Reconnecting to bitbake server...
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)...
NOTE: Retrying server connection (#1)...
NOTE: Starting bitbake server...
INFO: Recipe /z/build-master/6951/automation-hat/poky/build/workspace/recipes/python3-sn3218/python3-sn3218_git.bb has been automatically created; further editing may be required to make it fully functional

Looking at the resulting recipe:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# Recipe created by recipetool
# This is the basis of a recipe and may need further editing in order to be fully functional.
# (Feel free to remove these comments when editing.)

SUMMARY = "A module to drive the sn3218 i2c LED controller"
HOMEPAGE = "http://www.pimoroni.com"
# WARNING: the following LICENSE and LIC_FILES_CHKSUM values are best guesses - it is
# your responsibility to verify that the values are complete and correct.
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=be85037b505b21b386e82d53cd928b5e"

SRC_URI = "git://github.com/pimoroni/sn3218;protocol=https"

# Modify these as desired
PV = "1.2.7+git${SRCPV}"
SRCREV = "d497c6e9762c6d31bd9a7f9da9ccef0318b8e31c"

S = "${WORKDIR}/git/library"

inherit setuptools3

# WARNING: the following rdepends are determined through basic analysis of the
# python sources, and might not be 100% accurate.
RDEPENDS_${PN} += "python3-core"

# WARNING: We were unable to map the following python package/module
# dependencies to the bitbake packages which include them:
#    smbus

This recipe is looking pretty good already. I simply need to clean up some of the preamble tags, and remove most of the comments that devtool has placed into the file for my attention:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
SUMMARY = "A module to drive the sn3218 i2c LED controller"
HOMEPAGE = "https://github.com/pimoroni/sn3218"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=be85037b505b21b386e82d53cd928b5e"

SRC_URI = "git://github.com/pimoroni/sn3218;protocol=https"

PV = "1.2.7+git${SRCPV}"
SRCREV = "d497c6e9762c6d31bd9a7f9da9ccef0318b8e31c"

S = "${WORKDIR}/git/library"

inherit setuptools3

RDEPENDS_${PN} += "python3-smbus python3-core"

You may have noticed the reference to smbus by both recipes. smbus is another way of saying i2c, so I make a note that I probably want to add i2c user-space tools to my build. This makes sense since most (all?) of the peripherals on the Automation-HAT interface to the Raspberry Pi via i2c. This change needs to be done in my conf/local.conf.

But what about the smbus runtime dependency itself? If we search the layer index we find that there already is a recipe for python3-smbus in meta-python, which is part of meta-openembedded. Currently we're not using meta-openembeded in our build, therefore we need to clone that repository too, and add it to our build layers. We could do it manually or:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
$ bitbake-layers layerindex-fetch -b master meta-python
NOTE: Starting bitbake server...
Loading https://layers.openembedded.org/layerindex/api/;branch=master...
Layer                                              Git repository (branch)                                 Subdirectory
=============================================================================================================================
local:HEAD:openembedded-core                       git://git.yoctoproject.org/poky (master)                meta
  required by: meta-python meta-oe
layers.openembedded.org:master:meta-oe             git://git.openembedded.org/meta-openembedded (master)   meta-oe
  required by: meta-python
layers.openembedded.org:master:meta-python         git://git.openembedded.org/meta-openembedded (master)   meta-python
Cloning into '/opt/oe/configs/z/build-master/6951/automation-hat/poky/layers/poky/meta-openembedded'...
remote: Counting objects: 141039, done.
remote: Compressing objects: 100% (46829/46829), done.
remote: Total 141039 (delta 88137), reused 140214 (delta 87626)
Receiving objects: 100% (141039/141039), 37.93 MiB | 297.00 KiB/s, done.
Resolving deltas: 100% (88137/88137), done.
Updating files: 100% (5664/5664), done.
Adding layer "meta-oe" (/opt/oe/configs/z/build-master/6951/automation-hat/poky/layers/poky/meta-openembedded/meta-oe) to conf/bblayers.conf
Adding layer "meta-python" (/opt/oe/configs/z/build-master/6951/automation-hat/poky/layers/poky/meta-openembedded/meta-python) to conf/bblayers.conf

I'm not overly fond of where it placed the downloads, but the nice thing is that it took care of the dependency (meta-python depends on meta-oe) and it updated my conf/bblayers.conf file automatically for me.

Similarly to how we enabled the UART a while back, we have to explicitly tell the RaspberryPi that we want to enable i2c by setting the ENABLE_I2C to 1. NOTE: this is specific to the RaspberryPi due to the way its hardware was designed; having to explicitly enable a UART or i2c is not an OpenEmbedded thing or a thing in general. This only applies to the ENABLE_I2C setting, the other tweaks are common OpenEmbedded idioms.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
MACHINE = "raspberrypi3-64"
DL_DIR = "/opt/Downloads"
DISTRO = "poky"

ENABLE_UART = "1"
ENABLE_I2C = "1"
KERNEL_MODULE_AUTOLOAD += "i2c-dev"
CORE_IMAGE_EXTRA_INSTALL += " \
        ${MACHINE_EXTRA_RRECOMMENDS} \
        wpa-supplicant \
        i2c-tools \
        "

PACKAGECONFIG_append_pn-gdb = " tui"
PACKAGECONFIG_append_pn-gdb-cross-canadian-arm = " tui"

IMAGE_FSTYPES_append = " wic"
IMAGE_FSTYPES_remove = "wic.bz2 tar.bz2"
WARN_QA_append = " version-going-backwards"
ERROR_QA_remove = "version-going-backwards"
BB_DANGLINGAPPENDS_WARNONLY = "yes"
INHERIT += "buildhistory image-buildinfo buildstats-summary"
BUILDHISTORY_COMMIT = "1"

PACKAGE_CLASSES ?= "package_ipk"
SDKMACHINE = "x86_64"
EXTRA_IMAGE_FEATURES ?= "debug-tweaks"
USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS ??= "\
    STOPTASKS,${TMPDIR},1G,100K \
    STOPTASKS,${DL_DIR},1G,100K \
    STOPTASKS,${SSTATE_DIR},1G,100K \
    STOPTASKS,/tmp,100M,100K \
    ABORT,${TMPDIR},100M,1K \
    ABORT,${DL_DIR},100M,1K \
    ABORT,${SSTATE_DIR},100M,1K \
    ABORT,/tmp,10M,1K"
PACKAGECONFIG_append_pn-qemu-system-native = " sdl"
CONF_VERSION = "1"

Let's check to see if the new python3-sn3218 recipe will build:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
$ devtool build python3-sn3218
NOTE: Starting bitbake server...
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)...
Loading cache: 100% |                                                                                                                                                                          | ETA:  --:--:--
Loaded 0 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################################################################| Time: 0:00:16
Parsing of 2042 .bb files complete (0 cached, 2042 parsed). 3213 targets, 123 skipped, 0 masked, 0 errors.
Loading cache: 100% |###########################################################################################################################################################################| Time: 0:00:03
Loaded 3211 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################################################################| Time: 0:00:00
Parsing of 2042 .bb files complete (2040 cached, 2 parsed). 3213 targets, 123 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.47.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "aarch64-poky-linux"
MACHINE              = "raspberrypi3-64"
DISTRO               = "poky"
DISTRO_VERSION       = "3.1+snapshot-20201018"
TUNE_FEATURES        = "aarch64 armv8a crc cortexa53"
TARGET_FPU           = ""
meta                 
meta-poky            
meta-yocto-bsp       = "master:7cad26d585f67fa6bf873b8be361c6335a7db376"
meta-raspberrypi     = "master:6f85611576b7ccbfb6012631f741bd1daeffc9c9"
workspace            = "master:b1a0414a6df77674a860c365825a4500e6cd698b"
meta-oe              
meta-python          = "master:86a7820b7964ff91d7a26ac5c506e83292e347a3"

Initialising tasks: 100% |######################################################################################################################################################################| Time: 0:00:01
Sstate summary: Wanted 0 Found 0 Missed 0 Current 190 (0% match, 100% complete)
NOTE: Executing Tasks
NOTE: python3-sn3218: compiling from external source tree /z/build-master/6951/automation-hat/poky/build/workspace/sources/python3-sn3218/library
NOTE: Tasks Summary: Attempted 727 tasks of which 719 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 2 seconds
NOTE: Build completion summary:
NOTE:   do_populate_sysroot: 0.0% sstate reuse(0 setscene, 1 scratch)
NOTE:   do_deploy_source_date_epoch: 0.0% sstate reuse(0 setscene, 1 scratch)
NOTE:   do_package: 0.0% sstate reuse(0 setscene, 1 scratch)
NOTE:   do_packagedata: 0.0% sstate reuse(0 setscene, 1 scratch)

Success!

How about automation-hat?

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
$ devtool build automation-hat
NOTE: Starting bitbake server...
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)...
Loading cache: 100% |###########################################################################################################################################################################| Time: 0:00:00
Loaded 3211 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################################################################| Time: 0:00:00
Parsing of 2042 .bb files complete (2040 cached, 2 parsed). 3213 targets, 123 skipped, 0 masked, 0 errors.
Loading cache: 100% |###########################################################################################################################################################################| Time: 0:00:03
Loaded 3211 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################################################################| Time: 0:00:00
Parsing of 2042 .bb files complete (2040 cached, 2 parsed). 3213 targets, 123 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.47.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "aarch64-poky-linux"
MACHINE              = "raspberrypi3-64"
DISTRO               = "poky"
DISTRO_VERSION       = "3.1+snapshot-20201018"
TUNE_FEATURES        = "aarch64 armv8a crc cortexa53"
TARGET_FPU           = ""
meta                 
meta-poky            
meta-yocto-bsp       = "master:7cad26d585f67fa6bf873b8be361c6335a7db376"
meta-raspberrypi     = "master:6f85611576b7ccbfb6012631f741bd1daeffc9c9"
workspace            = "master:b1a0414a6df77674a860c365825a4500e6cd698b"
meta-oe              
meta-python          = "master:86a7820b7964ff91d7a26ac5c506e83292e347a3"

Initialising tasks: 100% |######################################################################################################################################################################| Time: 0:00:01
Sstate summary: Wanted 0 Found 0 Missed 0 Current 190 (0% match, 100% complete)
NOTE: Executing Tasks
NOTE: automation-hat: compiling from external source tree /z/build-master/6951/automation-hat/poky/build/workspace/sources/automation-hat/library
WARNING: automation-hat-0.2.3+git999-r0 do_packagedata: QA Issue: Package version for package automation-hat-src went backwards which would break package feeds (from 0:1.0+git999-r0 to 0:0.2.3+git999-r0) [version-going-backwards]
WARNING: automation-hat-0.2.3+git999-r0 do_packagedata: QA Issue: Package version for package automation-hat-dbg went backwards which would break package feeds (from 0:1.0+git999-r0 to 0:0.2.3+git999-r0) [version-going-backwards]
WARNING: automation-hat-0.2.3+git999-r0 do_packagedata: QA Issue: Package version for package automation-hat-staticdev went backwards which would break package feeds (from 0:1.0+git999-r0 to 0:0.2.3+git999-r0) [version-going-backwards]
WARNING: automation-hat-0.2.3+git999-r0 do_packagedata: QA Issue: Package version for package automation-hat-dev went backwards which would break package feeds (from 0:1.0+git999-r0 to 0:0.2.3+git999-r0) [version-going-backwards]
WARNING: automation-hat-0.2.3+git999-r0 do_packagedata: QA Issue: Package version for package automation-hat-doc went backwards which would break package feeds (from 0:1.0+git999-r0 to 0:0.2.3+git999-r0) [version-going-backwards]
WARNING: automation-hat-0.2.3+git999-r0 do_packagedata: QA Issue: Package version for package automation-hat-locale went backwards which would break package feeds (from 0:1.0+git999-r0 to 0:0.2.3+git999-r0) [version-going-backwards]
WARNING: automation-hat-0.2.3+git999-r0 do_packagedata: QA Issue: Package version for package automation-hat went backwards which would break package feeds (from 0:1.0+git999-r0 to 0:0.2.3+git999-r0) [version-going-backwards]
NOTE: Tasks Summary: Attempted 727 tasks of which 719 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 1 seconds
NOTE: Build completion summary:
NOTE:   do_populate_sysroot: 0.0% sstate reuse(0 setscene, 1 scratch)
NOTE:   do_deploy_source_date_epoch: 0.0% sstate reuse(0 setscene, 1 scratch)
NOTE:   do_package: 0.0% sstate reuse(0 setscene, 1 scratch)
NOTE:   do_packagedata: 0.0% sstate reuse(0 setscene, 1 scratch)

Summary: There were 7 WARNING messages shown.

Looking good!

Due to the fact we're adding these two new packages, plus both these packages have a runtime dependency on python3, plus we're adding i2c things and we want the i2c kernel module inserted at boot, it's probably time to build and flash a whole new image.

We could do the usual bitbake core-image-full-cmdline here, however if we do devtool build-image instead we'll find that devtool will add the recipes from our workspace (and their dependencies) into the resulting image, which is probably what we want.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
$ devtool build-image core-image-full-cmdline
NOTE: Starting bitbake server...
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)...
Loading cache: 100% |###########################################################################################################################################################################| Time: 0:00:00
Loaded 3211 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################################################################| Time: 0:00:00
Parsing of 2042 .bb files complete (2040 cached, 2 parsed). 3213 targets, 123 skipped, 0 masked, 0 errors.
INFO: Building image core-image-full-cmdline with the following additional packages: automation-hat python3-sn3218
Loading cache: 100% |###########################################################################################################################################################################| Time: 0:00:03
Loaded 3211 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################################################################| Time: 0:00:00
Parsing of 2042 .bb files complete (2039 cached, 3 parsed). 3213 targets, 123 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.47.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "aarch64-poky-linux"
MACHINE              = "raspberrypi3-64"
DISTRO               = "poky"
DISTRO_VERSION       = "3.1+snapshot-20201018"
TUNE_FEATURES        = "aarch64 armv8a crc cortexa53"
TARGET_FPU           = ""
meta                 
meta-poky            
meta-yocto-bsp       = "master:7cad26d585f67fa6bf873b8be361c6335a7db376"
meta-raspberrypi     = "master:6f85611576b7ccbfb6012631f741bd1daeffc9c9"
workspace            = "master:b1a0414a6df77674a860c365825a4500e6cd698b"
meta-oe              
meta-python          = "master:86a7820b7964ff91d7a26ac5c506e83292e347a3"

Initialising tasks: 100% |######################################################################################################################################################################| Time: 0:00:07
Sstate summary: Wanted 35 Found 0 Missed 35 Current 1508 (0% match, 97% complete)
NOTE: Executing Tasks
NOTE: python3-sn3218: compiling from external source tree /z/build-master/6951/automation-hat/poky/build/workspace/sources/python3-sn3218/library
NOTE: automation-hat: compiling from external source tree /z/build-master/6951/automation-hat/poky/build/workspace/sources/automation-hat/library
NOTE: Tasks Summary: Attempted 3998 tasks of which 3828 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 1 seconds
NOTE: Build completion summary:
NOTE:   do_populate_sysroot: 0.0% sstate reuse(0 setscene, 5 scratch)
NOTE:   do_deploy_source_date_epoch: 0.0% sstate reuse(0 setscene, 4 scratch)
NOTE:   do_package_qa: 0.0% sstate reuse(0 setscene, 7 scratch)
NOTE:   do_package: 0.0% sstate reuse(0 setscene, 7 scratch)
NOTE:   do_packagedata: 0.0% sstate reuse(0 setscene, 7 scratch)
NOTE:   do_package_write_ipk: 0.0% sstate reuse(0 setscene, 7 scratch)
NOTE:   do_populate_lic: 0.0% sstate reuse(0 setscene, 5 scratch)
INFO: Successfully built core-image-full-cmdline. You can find output files in /z/build-master/6951/automation-hat/poky/build/tmp/deploy/images/raspberrypi3-64

NOTE: on line 9 the build tells you that it is installing our two new packages into the image.

Flash this image to your µSD card, insert the card into the target, and apply power. After logging in:

1
2
3
4
5
6
7
root@raspberrypi3-64:~# python3
Python 3.8.5 (default, Jul 20 2020, 13:26:22) 
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import automationhat
>>> automationhat.output.one.on()
>>>

Looking at the target:


Success!

Finishing Up

We used devtool to help us create two recipes. Those recipes (and their sources) are still in our "workspace". We need to finish working with them and put them somewhere. But where? The layers we're using aren't under our control. We need to create our own layer so we have a place to store these new recipes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
$ bitbake-layers create-layer ../layers/devtool-additions
NOTE: Starting bitbake server...
Add your new layer with 'bitbake-layers add-layer ../layers/devtool-additions'

$ bitbake-layers add-layer ../layers/devtool-additions/
NOTE: Starting bitbake server...

$ devtool finish -f -r automation-hat ../layers/devtool-additions/
NOTE: Starting bitbake server...
WARNING: Source tree is not clean, continuing as requested by -f/--force
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)...
Loading cache: 100% |                                                                                                                                                                          | ETA:  --:--:--
Loaded 0 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################################################################| Time: 0:00:14
Parsing of 2043 .bb files complete (0 cached, 2043 parsed). 3214 targets, 123 skipped, 0 masked, 0 errors.
INFO: Updating SRCREV in recipe automation-hat_git.bb
INFO: Moving recipe file to /z/build-master/6951/automation-hat/poky/layers/devtool-additions/recipes-automation-hat/automation-hat
INFO: -r argument used on automation-hat, removing source tree. You will lose any unsaved work

$ devtool finish -f -r python3-sn3218 ../layers/devtool-additions/
NOTE: Starting bitbake server...
WARNING: Source tree is not clean, continuing as requested by -f/--force
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)...
Loading cache: 100% |###########################################################################################################################################################################| Time: 0:00:00
Loaded 3212 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################################################################| Time: 0:00:00
Parsing of 2043 .bb files complete (2041 cached, 2 parsed). 3214 targets, 123 skipped, 0 masked, 0 errors.
Removing 1 recipes from the cortexa53 sysroot: 100% |###########################################################################################################################################| Time: 0:00:00
Removing 1 recipes from the raspberrypi3_64 sysroot: 100% |#####################################################################################################################################| Time: 0:00:00
INFO: Updating SRCREV in recipe python3-sn3218_git.bb
INFO: Moving recipe file to /z/build-master/6951/automation-hat/poky/layers/devtool-additions/recipes-python3-sn3218/python3-sn3218
INFO: -r argument used on python3-sn3218, removing source tree. You will lose any unsaved work

$ rm -fr workspace/sources/

$ tree ../layers/devtool-additions/
../layers/devtool-additions/
├── COPYING.MIT
├── README
├── conf
│   └── layer.conf
├── recipes-automation-hat
│   └── automation-hat
│       └── automation-hat_git.bb
├── recipes-example
│   └── example
│       └── example_0.1.bb
└── recipes-python3-sn3218
    └── python3-sn3218
        └── python3-sn3218_git.bb

7 directories, 6 files

This looks good. The "example" boilerplate can be removed from the layer.

NOTE: if you want to build an image, you have to go back to using bitbake and you have to explicitly add the package(s) to your image. Thanks to dependencies, the only package that needs to be added to an image is automation-hat

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
MACHINE = "raspberrypi3-64"
DL_DIR = "/opt/Downloads"
DISTRO = "poky"

ENABLE_UART = "1"
ENABLE_I2C = "1"
KERNEL_MODULE_AUTOLOAD += "i2c-dev"
CORE_IMAGE_EXTRA_INSTALL += " \
        ${MACHINE_EXTRA_RRECOMMENDS} \
        wpa-supplicant \
        i2c-tools \
        automation-hat \
        "

PACKAGECONFIG_append_pn-gdb = " tui"
PACKAGECONFIG_append_pn-gdb-cross-canadian-arm = " tui"

IMAGE_FSTYPES_append = " wic"
IMAGE_FSTYPES_remove = "wic.bz2 tar.bz2"
WARN_QA_append = " version-going-backwards"
ERROR_QA_remove = "version-going-backwards"
BB_DANGLINGAPPENDS_WARNONLY = "yes"
INHERIT += "buildhistory image-buildinfo buildstats-summary"
BUILDHISTORY_COMMIT = "1"

PACKAGE_CLASSES ?= "package_ipk"
SDKMACHINE = "x86_64"
EXTRA_IMAGE_FEATURES ?= "debug-tweaks"
USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS ??= "\
    STOPTASKS,${TMPDIR},1G,100K \
    STOPTASKS,${DL_DIR},1G,100K \
    STOPTASKS,${SSTATE_DIR},1G,100K \
    STOPTASKS,/tmp,100M,100K \
    ABORT,${TMPDIR},100M,1K \
    ABORT,${DL_DIR},100M,1K \
    ABORT,${SSTATE_DIR},100M,1K \
    ABORT,/tmp,10M,1K"
PACKAGECONFIG_append_pn-qemu-system-native = " sdl"
CONF_VERSION = "1"


Posted by at
Labels: , , , , , , ,

9 Jun 2020

NAND Flash Basics

Introduction

NAND flash is a type of non-volatile data storage technology, and is often used on embedded devices much the same way a hard-drive would be used on a desktop machine.

NAND flash is built on cells. The original flash devices stored one bit of information per cell, and were called single-level cell (SLC) technology. Later two bits were stored per cell, so this became (unfortunately named) multi-level cell (MLC) technology. When the technology came along to store three bits per cell, it was named triple-level cell (TLC) technology. As you can see: "single" means one, "multiple" means two, and "triple" means three. According to wikipedia there now exist (or are in development) quad- devices with 4 bits per cell (QLC) and penta- (PLC) devices with 5.

Unlike traditional hard-drives (i.e. built on spinning platters), the individual cells used to store bits in NAND flash can not be twiddled indefinitely. New or newly-erased flash shows up as all bits set to 1. The act of writing data to a freshly erased NAND device is simply the process of changing the necessary bits from 1s to 0s. The moment the data you want to write needs to flip a 0 back to a 1, an erase cycle is needed. If, coincidentally, every time you needed to write data to a NAND all that was required was to flip bits from 1s to 0s, you would never need to perform an erase cycle. Flipping bits from 1s to 0s comes "for free", but erasing bits from 0s to 1s comes at a cost. Each time a bit is switched from a 0 to a 1 (i.e. erased) the oxide layer of the cell is degraded. Over time it will stop being possible to erase a cell.
 
Every device comes with a claim of being able to support a given minimum number of program/erase (P/E) cycles, but the maximum number is never known. In general SLC NAND supports a higher number of P/E cycles than MLC, and MLC supports more P/E cycles than TLC. In general the drop-off in P/E endurance between the different technologies is quite significant; on the order of magnitudes. You'll have to consult the datasheets for specific devices for the actual performance of any specific device, but in general SLC devices are good for ~50,000 to 100,000 P/E cycles, MLC ~1,000 to 10,000, and TLC under ~1,000 P/E cycles.

Internally, cells are combined to form bytes, bytes are combined to form pages, pages are gathered into blocks, and blocks are combined to form planes.

From outside a NAND device one can only access the data inside a NAND device one page at a time. If you want to read one byte of data, the entire page on which that byte is stored must be retrieved, then the specific byte of interest can be accessed. The same is true for writing; you can only write data to a NAND device one page at a time. More recent devices will often break a single page up into fixed, equal-sized sub-pages which can help when fetching or writing data. Erasing, however, needs to be performed a whole block at a time. Fancy devices can allow pages in separate planes to be read or written simultaneously. In general pages are for reading/writing, blocks for erasing, and operations can occur simultaneously to separate planes. You'll need to consult the datasheet of any specific device for each device's specific behaviour.

The sizes of pages, blocks, and planes on a NAND device varies by device. A typical device will have page sizes in the 2048-bytes/page range, will split a page into 4 sub-pages, will combine around 64 pages to form a block, and use roughly 1024 blocks for each plane.

Although storing more bits per cell improves storage density, there are trade-offs such as: slower access times, and reduced life expectancy of the device. So while TLC is newer than MLC and has some advantages over it, TLC doesn't displace MLC; MLC is still available. The same holds true for SLC; even though it is "older" technology, it is faster and has an order of magnitude or better P/E cycles over "newer" technology. Therefore SLC is still used quite extensively when the circumstances call for improved reliability or speed over size or cost. As you would expect SLC flash is the most expensive, with the price dropping with each increase in bits per cell.

ECC and OOB


In order to combat the strange situation of a device that will (by design) fail over time, in practice ECC checks are often calculated for the data that is stored on the device. In fact if you read the fine print in the datasheet regarding a device's P/E endurance you'll often find that these endurance claims are based on the expectation that ECC is being used. But if you're going to calculate an ECC for a chunk of data, where will you store this value? There's not much point to generating an ECC for a chunk of data if it isn't stored with the data so it can be checked later on. As such, each NAND flash device comes with extra storage space added to the device in order to store these ECC calculations (or any other book-keeping data you'd like to track). So if you buy (for example) a 512MB-sized NAND flash device, you might be given a device that has 512MB+8MB of storage. This extra area is referred to as the out of bounds or OOB area.

Typically ECCs are calculated per page of NAND data. When you ask a device for a page of data (the smallest size of data you can request from a NAND device) in addition to receiving that page worth of data you will also be given the data from that page's OOB area. The same is true for writing: when you want to write data to a NAND device, for each page you give it, you also have to provide the data for that page's OOB area.

By the way the answer is "yes it is" (if you're wondering whether or not the OOB area is susceptible to the same degradation as the rest of the data on a NAND device). One could add OOB areas to OOB areas ad infinitum, but one level is considered sufficient. A failure in either of the main data area or an OOB area is a failure for that page+OOB combination.

Raw vs FTL

Getting the most out of your NAND device requires not just ECC checks, but the ability to mark pages as bad, support for caching, tolerance for sudden power loss, wear-levelling, and other techniques that help it perform optimally and correctly. The addition of these algorithms is so necessary that many devices that use NAND flash internally will put a microcontroller and code between the user and the memory. These are called FTL or managed devices.

FTL stands for flash translation layer and it forms a level of indirection between what the user requests, and what actually happens to the memory. For example, the user might continuously read/modify/update one specific page over and over. But knowing that hammering on only one page might cause that page to wear out much faster than the rest of the device, internally the FTL actually maps the requests for modifications to a specific page that the user is commanding, to modifying a different physical page with every request. This lets the user think they are reading and writing the same page over and over while in fact those requests are being spread over the entire device so as to not wear out any one part of the chip prematurely. This called wear-levelling.

Managed devices will often present a different interface to the outside world than what one would expect from raw NAND. Therefore when working with a device that uses NAND technology, it's important to know whether you're dealing with a raw NAND chip, or one that is managed.

Most managed devices present themselves as hard-drives, so using them under Linux is simply a matter of having Linux treat the device the same as it would any other hard-drive. Once Linux is interfaced to it, you simply use it as any other hard-drive: you partition it, format with your favourite filesystem, mount it, then use it like normal.

If you're using a raw NAND device, then your best bet is to make use of raw NAND-handling software that is already available in order to deal with NAND's quirks. Under Linux, the MTD subsystem provides a uniform, though raw, interface to NAND. On top of the MTD subsystem you could use JFFS2, but it has been mostly superseded by UBI and UBIFS.

Unlike JFFS2, UBIFS can't sit directly on top of the MTD subsystem. It needs to sit inside a UBI container. It is UBI that has all the fancy logic and algorithms for managing the physical memory, providing fast boot times, handling power interruptions, wear-levelling, and so forth. UBIFS is a NAND-aware filesystem that sits on top of UBI with which it works well. A UBI can contain one or more UBIFS filesystems.

External Interface and ONFI

Interacting with NAND has been simplified and standardized thanks to the efforts of ONFI, the Open NAND Flash Interface. All NAND devices have either an 8- or 16-bit parallel I/O bus, plus a number of standardized control lines. Telling the NAND device what you want to do is simply a matter of some combination of giving it a command, providing it with an "address", then reading or writing the data. The I/O bus is multiplexed between commands, "addresses", and data. The NAND device distinguishes between these pieces of information based on the cycle and on the values of the logic levels on the control lines.

I use "address" in quotes because the address of any piece of data in a NAND device is not referenced the same way a piece of data would be referenced in, for example, RAM. As I've mentioned a couple times, you can only interface with a NAND device one page at a time. Telling the NAND device which page you want is a matter of specifying its column, its page address within a block, its block number, and specifying which plane it's on. The bits specifying this information are jumbled up together and sent to the device as either 3, 4, or 5 (depends on the device) 8-bit words (regardless whether the device has an 8- or 16-bit I/O interface) which are sent in subsequent I/O cycles during the "address" phase.

ONFI also specifies the bit patterns of the various commands that can be issued to a NAND device. In this way Linux's MTD software (for example) doesn't need to be chip-specific with regards to the command definitions.

Timing Charts

Like most pieces of silicon, NAND devices don't operate infinitely quickly; they certainly don't operate at the speed of the bus connecting your SoC to the NAND device. As such, one of the most important pieces of information contained in your device's datasheet is the table specifying minimum or maximum timings of various operations.

This table is usually found in a section called "AC Characteristics" and includes the timing information for around 3 dozen or so parameters. For example, the Address Latch Enable setup time is given as tALS. Sometimes the timing is specified as a minimum amount of time that one needs to wait for an event, other times it specifies a maximum time. Each parameter has an associated unit, usually nano-seconds, but sometimes micro-seconds.

Some parts of a NAND's datasheet aren't as important as others from a software point of view. But when working with a NAND chip at a low level, the timing information is certainly one of the more important sections.

SoCs and NAND Controllers

It would be pretty rare to see a micro-controller connected to a NAND device directly using nothing but GPIO lines. Part of the difficulty in controlling a NAND device directly would be to get the timing right and efficient. As such most SoCs include a dedicated NAND Controller.

The job of the NAND Controller is to handle the interaction between the SoC and the NAND so that the SoC is freed from the lowest-level details of handling the NAND; like a sort of buffer. The SoC creates a request by loading the controller's registers with the correct values, and it's the controller's job to twiddle the various control and I/O lines in the correct sequence, at the right times.

An SoC's NAND Controller will often incorporate logic for handling ECC calculations and manipulating portions of the OOB areas as appropriate. For example, I mentioned earlier that when providing data to the NAND, one must also supply the OOB area. In some cases the software only needs to provide the data, and the controller will calculate the ECC and supply the OOB data to the NAND device itself. The reverse also applies when reading data: the controller can be instructed to check the ECC and it will either correct the data itself (if it can, if an error is detected) or set flags to let the user know an issue was found (or both). In which case the user simply receives a page of corrected data.

The NAND Controller can only do its job properly if it is configured properly. For each SoC that has a NAND Controller, a portion of its registers need to be used to give the user a place to specify the configuration and timing parameters of the specific NAND device being used.

Configuration usually involves telling the NAND Controller the bus width (8 or 16), the page size, whether or not sub-pages are used, how many bytes to use when specifying the "address" (3, 4, or 5), and various other things.

For timing, the datasheet for the NAND device will always specify timing in absolute, "wall clock" values (e.g. 25[ns]), whereas the NAND Controller only knows how to count clock ticks. Therefore not only do you need to know the clock rate of the bus to which the NAND device is connected (which is almost guaranteed to not be the same as the clock rate of the CPU itself), but these values will need to be adjusted anytime the clock rate changes (e.g. in low power or power-saving modes). Telling the controller how many clock ticks to wait will always be specified as an integer number. Knowing your clock rate, you'll need to figure out how many ticks are required to get at least that much delay, then round up. For example a given timing parameter might specify a minimum delay of 25[ns], at a clock rate of 130[MHz] this would translate to 3.25 clocks. But since the controller can't count a quarter of a clock, this value needs to be rounded up to 4. At this clock rate 4 clocks actually gives a delay of 30.7[ns], but we can't specify 3 otherwise the controller won't wait long enough for the NAND device, and errors will result.

Unfortunately it's rare to find a controller that has a 1:1 mapping between the timing parameters provided in the NAND device's datasheet and the timing parameters required by the NAND Controller. For reasons that can only be described as masochistic, the NAND Controller will almost always want timing values that are calculations that, if you're lucky, will be based on values found in the NAND device's datasheet. For example, a typical device's datasheet will (thankfully) provide a timing parameter called tRHZ. But instead of asking for this value, the NAND Controller might say: I need NAND_TA and you calculate NAND_TA as:

((RD_HIGH - RD_LOW)/HCLK) + (NAND_TA/HCLK) ≥ tRHZ

RD_HIGH and RD_LOW are other timing parameters the controller wants, which you've already calculated in a manner similar to the above, but you must re-arrange the inequality to isolate NAND_TA. Thankfully tRHZ is found in the datasheet; sometimes the controller will request a parameter that isn't in the datasheet and you're left trying to figure out how to use the parameters the datasheet gives you to determine the value the controller wants.

Also, the above calculations depend on your ability to figure out the clock rate which requires an understanding of the clocking and PLL mechanisms of your SoC, which isn't trivial either.

Conclusion

NAND flash is an interesting technology, with its own advantages and quirks. To get NAND working on a specific device requires an understanding of the details of the specific NAND device you're using, as well as understanding the capabilities and limitations of your SoC.
Posted by at
Labels:

28 Apr 2020

The LPC32xx Project - introduction

For the last while I've been working on an extremely exciting project at work; certainly one of the best jobs to come my way. I've been lucky in my career to have worked on some really exciting things, and this project is certainly one of them.

We have a customer who has a range of hardware from some very old stuff to some very new stuff. They want the same versions of bootloader, kernel, and userspace running on all of them. Additionally they want, for each device, A/B updates, each of which is to run in its own container, and all built with OpenEmbedded/Yocto.

I'm not at all worried about finding support for the newish stuff. It's actually the oldest hardware that needs the most attention. For example, look at the mailing lists for the Linux kernel, U-Boot, or anything graphics-related and you'll find hundreds of patches from many developers every day all working on hardware so cutting edge, some of it isn't even available for purchase yet. But many fewer people are making sure the old stuff is still working. At best a developer will make sure their changes won't cause support for an older device to stop compiling, but sometimes that's not enough.

Earlier in my career I had an opportunity to do board bring-up. The company I was working for had just created their own custom board. It was based around a variant of the AMCC PowerPC 440 SoC. Our board, thankfully, was very closely modelled on the reference board for the specific SoC we were using, but with two key differences:
  1. we were using a brand of SDRAM that differed from the one on the reference board (and therefore the timing parameters needed tweaking), and
  2. whereas the reference board was running at a middle-of-the-road clocking, we wanted to run our board at the highest clock rate possible
Although upstream U-Boot already had support for the reference board, it was my job to update it to work on our board by completing these two tasks. Before starting this job, I hadn't even heard the phrase "board bring-up", but I was hooked! If I could have, I would have plotted a career from that time on that would have included a lot more board bring-up activities! But you take what you can get, and there are some other exciting things to do other than board bring-up.

What I enjoy most about board bring-up is it provides an opportunity to get deep down into the details of how an SoC works. We're all aware of SDRAM, and flash, and DMA, and all of the dozens of other pieces that fit together to make an SoC. But it's not every day one gets the opportunity to examine these things at the register level. Getting these two tasks working was very exciting for me; definitely a highlight.

Here I am, years later, and I find myself doing board bring-up again. However this time it's not with cutting-edge hardware, but rather with really old hardware: the NXP LPC32xx SoC. I realize it's strange for me to claim to be doing "board bring-up" for an SoC that clearly already has support in U-Boot and the Linux kernel. However the fact is the support for this device was added decades ago and (especially in the case of U-Boot) has bitrotted quite badly in the interim.

U-Boot is a very current and exciting project! It's not unusual for the daily patch count to run well into the hundreds. There are always new boards and SoCs in need of support, plus there are ongoing projects to improve the underlying structure of the code, its build system, and its test/ci infrastructure. With a code base moving this fast, an older device can quickly fall out of step with the rest of the project.

I've been busily working on this project for a while and enjoying every minute. I'll be blogging more about it!
Posted by at
Labels: , , , , ,

21 Oct 2019

OE Floating-Point Options for ARMv5 (ARM926EJ-S)

One of the (many) things I enjoy about OpenEmbedded is how easy it is to try out different configurations. Want to switch from sysvinit to systemd? Change the config, re-build, and there's your new image to test. Want to switch from busybox to coreutils? Change the config, re-build, and there's your new image.

Recently, I have been working with an ARMv5 device that was released in 2008: the NXP LPC3240 which is based on the ARM926EJ-S SoC. The specific device I'm using has a VFPv2 unit, however since the VFP was optional on the ARM926EJ-S, most distros/images are built with no floating-point support. From the standpoint of binary distributions, this makes the most sense: if you want to supply a binary to run on the most number of devices, build for the lowest common denominator. But when building your own distro/images from source using OpenEmbedded, you have the flexibility to tweak the parameters of your build to suit the specifics of your hardware.

Nowadays, a user has 3 choices when it comes to VFP on the ARM926EJ-S:
  1. soft: floating-point emulated in software (no hardware floating-point)
  2. softfp: enable hardware floating-point but have floating-point parameters passed in integer registers (i.e. use the soft calling conventions)
  3. hard: enable floating-point and have floating-point parameters passed in floating-point registers (i.e. use FPU-specific calling conventions)
The naming of option 2 (softfp) is unfortunate. To me, saying "soft floating-point" implies the floating-point is being emulated in software. However, its name was meant to contrast its calling convention with that of hard floating-point, not to imply the floating-point is being emulated in software.

By default in OpenEmbedded, specifying tune-arm926ejs.inc sets the DEFAULTTUNE to "armv5te" which disables VFP. By tweaking DEFAULTTUNE in your machine.conf file (or local.conf) you can try out all the options. Personally, when setting DEFAULTTUNE, I also like to tweak TUNE_CCARGS.

To try out the different options, set the following parameters:
  1. soft:
    DEFAULTTUNE = "armv5te"
    TUNE_CCARGS = "-mcpu=arm926ej-s -marm"
  2. softfp:
    DEFAULTTUNE = "armv5te-vfp"
    TUNE_CCARGS = "-mcpu=arm926ej-s -mfpu=vfp -mfloat-abi=softfp -marm"
  3. hard:
    DEFAULTTUNE = "armv5tehf-vfp"
    TUNE_CCARGS = "-mcpu=arm926ej-s -mfpu=vfp -mfloat-abi=hard -marm"
The meta-openembedded/meta-oe layer provides a number of recipes for benchmark applications. Interesting performance benchmark programs include: whetstone, dhrystone, linpack, nbench, and the "cpu" test of sysbench.

STD BENCHMARK DISCLAIMER: when it comes to benchmarks it's always important to remember that they are synthetic. That is: they are programs created to measure the performance of some artificial work-load of their choosing. If you want to know how the performance of your program will change under different settings, the only real way to determine that is to build and test your specific program under the different settings. It's also worth pointing out that during the era when benchmark programs were a really hot topic (late 90's-ish?) many vendors would tailor their hardware towards the popular benchmark programs of the time, skewing the results dramatically. In other words, a specific piece of hardware would be tuned to run a specific benchmark really well, but "real" workloads wouldn't see much improvement. Therefore YMMV.

For this experiment I created three images; each one built using one of the three floating-point tunings given above but all containing the same contents and the same versions of all the contents. I then loaded each of the images on my hardware in turn, so I could run the benchmark programs to generate performance data.

As of the time these images were built (Oct 11, 2019), the HEAD revision of openembedded-core was 59938780e7e776d87146002ea939b185f8704408 and the head revision of meta-openembedded/meta-oe was fd1a0c9210b162ccb147e933984c755d32899efc. At that time, the compiler being used was gcc-9.2, and the versions of various components are: glibc:2.30, bash:5.0, dhrystone:2.1, linpack:1.0, nbench:2.2.3, sysbench:0.4.12, and whetstone:1.2.

First Impressions

One of the first interesting things to note is the size of the various binaries:


 soft softfp hard




whetstone 33,172 20,236 20,444




dhrystone 13,752 9,660 9,660




sysbench 81,268 77,176 77,176




linpack 13,744 9,652 9,652




nbench 47,308 43,216 43,216

    Looking at the disassembly of each of these binaries, it's not hard to see why this is. Disassembling the binaries is as simple as:
    $ arm-oe-linux-gnueabi-objdump -d whetstone
    While the softfp and hard programs are filled with VFP instructions (e.g. vldr, vmul.f64, vsub.f64, etc.) the soft program contains calls to various __aeabi_* functions and __adddf3. These functions come from libgcc, a library written by the gcc people to help shore up things that are missing from standard C libraries (such as software emulation of floating-point, see here for more info). Interestingly, the code of these functions is linked into the executable itself (and not as a shared library). As you can imagine, emulating floating-point operations in software is going to take a lot of code!

    If you have floating-point hardware, taking advantage of it will shrink the size of your executables (if they use floating-point math).

    Whetstone

    whetstone is a benchmark program whose primary purpose is to measure floating-point performance. In each image I ran the whetstone program 5 times, timing each run with time, and had it run 1,000,000 loops:
    # time whetstone 1000000
    The averages of each test are as follows. Higher MIPS is better, lower time is better:

    soft softfp hard
    MIPS duration [s] MIPS duration [s] MIPS duration [s]
    100.16 998.4 1872.84 53.4 1872.84 53.4

    Dhrystone

    dhrystone is a benchmark used to evaluate integer performance. In each image I ran the whetstone program 5 times, timing each run, and performing 1,000,000 iterations per run:
    # time echo 1000000 | dhry
    The averages are as follows. Higher dhry/sec is better, lower time is better:

    soft softfp hard
    dhry/sec duration [ms] dhry/sec duration [ms] dhry/sec duration [ms]
    432527.22 2.3 431037.7 2.3 429554.58 2.3

    Sysbench (cpu)

    sysbench is a benchmark which includes a bunch of sub-benchmarks, one of which is the "cpu" test. On each image I ran the cpu test 5 times, capping the run-time to 300[s]. The benchmark appears to perform prime factorization, measuring something called "events", and recording run time per event.
    # time sysbench --max-time=300 --test=cpu run
    soft softfp hard
    events duration/event [ms] events duration/event [ms] events duration/event [ms]
    1157.2 259.29 2951.6 101.638 2951 101.662


    As a final test, on each image I ran the cpu test just once without a time limitation, to see how much time it would otherwise take.
    # time sysbench --test=cpu run
    soft softfp hard
    events test duration events test duration events test duration
    10000 43m0.50s 10000 16m56.499s 10000 16m56.777

    Linpack

    linpack is a benchmark testing a computer's ability to perform numerical linear algebra. The program takes one required parameter: the size of the array to use. If you pass "200", it will calculate a 200x200 array. As it runs, it determines how many repetitions to perform, it bases the repetitions on its performance. For each repetition it records how much time it took. When it's done a set of repetitions, it calculates a KFLOPS count, then starts over with a different repetition count.

    For each image I ran the program once with "200" and once with "500". With no hardware floating point support calculating a 200x200 array it starts with 1 repetition, then tries 2, then 4, 8, etc. With hardware floating-point on a 200x200 array it starts with 8 repetitions, then 16, 32, etc. On a 200x200 array the repetition counts common to all images are 8, 16, and 32. On a 500x500 array the repetition counts common to all images are 1 and 2.

    The program never terminates; it keeps increasing the repetition count and going until explicitly killed.
    # echo 200 | linpack
    soft softfp hard
    reps time/rep KFLOPS reps time/rep KFLOPS reps time/rep KFLOPS
    8 4.3 2718.669 8 0.64 18553.356 8 0.62 19223.389
    16 8.6 2718.614 16 1.29 18552.917 16 1.25 19214.278
    32 17.2 2718.792 32 2.58 18552.361 32 2.49 19212.128
    # echo 500 | linpack
    soft softfp hard
    reps time/rep KFLOPS reps time/rep KFLOPS reps time/rep KFLOPS
    1 8.1 2674.928 1 1.38 15876.865 1 1.38 15883.324
    2 16.17 2674.871 2 2.74 15878.365 2 2.74 15882.516

    nbench

    nbench (aka BYTEmark) runs a bunch of sub-tests (including: numerical sort, string sort, bitfield, fp emulation, fourier, assignment, IDEA, huffman, neural net, and LU decomposition) then generates both an integer index and a floating-point index. These indices are relative to what were considered capable machines of the time (mid-1990's).

    This benchmark was run twice on each image, the averaged results are:

    soft softfp hard


    integer idx fp idx integer idx fp idx integer idx fp idx


    1.054 0.1 1.1095 0.961 1.109 0.979

    Conclusions

    Since software floating-point emulation gets added statically to C programs, using hardware floating point makes binaries smaller in programs that perform floating-point calculations. Enabling floating-point in such programs also improves the performance of floating-point operations noticeably. Interestingly, it appears as though integer performance is ever so slightly impacted in the hard case relative to softfp. Therefore it would seem to be that if your entire work-load is floating-point, then go with hard, otherwise if there is both floating-point and considerable integer calculations, softfp might be best.

    As always, test your own application to know which mode is best in your scenario.
    Posted by at
    Labels: , , , , , , , , , , , ,
    Subscribe to: Posts (Atom)