Showing posts with label open-embedded. Show all posts
Showing posts with label open-embedded. Show all posts

30 Sept 2015

OE Build of glmark2 Running on Cubietruck with Mali

Here are the steps you can perform to use OpenEmbedded to build an image for the Cubietruck in order to run the glmark2-es2 benchmark accelerated via the binary-only mali user-space driver.

First off, grab the scripts that will guide you through your build:

$ git clone git://github.com/twoerner/oe-layerindex-config.git
Cloning into 'oe-layerindex-config'...
remote: Counting objects: 45, done.
remote: Total 45 (delta 0), reused 0 (delta 0), pack-reused 45
Receiving objects: 100% (45/45), 11.04 KiB | 0 bytes/s, done.
Resolving deltas: 100% (24/24), done.
Checking connectivity... done.


Kick off the build configuration by sourcing the main script

$ . oe-layerindex-config/oesetup.sh

This script will use wget to grab some items from the OpenEmbedded layer index, then ask you which branch you would like to use:

Choose "master".

Then you'll be asked which board you'd like to build for:





 Scroll down and choose "cubietruck". At this point the scripts will download meta-sunxi for you and configure your build to use this BSP layer by adding it to your bblayers.conf.

Now you need to choose your distribution:




Select "meta-yocto". Now this layer will be fetched and added to your build. Since meta-yocto includes a number of poky variants we will be asked to choose one:




Select "poky".

We now need to tell our build where it can place its downloads (notice the question at the bottom of the following screen):

You can enter your download directory. This entry box uses readline which, among other things, allows you to use tab-completion when entering your download path. I would recommend using a fully-qualified path for this information. It is common to use one location for all the downloads of all builds taking place on a given machine.

Now your configuration step is mostly finished:




If you simply wanted to perform a small core-image-minimal build, you could go ahead and bitbake that up now. But we want to add glmark2 to our image, and its metadata isn't found in the layers we currently have. Therefore we need to add one more layer:




Before we can build, we need to tweak our configuration to let bitbake know which kernel we want to use, which u-boot we want to use, what packages we want to add to our image, and so forth. Edit conf/local.conf and make yours look similar to:




Now we are ready to build!

$ bitbake core-image-full-cmdline




When our build completes there are some warnings that are issued, we can safely ignore these for now:




We will find our artifacts in:




Simply dd the sdimg to a microSD card:

# dd if=core-image-full-cmdline-cubietruck.sunxi-sdimg of=/dev/<your microSD device> bs=1M

Note that on some distributions you have to be the root user to perform the above step and that you need to figure out what jumble of letters to fill in for the <your microSD device> part.

Pop the microSD into the cubietruck's microSD slot, attach an HDMI monitor, a 3.3V console cable, and apply power.

When the device finishes booting you can now run the glmark2-es2 demo. Unfortunately it only works in full-screen mode. If you want to see which tests it is currently running and the current FPS count while the tests are running you can specify the --annotate option.

There are two common ways to do this:

  1. plug a USB keyboard into the cubietruck and type the command into the matchbox-terminal console
    • # glmark2-es2 --fullscreen --annotate
  2. from the serial console run:
    • # export DISPLAY=:0
    • # glmark2-es2 --fullscreen --annotate



There is also a "sunximali-test" demo app you can try.


Build Help

If you're having trouble building there are a couple things you can try

  • If this is your first time using OpenEmbedded or you're rather new to it, you could try familiarizing yourself with the project and more basic builds to start:
    •  https://www.yoctoproject.org
    •  https://www.yoctoproject.org/documentation
    •  http://www.yoctoproject.org/docs/1.8/yocto-project-qs/yocto-project-qs.html
  • In the build screenshot you can see the exact repositories my build is using, and what the latest commit was for each. You could try checking out those exact same commits for each of the layers being used to see if that helps.
The build configuration for this build is:
Build Configuration:
BB_VERSION        = "1.27.1"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "openSUSE-project-13.2"
TARGET_SYS        = "arm-poky-linux-gnueabi"
MACHINE           = "cubietruck"
DISTRO            = "poky"
DISTRO_VERSION    = "1.8+snapshot-20150930"
TUNE_FEATURES     = "arm armv7a vfp neon callconvention-hard vfpv4 cortexa7"
TARGET_FPU        = "vfp-vfpv4-neon"
meta-sunxi        = "master:14da837096f2c4bf1471b9cce5cf7fd30f55999b"
meta              = "master:4a1dec5c61f73e7cfa430271ed395094bb262f6b"
meta-yocto        = "master:613c38fb9b5f20a89ca88f6836a21b9c7604e13e"
meta-oe           = "master:f4533380c8a5c1d229f692222ee0c2ef9d187ef8"


The conf/local.conf is:
PREFERRED_PROVIDER_virtual/kernel = "linux-sunxi"
PREFERRED_PROVIDER_u-boot = "u-boot-sunxi"
PREFERRED_PROVIDER_virtual/bootloader = "u-boot-sunxi"
DEFAULTTUNE = "cortexa7hf-neon-vfpv4"
CORE_IMAGE_EXTRA_INSTALL = "packagegroup-core-x11-base sunxi-mali-test glmark2"
MACHINE_EXTRA_RRECOMMENDS = " kernel-modules"
IMAGE_FSTYPES_remove = "tar.bz2"
DISTRO_FEATURES_append = " x11"
PACKAGECONFIG_pn-glmark2 = "x11-gles2"
PREFERRED_PROVIDER_jpeg = "jpeg"
PREFERRED_PROVIDER_jpeg-native = "jpeg-native"
PACKAGE_CLASSES ?= "package_ipk"
EXTRA_IMAGE_FEATURES = "debug-tweaks"
USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
OE_TERMINAL = "auto"
BB_DISKMON_DIRS = "\
    STOPTASKS,${TMPDIR},1G,100K \
    STOPTASKS,${DL_DIR},1G,100K \
    STOPTASKS,${SSTATE_DIR},1G,100K \
    ABORT,${TMPDIR},100M,1K \
    ABORT,${DL_DIR},100M,1K \
    ABORT,${SSTATE_DIR},100M,1K"
PACKAGECONFIG_append_pn-qemu-native = " sdl"
PACKAGECONFIG_append_pn-nativesdk-qemu = " sdl"
ASSUME_PROVIDED += "libsdl-native"
CONF_VERSION = "1"

8 May 2015

OE Builds in a VM

Everyone knows that doing an OE build can take a bit of time (there are good reasons for this being true) so it follows that performing an OE build in a VM will take even longer. But when you do a build "natively" you get to potentially use all of the computer's free memory and all of its processing power. A VM running on that same machine will, at most, only allow you to use half its processing power and memory.

The question I wanted to answer was: if I performed a build "natively" but only used half the memory+cpu of a computer, how would that compare to a build in a VM that thought it was using the full computer's resources but had been constrained to only half the resources due to the use of the VM?

When performing an OE build there are two variables you can set that allow you to restrict the amount of cpu resources that are used: PARALLEL_MAKE and BB_NUMBER_THREADS. But using these variables isn't really the same as building on a machine with half the processing resources; the initial parse, for example, will use all available cpu resources (it can't know how much you want to restrict the build until it has actually completed parsing all the configuration files). Plus, there are no OE variables you can tweak to say "only use this much memory during the build".

So in order to better measure a VM's performance we need to find a better way to perform a restricted "native" build (other than just tweaking some configuration parameters). Answer: cgroups.

To be honest, I was hoping that I could demonstrate that if I used qemu+kvm and used virtio drivers everywhere I could (disk, network, etc) that the performance of a VM would at least approach that of doing a "native" build that was restricted via cgroups to the same amount of resources as was available to the VM. My findings, however, didn't bear that out.

First I'll present my results (since that's really want you want to see), then I'll describe my test procedure (for you to pick holes in ;-) ). I tried my tests on two computers and ran each test 5 times.

First I ran a build on the "raw/native" computer using all its resources (C=1):

Computer 1Computer 2
100:24:1901:02:42
200:25:0301:02:19
300:24:5101:02:34
400:24:5501:02:21
500:25:0001:02:57
avg00:24:5001:02:35

I'm using "C" to represent the CPU resources; "1" meaning "all CPUs", i.e. using a value of "oe.utils.cpu_count()" for both PARALLEL_MAKE and BB_NUMBER_THREADS. A value of "0.5" means I've adjusted these parameters to be half of what "oe.utils.cpu_count()" would give.

Then I ran the same build again on the "native" machine but this time using BB_NUMBER_THREAD/PARALLEL_MAKE to only use half the cpus/threads (C=0.5):

Computer 1Computer 2
100:24:2000:57:10
200:24:3400:57:32
300:25:1100:57:08
400:24:3901:03:31
500:24:3101:01:50
avg00:24:3900:59:26

Counter-intuitively, when restricting the resources, the builds performed ever-so-slightly better than allowing the build to use of all the computer's resources. Perhaps these builds aren't CPU-bound, and this set just happened to come out slightly better than the "full resources" builds. Or (for this workload) these Intel CPUs aren't really able to make much use of CPU threads, it's CPU cores that count. (??)

Then I performed the same set of builds on the "native" computers, but after having restricted their resources via cgroups.

C=1 (restricted by half via cgroups, same for memory):

Computer 1Computer 2
100:28:0501:05:57
200:28:1801:05:33
300:28:2101:05:46
400:28:1401:05:22
500:27:3701:05:37
avg00:28:0701:05:39

C=0.5 (further restricted by half (again) via cgroups, same for memory):

Computer 1Computer 2
100:27:2001:04:08
200:27:1501:07:39
300:27:0801:07:42
400:27:1701:07:57
500:27:1301:08:16
avg00:27:1501:07:08

So there's obviously a difference between restricting a build's resources via BB_NUMBER_THREAD/PARALLEL_MAKE versus setting hard limits using cgroups. That's not too surprising. But again there's very little difference between using "cpu_count()" number of CPU resources versus using half that amount, in fact, for Computer 1 the build time improved slightly.

Now here's the part where I used a VM running under qemu+kvm. I had been hoping these times would be comparable to the times I obtained when restricting the build via cgroups, but that wasn't the case.

C=1 (restricted by half via VM, same for memory):

Computer 1Computer 2
100:41:3601:45:42
200:41:2201:47:52
300:41:4101:44:31
400:41:1601:50:25
500:41:1201:41:41
avg00:41:2501:46:02

C=0.5 (further restricted again by half via VM, same for memory):

Computer 1Computer 2
100:42:0201:30:23
200:42:0701:34:43
300:43:0501:31:14
300:43:1401:36:46
400:42:1201:42:05
avg00:42:3201:35:02

Analysis

Using the first build as a reference (letting the build use all the resources it wants on a "raw/native" machine):
  • constraining the build to use half the machine's resources via cgroups results in build times that are from 4.90% to 13.22% slower.
  • performing the same build in a qemu+kvm VM results in build times that are from 59.90% to 72.55% slower.

Specifics

  • bitbake core-image-minimal
  • fido release, e4f3cf8950106bd420e09f463f11c4e607462126, 2138 tasks
  • DISTRO=poky (meta-poky)
  • a "-c fetchall" was performed initially, then all directories but "conf" were deleted and the timed build was performed with "BB_NO_NETWORK=1"
  • between each build everything would be deleted except for the "conf" directory; therefore no sstate or tmp or cache etc
  • /tmp implemented on a tmpfs
  • because the VMs had limited disk space, all builds were performed with "INHERIT+=rm_work"
  • to help load/manage the VMs I use a set of scripts I created here: https://github.com/twoerner/qemu_scripts
  • in the VMs, both the "Download" directory and the source/recipes are mounted and shared from the host (using virtio)
To restrict a build using cgroups I created a cgroup named "oebuild" in both the "cpuset" and "memory" groups and placed my shell in them.

So, for example, say I'm running a shell, bash, and its PID is 1234. As root goto /sys/fs/cgroup and:
  • mkdir cpuset/oebuild
  • mkdir memory/oebuild
If your system has 8 CPUs and 16GB of RAM (adjust your numbers accordingly):
  • echo "0-3" > cpuset/oebuild/cpuset.cpus
  • echo 0 > cpuset/oebuild/cpuset.mems
  • echo 8G > memory/oebuild/memory.limit_in_bytes
And finally ("1234" is the PID of the shell I want to put in the "oebuild" cgroup, I then use this shell to run the build):
  • echo 1234 > cpuset/oebuild/tasks
  • echo 1234 > memory/oebuild/tasks
To confirm your shell is now running in this cgroup, run the following command in your shell:
$ cat /proc/self/cgroup
10:hugetlb:/
9:perf_event:/
8:blkio:/
7:net_cls,net_prio:/
6:freezer:/
5:devices:/
4:memory:/oebuild
3:cpu,cpuacct:/
2:cpuset:/oebuild
1:name=systemd:/user.slice/user-1000.slice/session-625.scope
Here you see both the "memory" and "cpuset" cgroups are constrained by the oebuild cgroup for this process (i.e. /proc/self).

6 Jan 2015

The Yocto Project: Introducing devtool - updates!

Now that The Yocto Project's devtool is in the main trunk, some of the instructions from a previous post need to be updated. Here is a link to the updated instructions, enjoy!

29 Dec 2014

Generating Yocto Documents in openSUSE 13.2

If you want to keep up-to-date with the latest Yocto Project documentation, you should be cloning git://git.yoctoproject.org/yocto-docs.

Once cloned, you can build (for example) a PDF version of the Developer's Manual by:
$ pushd documentation
$ make pdf DOC=ref-manual
$ popd
However, for some reason when I try this on my openSUSE 13.2 system I get:
cd dev-manual; ../tools/poky-docbook-to-pdf dev-manual.xml ../template; cd ..
warning: failed to load external entity "http://docbook.sourceforge.net/release/xsl/current/template/titlepage.xsl"
cannot parse http://docbook.sourceforge.net/release/xsl/current/template/titlepage.xsl
A quick work-around for this is to (temporarily) uninstall the docbook-xsl-stylesheet package:
#  rpm -e --nodeps docbook-xsl-stylesheets
Once done building the Yocto Project documentation simply re-install the package:
# zypper install docbook-xsl-stylesheets

22 Dec 2014

The Yocto Project: Introducing devtool

Paul Eggleton, and a group of his co-workers at the Intel Open Source Technology Centre, have been working on a new tool to help developers and build engineers work better and more efficiently together. Integrating The Yocto Project into a developer's workflow has traditionally been a source of pain, and has held back some projects from adopting The Yocto Project and its tools.

I have written a quick tutorial on this new tool (called devtool) which you can find here. Enjoy!

2 May 2014

ELC2014 Report - OE/Qt

On Tuesday, at the Linux Foundation's Embedded Linux Conference in San Jose, Denys Dmytriyenko gave a fabulous talk entitled "Qt5 & Yocto - adding SDK and easy app migration from Qt4" where he talked about some work he has been doing and is hoping to present for inclusion into the OE mainline.

In essence, if you have a Qt application that you want to add to your OE build and you want it to work regardless of whether your build is adding Qt4 or Qt5 to your device, you will most definitely want to take a look at the work Denys is doing. Specifically have a look at meta-arago and the qt-provider.bbclass and qt-vars.bbclass.

He then discussed how someone would use the various SDK options to work on their Qt code in conjunction with OE. I won't repeat that information here since it's all readily available in the docs and various other places.

25 Mar 2014

Using bmaptool To Create A Memory Card

Here's the scenario: I have just used OE to build a core-image-minimal which I want to run on my Wandboard-dual, I insert my 4GB microSD card into my desktop, use dd to write the image to the card, insert the card into my board, boot, and get:

Size=62.0M Used=19.1M

But it's a 4GB card?! Where's the rest of my disk?

OE has no idea how big of a card you want to use, so by default it makes an image that is just a bit bigger than required (or 8MB, whichever is larger).

Writing this small image is quick:

0m18.450s

If I want to use (roughly) the entire 4GB card I simply ask OE to build an image of that size. Edit conf/local.conf and add/edit:
IMAGE_ROOTFS_SIZE = "3800000"
Now when I build my image, the output from OE will be roughly 3.7GB in size. Writing this image to a card will take much longer:

49m57.326s

The funny thing is, the data hasn't changed; I'm still using the same amount of data on the card. What has changed is that I now have access to (roughly) the entire card, but at the cost of having it take ~160 times longer to write the image!

Size=3.4G Used=86.9M

In this case we're wasting lots of time (and flash write cycles) writing empty portions of the image to the disk. This is where the bmaptools come in. In essence, bmaptool looks at your image and determines which parts are important, and which are empty. When you go to actually write your image, only the non-empty parts are transferred -- saving you lots of write time (and flash cycles).

Using bmaptool is a two-step process:
  1. use bmaptool create to create a mapping file
  2. use bmaptool copy to write your image to a disk (with the help of the mapping file you just created)
Applying bmaptool to our 4GB image:

13m54.933s

It's not the 18 seconds from above (i.e. dd'ing the 80MB image), but it's still better than the 49 minutes required to dd the 4GB image. The image written with bmaptool works:

Size=3.4G Used=86.9M

Note that if I use bmaptool on the first (80MB) image, there isn't much savings:

0m17.826s

The real benefits are seen when trying to write an image such that most of the card is then available for use, and most of the image to be written is empty.

21 Jan 2014

OE/Yocto Bug Weekend - Jan 17 to 20, 2014 -- RESULTS

"Had a bug sprint week last week, resolved 26 bugs, better than we normally do."
-- https://www.mail-archive.com/yocto@yoctoproject.org/msg17312.html
Awesome!! Thanks to everyone who participated, especially on such short notice :-D

17 Jan 2014

OE/Yocto Bug Weekend - Jan 17 to 20, 2014

Starting today the OE an Yocto projects are having a bug squashing weekend! The purpose is to raise awareness of the ever-increasing open bug counts and to inspire people to take a look at the issues and see if they can work on fixing one or more of them.

https://bugzilla.yoctoproject.org/

Please take the opportunity to have a look at the open issues in the bugzilla database. If you don't have an account, please consider signing up. Play with the "Search" capability and see if there are issues to which you might want to contribute.

Obviously if you're a maintainer or some sort of a developer there are issues you could consider addressing. But there are also lots of issues an OE/Yocto "user" could investigate as well -- for example there are documentation issues, there are several issues in the "NEEDINFO" state, and sometimes just being able to reproduce a bug (or not) and confirm (or not) that an issue can be demonstrated on more than one host can be valuable information for the person who does eventually get assigned to solve the problem.

There are usually plenty of friendly, knowledgeable people around who can help. You can use the mailing lists, IRC channels, or bugzilla itself to communicate.

Thanks for your participation!

29 Dec 2013

Using OE/Yocto to create VMs

By default most OE builds will create a filesystem image, a kernel, and (perhaps) a bundle of kernel modules. This set of outputs makes it easy if you're looking to flash your build into real hardware or run it in emulation via qemu (i.e. runqemu). But what if you're looking to run your image in a virtual machine?

To run an image in a virtual machine you'll need a virtual disk image which contains a partition map and a bootloader. Fortunately OE provides support for generating one such type of virtual disk image: the vmdk.

Configure OE to Generate a VMDK

 

In order to get OE to generate a vmdk artifact simply edit your conf/local.conf to add vmdk to IMAGE_FSTYPES. Unfortunately, due to the way in which variables are handled in bitbake, it isn't possible to use the _append operator, only the += operator can be used:
IMAGE_FSTYPES += vmdk
Now when you perform a successful OE build, a vmdk will appear in your deploy/images folder.

Running a VMDK in a Virtual Machine

 

In order to run a vmdk in a virtual machine such as VirtualBox simply run through VirtualBox's "add a new machine" wizard. When it asks you about the disk, tell it you already have a disk, and select the vmdk generated by your OE build.

Unfortunately by default the VirtualBox wizard will add your disk under the SATA controller:



You'll need to switch it to be connected to the IDE controller:



Notes:

  • currently OE is only able to generate vmdk's for x86 machines since the current procedure requires syslinux (which only runs on x86 machines)
  • it is also possible to use runqemu to run a vmdk
  • my MACHINE setting for this example is "qemux86", and I'm building distroless