Using bmaptool To Create A Memory Card

Here's the scenario: I have just used OE to build a core-image-minimal which I want to run on my Wandboard-dual, I insert my 4GB microSD card into my desktop, use dd to write the image to the card, insert the card into my board, boot, and get:

But it's a 4GB card?! Where's the rest of my disk?

OE has no idea how big of a card you want to use, so by default it makes an image that is just a bit bigger than required (or 8MB, whichever is larger).

Writing this small image is quick:

If I want to use (roughly) the entire 4GB card I simply ask OE to build an image of that size. Edit conf/local.conf and add/edit:

IMAGE_ROOTFS_SIZE = "3800000"

Now when I build my image, the output from OE will be roughly 3.7GB in size. Writing this image to a card will take much longer:

The funny thing is, the data hasn't changed; I'm still using the same amount of data on the card. What has changed is that I now have access to (roughly) the entire card, but at the cost of having it take ~160 times longer to write the image!

In this case we're wasting lots of time (and flash write cycles) writing empty portions of the image to the disk. This is where the bmaptools come in. In essence, bmaptool looks at your image and determines which parts are important, and which are empty. When you go to actually write your image, only the non-empty parts are transferred -- saving you lots of write time (and flash cycles).

Using bmaptool is a two-step process:

1. use bmaptool create to create a mapping file
2. use bmaptool copy to write your image to a disk (with the help of the mapping file you just created)

Applying bmaptool to our 4GB image:

It's not the 18 seconds from above (i.e. dd'ing the 80MB image), but it's still better than the 49 minutes required to dd the 4GB image. The image written with bmaptool works:

Note that if I use bmaptool on the first (80MB) image, there isn't much savings:

The real benefits are seen when trying to write an image such that most of the card is then available for use, and most of the image to be written is empty.

Using OE/Yocto to create VMs

By default most OE builds will create a filesystem image, a kernel, and (perhaps) a bundle of kernel modules. This set of outputs makes it easy if you're looking to flash your build into real hardware or run it in emulation via qemu (i.e. runqemu). But what if you're looking to run your image in a virtual machine?

To run an image in a virtual machine you'll need a virtual disk image which contains a partition map and a bootloader. Fortunately OE provides support for generating one such type of virtual disk image: the vmdk.

Configure OE to Generate a VMDK

 

In order to get OE to generate a vmdk artifact simply edit your conf/local.conf to add vmdk to IMAGE_FSTYPES. Unfortunately, due to the way in which variables are handled in bitbake, it isn't possible to use the _append operator, only the += operator can be used:

IMAGE_FSTYPES += vmdk

Now when you perform a successful OE build, a vmdk will appear in your deploy/images folder.

Running a VMDK in a Virtual Machine

 

In order to run a vmdk in a virtual machine such as VirtualBox simply run through VirtualBox's "add a new machine" wizard. When it asks you about the disk, tell it you already have a disk, and select the vmdk generated by your OE build.

Unfortunately by default the VirtualBox wizard will add your disk under the SATA controller:

You'll need to switch it to be connected to the IDE controller:

Notes:

• currently OE is only able to generate vmdk's for x86 machines since the current procedure requires syslinux (which only runs on x86 machines)
• it is also possible to use runqemu to run a vmdk
• my MACHINE setting for this example is "qemux86", and I'm building distroless

Video Screen Captures in VirtualBox

A couple weeks ago, Oracle released VirtualBox 4.3.x and with it the ability to take video captures of whatever is going on in a VM! It has always been possible to take screen shots of whatever is going on in a VM, but taking videos is something new.

If you look at the Display options of the Settings panel, you'll notice a new tab entitled Video Capture.

From this panel you can modify various parameters related to the capture, and you can toggle the capture on and off. You can also toggle the video capture from the buttons at the bottom-right corner of a non-full-screened VM, or via the Devices menu for a VM that is full-screened.

Here's a video I shot of a *.vmdk image I created using Yocto (core-image-minimal) booting up:

Extending git a little bit: patch-set

A while back, Peter Hutterer posted to his blog about some git extensions he had written to make the patch submission processes somewhat easier using a git workflow. Instead of using

$ git format-patch <...>

to create a set of patches, he wrote a new script which is invoked as

$ git patch-set <...>

Each invocation creates a new patches/patches-<date>-<revlist> directory under a project's top-level directory and stores the set of patches there as one group. As a convenience, it also creates a <top>/patches/latest link to point to the location of the set of patches produced by the most recent git patch-set invocation.

Why would you ever need something like this? If you're working with a set of patches and some of them need to be re-worked, it can become confusing to remember which set of all the xxxx-your-change.patch files in your current directory need to be sent with git send-email. Using this tool helps keep each group of patches you generate separate from each other so when it comes time to send your patches upstream, you only need to run:

$ git send-email patches/latest/*patch

I decided to give Peter's extension a try today and found it quite nice.

As I was exploring his git extensions I also stumbled upon git-extras which is quite nice too, and a little more polished.

While using Peter's git patch-set, it seems obvious to me that he uses a specific set of options to git format-patch which, I assume, are what he needs to use when submitting patches to the projects on which he works. Unfortunately I need to use a different set of options. So instead of hard-coding the set I want for my needs, I decided to fork his repository and update his git-patch-set script to be more generic. My changes assume the last argument in the git patch-set <...> command is the rev-list, but that all other options are to be passed to the internal call to git format-patch without assuming any defaults. So now I can use the following command to get my patches made the way I need:

$ git patch-set -M --subject-prefix="component][PATCH" \ --cover-letter master..mywork

I also added a Makefile in order to make it easier to install into one's system.

Rebinding Bash Readline C-w

For the past couple months I have been forcing myself to learn and (most importantly) use more bash/readline shortcuts. Since I am more accustomed to vi than to emacs, I've been using vi input mode.

For example, say I'm typing a very long command and just about at the end of typing said command I realize that this command won't work just yet, I have to do something else before issuing the current command. Up until this point I would normally solve this problem by:

1. holding Backspace for a couple seconds to delete the current, long command
2. typing and issuing the new command
3. typing in the long command all over again

Using bash/readline vi shortcuts this sequence can be improved to:

1. hitting Esc to get out of input mode
2. entering # to comment out but run the current command (although this appears to do nothing, it actually does put it into the history)
3. typing and issuing the new command
4. hitting Esc 
5. pressing k (to go "up" in the history) twice
6. hitting x to delete the initial # 
7. then Enter to run the already-typed long command

Although the second sequence appears longer and more complicated, it does save a lot of typing and is actually faster and simpler (once you get the hang of it).

The one thing that has been rather annoying is the C-w binding.

While typing (i.e. in input mode) you can press C-u (for example) to delete everything on the current line you've typed in so far. Or you can use C-w to just delete the last "word". The reason I put "word" in quotes is because different commands use different definitions for word boundaries. If I'm typing in a path (e.g. /usr/local/bin/cmd), does the last path element count as a word, or the entire path? Or if I'm typing in a bunch of words separated by dashes (e.g. core-image-minimal) should C-w delete the entire phrase, or just back to the last dash?

As it turns out, by default C-w only considers "words" to be separated by whitespace. But more often I would prefer that punctuation be included too.

Thankfully the distinction between using whitespace-delimited words and punctuation-delimited words has already been considered. By default C-w is linked to the unix-word-rubout command (which only considers whitespace), but what I really want is for it to be linked to the backward-word-kill command (which also considers punctuation).

Following the advice here I simply edited my .bashrc to add:

stty werase undef
bind '"\C-w":backward-kill-word'

and now C-w will erase backwards to either the most recent whitespace or the most recent punctuation.

Note: I found that using

$ bind -P

was helpful to see which commands are currently linked to which control sequences.