25 Mar 2014

Using bmaptool To Create A Memory Card

Here's the scenario: I have just used OE to build a core-image-minimal which I want to run on my Wandboard-dual, I insert my 4GB microSD card into my desktop, use dd to write the image to the card, insert the card into my board, boot, and get:

Size=62.0M Used=19.1M

But it's a 4GB card?! Where's the rest of my disk?

OE has no idea how big of a card you want to use, so by default it makes an image that is just a bit bigger than required (or 8MB, whichever is larger).

Writing this small image is quick:


If I want to use (roughly) the entire 4GB card I simply ask OE to build an image of that size. Edit conf/local.conf and add/edit:
Now when I build my image, the output from OE will be roughly 3.7GB in size. Writing this image to a card will take much longer:


The funny thing is, the data hasn't changed; I'm still using the same amount of data on the card. What has changed is that I now have access to (roughly) the entire card, but at the cost of having it take ~160 times longer to write the image!

Size=3.4G Used=86.9M

In this case we're wasting lots of time (and flash write cycles) writing empty portions of the image to the disk. This is where the bmaptools come in. In essence, bmaptool looks at your image and determines which parts are important, and which are empty. When you go to actually write your image, only the non-empty parts are transferred -- saving you lots of write time (and flash cycles).

Using bmaptool is a two-step process:
  1. use bmaptool create to create a mapping file
  2. use bmaptool copy to write your image to a disk (with the help of the mapping file you just created)
Applying bmaptool to our 4GB image:


It's not the 18 seconds from above (i.e. dd'ing the 80MB image), but it's still better than the 49 minutes required to dd the 4GB image. The image written with bmaptool works:

Size=3.4G Used=86.9M

Note that if I use bmaptool on the first (80MB) image, there isn't much savings:


The real benefits are seen when trying to write an image such that most of the card is then available for use, and most of the image to be written is empty.


1 comment:

  1. The 4GiB image is 99.8% mapped, according to the log. Do you have gigabytes of data there? If not, then OE does not create images "optimally", meaning the resulting image is not sparse enough.

    So the speed-up that you see is very small comparing to what it could be if the image was really sparse. The speed-up is mostly due to other optimizations (e.g., bmaptool chooses the 'noop' I/O scheduler, etc).

    IOW, to get _real_ speed-up, you need to look at how OE works.

    If you get, say, 100MiB of data, and make a 4GiB image, it should have only 100MiB + a little bit of blocks mapped. And then bmaptool will only write 100MiB of data when you flash.