Creating sparse files on Linux hosts with dd

Sparse files have become pretty common in the virtualization arena, since they allow you to present a large chunks of disk space to guests without having to reserve the space in an actual backing store. This has a couple of benefits:

1. Most guests use a fraction of the disk space allocated to them, so sparsely allocating files cuts down on the amount of disk space required to host them

2. Additional storage can be added over time to support growth (disk storage keeps getting cheaper, so this can actually lead to a reduction in operational costs)

3. Backing up the guests requires less tape / disk space (this assumes your backup solution supports sparse files)

To create a sparse file on a Linux host, you can run dd with a count size of zero (this tells dd not to write any data to the file), and then use the seek option to extend the file to the desired size:

$ dd if=/dev/zero of=xen-guest.img bs=1 count=0 seek=8G

Once the sparse fie is created, you can use dd to verify how much space is allocated to it:

$ du -sh xen-guest.img
0 xen-guest.img

$ du -sh –apparent-size xen-guest.img
8.0G xen-guest.img

Sparse files are extremely handy, though it’s important to know when and when not to use them.

4 Comments

no  on July 31st, 2010

“you can use dd to verify” -> “you can use du to verify”

slydog  on March 9th, 2011

Thank you for posting this. You close your post with the comment, “Sparse files are extremely handy, though it’s important to know when and when not to use them.” What are your thoughts on when not to use sparse files.

I would consider performance and ensuring capacity as two reasons. I’m curious if you had other thoughts.

mark  on June 29th, 2011

In order to copy an existing partition device (or zcat a compressed disk image) but make the output sparse, you can do something like this:

cat /dev/sda3 | cp –sparse=always /dev/stdin sparse_sda3_copy.img

michel  on June 15th, 2013

Thanks for the post. you can also use your dd command to extend existing [sparse] files ( I used it to grow images)

Leave a Comment