While perusing packages on my Debian 5 host, I came across the ext3grep utility. Ext3grep allows you to poke and prod ext file system metadata structures (superblocks, inode bitmaps, block details, etc.), and has the ability to recover deleted files. To see first hand how file recovery works (this is a useful thing to have in your bag of tricks), I first created an ext3 file system to test this out:
$ dd if=/dev/zero of=/fstmp bs=512 count=4192
4192+0 records in
4192+0 records out
2146304 bytes (2.1 MB) copied, 0.0218335 s, 98.3 MB/s
$ mkfs.ext3 /fstmp
mke2fs 1.41.3 (12-Oct-2008)
/tmp/fstmp is not a block special device.
Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
264 inodes, 2096 blocks
104 blocks (4.96%) reserved for the super user
First data block=1
Maximum filesystem blocks=2359296
1 block group
8192 blocks per group, 8192 fragments per group
264 inodes per group
Writing inode tables: done
Creating journal (1024 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 32 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
$ mount -o loop /fstmp /mnt
Once the file system was mounted, I copied over and immediately removed a file:
$ cd /mnt
$ cp /etc/services . ..
$ ls -la
total 37
drwxr-xr-x 3 root root 1024 2009-03-28 12:43 .
drwxr-xr-x 22 root root 4096 2009-03-28 12:41 ..
drwx------ 2 root root 12288 2009-03-28 12:41 lost+found
-rw-r--r-- 1 root root 18480 2009-03-28 12:43 services
$ rm -f services
$ cd /
$ umount /mnt
After the file was removed, I used the ext3grep utilities “–dump-name” option to display a list of file names:
$ ext3grep --dump-name /fstmp
Running ext3grep version 0.8.0
WARNING: I don't know what EXT3_FEATURE_COMPAT_EXT_ATTR is.
Number of groups: 1
Minimum / maximum journal block: 60 / 1089
Loading journal descriptors... sorting... done
The oldest inode block that is still in the journal, appears to be from 1238258603 = Sat Mar 28 12:43:23 2009
Number of descriptors in journal: 16; min / max sequence numbers: 2 / 3
Finding all blocks that might be directories.
D: block containing directory start, d: block containing more directory entries.
Each plus represents a directory start that references the same inode as a directory start that we found previously.
Searching group 0: DD+
Writing analysis so far to 'fstmp.ext3grep.stage1'. Delete that file if you want to do this stage again.
Result of stage one:
2 inodes are referenced by one or more directory blocks, 2 of those inodes are still allocated.
1 inodes are referenced by more than one directory block, 1 of those inodes is still allocated.
0 blocks contain an extended directory.
Result of stage two:
2 of those inodes could be resolved because they are still allocated.
All directory inodes are accounted for!
Writing analysis so far to 'fstmp.ext3grep.stage2'. Delete that file if you want to do this stage again.
lost+found
services
In the output above, we can see that the services file I previously removed is listed. To recover deleted files, you can run ext3grep with the “–restore-file” option to restore individual files, or with the “–restore-all” option to restore all deleted files:
$ ext3grep --restore-all /fstmp
Running ext3grep version 0.8.0
WARNING: I don't know what EXT3_FEATURE_COMPAT_EXT_ATTR is.
Number of groups: 1
Minimum / maximum journal block: 60 / 1089
Loading journal descriptors... sorting... done
The oldest inode block that is still in the journal, appears to be from 1238260343 = Sat Mar 28 13:12:23 2009
Number of descriptors in journal: 23; min / max sequence numbers: 21 / 25
Loading fstmp.ext3grep.stage2... done
Restoring services
After the restore operation completes, you can change to the RESTORED_FILES directory to view the recovered files:
$ cd RESTORED_FILES/
$ ls -la
total 32
drwxr-xr-x 3 root root 4096 2009-03-28 13:15 .
drwxr-xr-x 6 root root 4096 2009-03-28 13:15 ..
drwx------ 2 root root 4096 2009-03-28 12:41 lost+found
-rw-r--r-- 1 root root 18480 2009-03-28 13:12 services
$ openssl md5 services /etc/services
MD5(services)= b92ef61d6606c6589df9c2c406632457
MD5(/etc/services)= b92ef61d6606c6589df9c2c406632457
This is pretty fricking sweet, though there are a few caveats I should mention. It is possible that blocks will get overwritten between the file removal and recovery steps, which can hinder recovery. It is also possible that an inode may be recycled, which would indeed make recovery a bit more difficult. Additionally, any recovery operation should work off a copy of the original data to avoid tainting the original file system. Those things said, ext3grep is an incredible tool, and Carlo’s write up is a must read for those wanting to understand EXT file system structures and what is takes to recover files. Great work Carlo!