I ran into an issue last week where two nodes using shared storage lost the partition table on one of the storage devices they were accessing. This was extremely evident in the output from fdisk:
$ fdisk -l /dev/sdb /dev/sdb1 /dev/sdb9
Disk /dev/sdb: 107.3 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 *** 512 = 8225280 bytes
Device Boot Start End Blocks Id System
Fortunately the system in question still had the /proc/partition data in tact, so I stashed that in a safe location and came up with a plan to re-recreate the partition tables using this data. Given the following /proc/partitions data for the device listed above:
$ grep sdb /proc/partitions
8 16 104857600 sdb
8 17 99161148 sdb1
8 18 5695042 sdb2
I can see that the device had two partion table entries, and the device was approximately 100GB in size. To re-create the partition tables, I used the parted mkpart command passing it the starting and ending sector numbers I wanted written to the partition table:
$ mkpart /dev/sdb /dev/sdb1 /dev/sdb9
(parted) mkpart primary ext3 128s 198322424s
(parted) mkpart primary ext3 198322425s 209712509s
Now you may be asking yourself where did I get the starting and ending sectors from? In my specific case I start the partition tables at sector 128 to ensure alignment with the storage arrays we use, and the end sector was calculated by taking the partion table entry from /proc/partitions and multiplying it by 2 (the /proc/partitions sizes are in 1k chunks). I also had to add the 128 sector offset to this value.
To get the values for sdb2, I added one to the number produced above, and then multiplied the sdb2 value from /proc/partitions by two to get the following numbers:
198322424 + 1 = 198322425 start sector or sdb2
198322425 + 11390084 = 209712509 end sector of sdb2
Once I wrote out my changes, I verified that they were correct with sfdisk:
$ sfdisk -uS -l /dev/sdb /dev/sdb1 /dev/sdb9
Disk /dev/sdb: 13054 cylinders, 255 heads, 63 sectors/track
Units = sectors of 512 bytes, counting from 0
Device Boot Start End #sectors Id System
/dev/sdb1 128 198322424 198322297 83 Linux
/dev/sdb2 198322425 209712509 11390085 83 Linux
/dev/sdb3 0 - 0 0 Empty
/dev/sdb4 0 - 0 0 Empty
And verified that /proc/partitions matched up with what was previously there:
$ grep sdb /proc/partitions
8 16 104857600 sdb
8 17 99161148 sdb1
8 18 5695042 sdb2
Now that the partition table was restored, I used the fsck “-n” option to verify the file system was complete (make sure to read the fsck manual page before running it, as the “-n” option WILL modify some file systems!!):
$ fsck -n /dev/sdb1
fsck 1.39 (29-May-2006)
e2fsck 1.39 (29-May-2006)
/dev/sdb1: clean, 11/12402688 files, 437281/24790287 blocks
$ fsck -n /dev/sdb2
fsck 1.39 (29-May-2006)
e2fsck 1.39 (29-May-2006)
/dev/sdb2: clean, 11/712448 files, 57952/1423760 blocks
And then proceeded to mount them up:
$ mount /dev/sdb1 /a
$ mount /dev/sdb2 /bin /boot
$ df -h | grep sdb
/dev/sdb1 94G 188M 89G 1% /a
/dev/sdb2 5.4G 139M 5.0G 3% /b
I’m not a data recovery person / service (just a guy who reads a lot and understands the internals of how file systems and devices are structured), so use this information at your own risk!