Archive
Posts in Storage
Removing a gluster volume doesn't remove the volume's contents
I made another interesting discovery this weekend while playing around with the gluster volume deletion option. Prior to creating a volume with a new layout, I went through the documented process to remove my volume: I then re-created it using the documented process: Once the new volume was created and started I mounted in on my clients. When I went to access the volume I was quite intrigued to find that the data that was written to the previous gluster volume was still present: Ey? Since 'gluster delete volume' spit out the message "Deleting volume will erase all information about the volume", I figured the contents of the volume would be nuked (never assume, always confirm!)…
$ read more →Some interesting insights on the gluster replicated volume replica value
While playing around with gluster, I had an interesting finding about the way gluster handles replicated volumes. The gluster volume I am using for testing is a replicated volume with a replica factor of 2 (the replica factor determines how many copies of your data will be made). I wanted to add a third replica to my volume, and thought it would be as simple as using the "add-brick" option: Hmmmm -- no go. At first I thought this was no big deal, I figured there was an option or setting I needed to change to increase my replica count…
$ read more →Centos 6 Linux VMs running inside vSphere 4.1 appear to dynamically discover new LUNs
I came across an interesting discovery yesterday while working on a CentOS 6 gluster node. The node was virtualized inside vSphere 4.1 and needed some additional storage added to it. I went into the VI client and added a new disk while the server was running, expecting to have to reboot or rescan the storage devices in the server. Well, I was pleasantly surprised when the following messages popped up on the console: Nice, it looks like the device was added to the system dynamically…
$ read more →Installing gluster on a CentOS machine via rpmbuild
I talked previously about my experience getting gluster up and running on Fedora and CentOS Linux servers. The installation process as it currently stands is different between Fedora and CentOS servers. The Fedora package maintainers have build RPMs for gluster, so you can use yum to install everything needed to run gluster: Gluster packages aren't currently available for CentOS 6 (at least they aren't in extras or centosplus as of this morning), so you are required to build from source if you want to use CentOS as your base operating system. The build process is pretty straight forward, and I'll share my notes and gotchas with you…
$ read more →Forcing glusterfs to heal bricks that are inconsistent
I mentioned in my previous post [creating clustered file systems with glusterfs] (/blog/2011/11/13/creating-clustered-file-systems-with-glusterfs-on-centos-and-f edora-linux-servers/) that gluster self-healing has a few issues. To see what I'm referring to we can access a gluster client and cp a bunch of data to the glusterfs file system located at /gluster: Once I started the copy operation I immediately reboot a brick to stop I/O from going to it. When the box came back up there was a discrepancy between the contents of the two bricks: Gluster brick #1: Gluster brick #2: Bizarre, ey? This isn't fixed in due time, you need to follow the instructions in the official documentation and stat the contents of the file system to make them consistent again: This return things to normal on the brick that was rebooted: There are numerous reasons you could encounter this situation…
$ read more →