Working around problems with a CentOS 6 kickstart upgrade

This article was posted by Matty on 2011-11-26 16:07:00 -0400 -0400

Each time I go to deploy a new release of Centos or Fedora I encounter a new “feature” or “change” that breaks configurations that previously worked. I recently encountered several problems upgrading several hosts to FEDORA 16, and this afternooon I encountered another issue while attempting to upgrade several Centos 5 servers to Centos 6.

The issue with the CentOS upgrade revolves around changes to the software installation groups. Groups that worked perfectly fine with CentOS 5 no longer work with CentOS 6. This results in the installer spitting out “This group does not exist” errors when it parses the “%packages” section in your kickstart.cfg:

Groups

So what exactly changed? In CentOS 5.7 we had the following groups available to the installer:

$ yum grouplist

Installed Groups:
Administration Tools
Dialup Networking Support
GNOME Software Development
Legacy Network Server
Mail Server
Network Servers
Server Configuration Tools
System Tools
Web Server
Windows File Server
Yum Utilities
Available Groups:
Author: Matty
Base
Beagle
Cluster Storage
Clustering
DNS Name Server
Development Libraries
Development Tools
Editors
Emacs
Engineering and Scientific
FTP Server
FreeNX and NX
GNOME Desktop Environment
Games and Entertainment
Graphical Internet
Graphics
Horde
Java
Java Development
KDE (K Desktop Environment)
KDE Software Development
KVM
Legacy Software Development
Legacy Software Support
Mono
MySQL Database
News Server
Office/Productivity
OpenFabrics Enterprise Distribution
PostgreSQL Database
Printing Support
Ruby
Sound and Video
Text-based Internet
Tomboy
Virtualization
X Software Development
X Window System
XFCE-4.4

And in CentOS 6 this list was revised and expanded to the following groups:

$ yum grouplist

Installed Groups:
Base
E-mail server
Graphical Administration Tools
Hardware monitoring utilities
Legacy UNIX compatibility
Networking Tools
Optional packages
Performance Tools
Perl Support
Available Groups:
Additional Development
Afrikaans Support
Albanian Support
Amazigh Support
Arabic Support
Armenian Support
Assamese Support
Azerbaijani Support
Backup Client
Backup Server
Basque Support
Belarusian Support
Bengali Support
Bhutanese Support
Brazilian Portuguese Support
Breton Support
Bulgarian Support
CIFS file server
Catalan Support
Chhattisgarhi Support
Chichewa Support
Chinese Support
Compatibility libraries
Console internet tools
Coptic Support
Croatian Support
Czech Support
Danish Support
Debugging Tools
Desktop
Desktop Debugging and Performance Tools
Desktop Platform
Desktop Platform Development
Development tools
Dial-up Networking Support
Directory Client
Directory Server
Dutch Support
Eclipse
Emacs
English (UK) Support
Esperanto Support
Estonian Support
Ethiopic Support
FCoE Storage Client
FTP server
Faroese Support
Fijian Support
Filipino Support
Finnish Support
Fonts
French Support
Frisian Support
Friulian Support
Gaelic Support
Galician Support
General Purpose Desktop
Georgian Support
German Support
Graphics Creation Tools
Greek Support
Gujarati Support
Hebrew Support
High Availability
High Availability Management
Hiligaynon Support
Hindi Support
Hungarian Support
Icelandic Support
Indonesian Support
Infiniband Support
Input Methods
Interlingua Support
Internet Applications
Internet Browser
Inuktitut Support
Irish Support
Italian Support
Japanese Support
Java Platform
KDE Desktop
Kannada Support
Kashmiri Support
Kashubian Support
Kazakh Support
Khmer Support
Kinyarwanda Support
Konkani Support
Korean Support
Kurdish Support
Lao Support
Large Systems Performance
Latin Support
Latvian Support
Legacy X Window System compatibility
Lithuanian Support
Load Balancer
Low Saxon Support
Luxembourgish Support
Macedonian Support
Mainframe Access
Maithili Support
Malagasy Support
Malay Support
Malayalam Support
Maltese Support
Manx Support
Maori Support
Marathi Support
Messaging Client Support
Messaging Server Support
Mongolian Support
MySQL Database client
MySQL Database server
Myanmar (Burmese) Support
NFS file server
Nepali Support
Network Infrastructure Server
Network Storage Server
Network file system client
Northern Sotho Support
Norwegian Support
Occitan Support
Office Suite and Productivity
Oriya Support
PHP Support
Persian Support
Polish Support
Portuguese Support
PostgreSQL Database client
PostgreSQL Database server
Print Server
Printing client
Punjabi Support
Remote Desktop Clients
Resilient Storage
Romanian Support
Russian Support
SNMP Support
Sanskrit Support
Sardinian Support
Scalable Filesystems
Scientific support
Security Tools
Serbian Support
Server Platform
Server Platform Development
Sindhi Support
Sinhala Support
Slovak Support
Slovenian Support
Smart card support
Somali Support
Southern Ndebele Support
Southern Sotho Support
Spanish Support
Storage Availability Tools
Swahili Support
Swati Support
Swedish Support
System Management
System administration tools
Tagalog Support
Tajik Support
Tamil Support
TeX support
Technical Writing
Telugu Support
Tetum Support
Thai Support
Tibetan Support
Tsonga Support
Tswana Support
TurboGears application framework
Turkish Support
Turkmen Support
Ukrainian Support
Upper Sorbian Support
Urdu Support
Uzbek Support
Venda Support
Vietnamese Support
Virtualization
Virtualization Client
Virtualization Platform
Virtualization Tools
Walloon Support
Web Server
Web Servlet Engine
Web-Based Enterprise Management
Welsh Support
X Window System
Xhosa Support
Zulu Support
iSCSI Storage Client

So as you can see various groups were removed, some were renamed and others were added. I’ve come to expect problems like this with every CentOS and Fedora upgrade, since there appears to be little to no thought given to compatibility between releases. So with that said, my applications just finished compiling so it’s time to take CentOS 6 for a ride. :)

Adding a new CentOS or Fedora CD or DVD image to a kickstart server

This article was posted by Matty on 2011-11-26 15:05:00 -0400 -0400

I’ve talked in the past about Yum repositories, and how you can create them. If you are using kickstart and want to import the contents of a CD for use by the installer, there are numerous ways you can tackle this issue. One of the easiest methods is to loopback mount the CDs or DVDs and copy them to your kickstart directory hierarchy:

$ mount -o loop CentOS-6.0-x86_64-bin-DVD1.iso /cd1

$ mount -o loop CentOS-6.0-x86_64-bin-DVD2.iso /cd2

$ cd /cd1 && cp -rp repodata /kickstart/centos/6/x86_64/os &

$ cd /cd1 && cp -rp images /kickstart/centos/6/x86_64/os &

$ cd /cd1 && cp -rp Packages /kickstart/centos/6/x86_64/os &

$ cd /cd2 && cp -rp Packages /kickstart/centos/6/x86_64/os &

You will need to make sure to copy the Packages, repodata and images directories. After the copy completes you can point your kickstart.cfg file at the top level directory of the release you want to install:

url --url=http://kickstart.prefetch.net/kickstart/centos/6.0/x86_64/os/

Posting here so I have this procedure for future reference.

Upgrading from vSphere 4.0 to vSphere 4.1 update 2

This article was posted by Matty on 2011-11-26 14:21:00 -0400 -0400

This weekend I decided to take the plunge and upgrade my vSphere 4.0 servers to vSphere 4.1 update 2. The upgrades went amazingly well, and I was amazed at just how easy it is to upgrade a cluster with VMWare update manager. The upgrade process took about 30-minutes per host, and I used the following two resources to guide me through the upgrade:

It’s amazing how big VMWare is getting. It seems like just yesterday I was running their desktop virtualization product, and people who didn’t have an extensive IT background would mumble “Virtual what?” Oh how times have changed.

Forcing glusterfs to heal bricks that are inconsistent

This article was posted by Matty on 2011-11-16 09:20:00 -0400 -0400

I mentioned in my previous post creating clustered file systems with glusterfs that gluster self-healing has a few issues. To see what I’m referring to we can access a gluster client and cp a bunch of data to the glusterfs file system located at /gluster:

$ cd /gluster

$ mkdir etc

$ cd etc

$ cp -rp /etc/* .

Once I started the copy operation I immediately reboot a brick to stop I/O from going to it. When the box came back up there was a discrepancy between the contents of the two bricks:

Gluster brick #1:

$ pwd

/gluster/vol01/etc

[root@fedora-cluster01 etc]# **du -sh . ..
35M .

Gluster brick #2:

$ pwd

/gluster/vol01/etc

[root@fedora-cluster02 etc]# **du -sh . ..
4.6M .

Bizarre, ey? This isn’t fixed in due time, you need to follow the instructions in the official documentation and stat the contents of the file system to make them consistent again:

$ find /gluster -noleaf -print0 | xargs --null stat >/dev/null

This return things to normal on the brick that was rebooted:

$ pwd

/gluster/vol01/etc

$ du -sh .

35M .

There are numerous reasons you could encounter this situation. A server could panic and reboot. You may need to reboot the server to apply updates. Or you may need to perform hardware maintenance (disk replacement, rack maintenance, etc.) which requires the server to be offline for a while. Jeff Darcy posted a nice reply to my original gluster post, and it looks like he’s in the midst of working on a permanent fix for this. It will surely be welcomed from folks who want to put gluster into production. :)

My Top Ten technology books of all time

This article was posted by Matty on 2011-11-14 21:17:00 -0400 -0400

I was having lunch with a friend last week and we caught up on tons of stuff. Both of us are doing a lot of work with VMWare, storage, Linux and interestingly enough we are both trying to learn about Windows Server 2008. As we were chatting my buddy asked me what I thought the best ten technology books of all time were. I immediately said the entire Richard Stevens collection, but then started to think which other books have had the biggest impact on my career. After a few days of thought I think I came up with my top ten:

This was a hard list to come up with, but each of these books has had a profound impact on my love of UNIX and technology in general. Given the recent passing of the creator of UNIX Dennis Ritchie, I thought it only fitting that K&R would be placed on the top of my list. Pointers drove me nuts until I finally understood them, but the beauty and elegance of C still amazes me to this day. Which books have touched you the most?