Managing Solaris patches with pca

I have written repeatedly about the problems with the Solaris patch tools, and decided to test out the pca utility after Chris and Frank recommended it so highly. Pca is not only an awesome patching tool, but it blows away everything that is currently offered by Sun (pca has yet to throw Java exceptions and die in mysterious ways). To see what I mean, all you need to do is run pca with the help option:

$ pca -h

Usage: bin/pca [OPTION] .. [OPERAND] ..

Operands:
  patch group:    missing, installed, all, unbundled, bad
                  Add r, s or rs at the end to list Recommended,
                  Security or Recommended/Security patches only.
  patch ID:       123456, 123456-78
  patch file:     123456-78.zip, 123456-78.tar.Z
  file name:      patchlist.txt
  pattern:        /dtmail/

Options:
  -l              List patches
  -L              List patches, produce HTML output
  -d              Download patches
  -i              Install patches
  -I              Pretend to install patches
  -x              Download patch cross-reference file
  -y              Do not check for updated patch cross-reference file
  -X dir        Set location of patches cross-reference file
  -P dir        Set patch download directory
  -R dir        Set alternative root directory
  -n              Install only patches which do not require a reboot
  -k              Make patchadd not back up files to be patched
  -G              Make patchadd modify packages in the current zone only
  -a              Ask for SunSolve authentication data interactively
  -H              Don't display descriptive headers
  -r id         Display patch README
  -f dir        Read uname/showrev/pkginfo output from files in dir
  -h              Display this help
  -V              Display debug output
  -v              Display version information

Pca has several modes of operation. It can list patches that are outdated on your system, retrieve patches from Sunsolve, and most importantly it can be used to install individual patches and groups of patches on a server. To list patches that are available for a server, pca can be run with the “-l” option (or “-L” if you want HTML reports):

$ pca -l

Retrieving xref-file to /var/tmp/patchdiag.xref ... done
Using /var/tmp/patchdiag.xref from Jul/14/06
Host: neutron (SunOS 5.10/i386/i86pc)

Patch  IR   CR RSB Age Synopsis
------ -- - -- --- --- -------------------------------------------------------
117464 01 < 02 ---   2 SunOS 5.10_x86: passwdutil Patch
119131 21 < 22 R--   4 SunOS 5.10_x86: Sun Fibre Channel Device Drivers
119214 07 < 09 RSB   2 NSS_NSPR_JSS 3.11.2_x86: NSPR 4.6.2 / NSS 3.11.2 / JSS 4.2.4
119471 05 < 06 ---   5 SunOS 5.10_x86: Sun Enterprise Network Array firmware and utilitie
119686 06 < 07 ---   2 SunOS 5.10_x86: lib/svc/bin/svc.startd Patch
120037 03 < 05 ---   2 SunOS 5.10_x86: libldap patch
120053 02 < 03 ---   2 SunOS 5.10_x86: pam library patch
120200 04 < 05 ---   3 SunOS 5.10_x86: sysidtool Patch
121003 02 < 03 ---   2 SunOS 5.10_x86: pax patch
121005 01 < 02 RS-   2 SunOS 5.10_x86: sh patch
121011 01 < 02 ---   2 SunOS 5.10_x86: rpc.metad patch
123327 -- < 01 ---   5 SunOS 5.10_x86: tail patch
123521 -- < 01 ---   2 SunOS 5.10_x86: dirname & basename patch
123525 -- < 01 ---   2 SunOS 5.10_x86: psrinfo patch

The first column lists the patchid, the second column lists the version of the patch installed, the fourth column lists the updated version of the patch that is available on Sunsolve, and the fifth column indicates if the patch addresses a security or reliability problem. To install patches with pca, you can run pca with the "-i" option to install all available patches, or you can install individual patches by passing the patchid(s) to "-i":

$ pca -i 121005 119214

Retrieving xref-file to /var/tmp/patchdiag.xref ... done
Using /var/tmp/patchdiag.xref from Jul/14/06

Downloading patches to /home/matty
------------------------------------------------------------------------------
Retrieving 119214-09 (1/2) ... done
Retrieving 121005-02 (2/2) ... done

Summary: 2 total, 2 successful, 0 failed

Installing patches
------------------------------------------------------------------------------
Installing 119214-09 (1/2) ... done
Installing 121005-02 (2/2) ... done

Summary: 2 total, 2 successful, 0 skipped, 0 failed

This is a super useful piece of software, and I wish Sun would include something similar in Solaris (smpatch is not the answer!).

Useful DTrace links

I came across a couple of super useful DTrace links, and thought I would pass them on:

Brendan Gregg’s DTrace presentation in London:
http://www.context-switch.com/performance/dtrace_workshop01_slides.pdf

Opensolaris student guide (the chapter on using DTrace to debug device drivers is awesome):
http://opensolaris.org/os/community/documentation/files/studentguide.pdf

Addicted to the Sopranos

The Sopranos t.v. series has been on the air since 1999, but I have never watched enough television to justifiy the cost of permium programming. On the recommendation of a friend, I decided to get the first season of the Sopranos from Netflix. I finished up season 2 last night, and am hoping to start watching season three tomorrow. The Sopranos is one of the best television series I have ever watched (I also like The Godfather, Casino, Goodfellas, etc.), and in my opinion it’s one of the best television series to hit the airwaves in the past 10 – 20 years. I miss classic television series like Macgyver, Airwolf, Knight Rider, V, Seinfeld, and for that matter, the rest of the awesome programming that made up the 80s. Luckily I have Modern Marvels, and three more seasons of The Sopranos. :)

Building Perl modules for Solaris

This week I needed to install a few Perl modules on a Solaris 10 host. I didn’t want to download and install a fourth perl interpreter (Solaris 10 comes with 5.6.1, 5.8.3 and 5.8.4 for some reason), since Solaris 10 comes with a relatively recent version of Perl (5.8.4). To build the module in question (DBD::mysql), I downloaded the module from CPAN, verified that the MD5 checksum was correct, and used the following steps to compile the module:

$ perl Makefile.PL

$ make

$ make install

The ‘make Makefile.PL’ completed succesfully, but the make failed with the following errors:

$ make

cp lib/DBD/mysql.pm blib/lib/DBD/mysql.pm
cp lib/DBD/mysql/GetInfo.pm blib/lib/DBD/mysql/GetInfo.pm
cp lib/Mysql.pm blib/lib/Mysql.pm
cp lib/DBD/mysql/INSTALL.pod blib/lib/DBD/mysql/INSTALL.pod
cp lib/Mysql/Statement.pm blib/lib/Mysql/Statement.pm
cp lib/Bundle/DBD/mysql.pm blib/lib/Bundle/DBD/mysql.pm
cc -c  -I/usr/perl5/site_perl/5.8.4/i86pc-solaris-64int/auto/DBI -I/home/apps/mysql/mysql/include/mysql -DDBD_MYSQL_INSERT_ID_I
S_GOOD -g  -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_TS_ERRNO -xO3 -xspace -xildoff    -DVERSION=\"3.0006\"  -DXS_VERSION=\
"3.0006\" -KPIC "-I/usr/perl5/5.8.4/lib/i86pc-solaris-64int/CORE"   dbdimp.c
cc: unrecognized option `-KPIC'
cc: language ildoff not recognized
cc: dbdimp.c: linker input file unused because linking not done
/usr/bin/perl -p -e "s/~DRIVER~/mysql/g" /usr/perl5/site_perl/5.8.4/i86pc-solaris-64int/auto/DBI/Driver.xst > mysql.xsi
/usr/bin/perl /usr/perl5/5.8.4/lib/ExtUtils/xsubpp  -typemap /usr/perl5/5.8.4/lib/ExtUtils/typemap  mysql.xs > mysql.xsc && mv 
mysql.xsc mysql.c
Warning: duplicate function definition 'do' detected in mysql.xs, line 224
Warning: duplicate function definition 'rows' detected in mysql.xs, line 567
cc -c  -I/usr/perl5/site_perl/5.8.4/i86pc-solaris-64int/auto/DBI -I/home/apps/mysql/mysql/include/mysql -DDBD_MYSQL_INSERT_ID_I
S_GOOD -g  -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_TS_ERRNO -xO3 -xspace -xildoff    -DVERSION=\"3.0006\"  -DXS_VERSION=\
"3.0006\" -KPIC "-I/usr/perl5/5.8.4/lib/i86pc-solaris-64int/CORE"   mysql.c
cc: unrecognized option `-KPIC'
cc: language ildoff not recognized
cc: mysql.c: linker input file unused because linking not done
Running Mkbootstrap for DBD::mysql ()
chmod 644 mysql.bs
rm -f blib/arch/auto/DBD/mysql/mysql.so
LD_RUN_PATH="/home/apps/mysql/mysql/lib/mysql:/lib:/usr/lib" /usr/bin/perl myld cc  -G dbdimp.o  mysql.o  -o blib/arch/auto/DBD
/mysql/mysql.so   -L/home/apps/mysql/mysql/lib/mysql -lmysqlclient -lz -lposix4 -lcrypt -lgen -lsocket -lnsl -lm   
cc: dbdimp.o: No such file or directory
cc: mysql.o: No such file or directory
*** Error code 1
make: Fatal error: Command failed for target `blib/arch/auto/DBD/mysql/mysql.so'

Since I was building the module with gcc, the compiler and linker got a bit confused when they were passed Sun studio compiler flags (i.e., -KPIC in this example). There are two fixes for this problem. If you want to build a single module with gcc, you can edit the Makefile that was produced by ‘perl Makefile.PL’, and remove the “-KPIC” and “-xO3 -xspace -xildoff” values from the following variables:

$ egrep ‘(KPIC|O3)’ Makefile
CCCDLFLAGS = -KPIC
OPTIMIZE = -xO3 -xspace -xildoff

If you want to use gcc to build all Perl modules on a system, you can permanently* remove the Sun Studio compiler references by adjusting the “cccdlflags” and “optimize” variables in /usr/perl5/5.8.4/lib/sun4-solaris-64int/Config.pm:

$ egrep ‘(KPIC|O3)’ Config.pm
cccdlflags=’-KPIC’
optimize=’-xO3 -xspace -xildoff’

Since I don’t want to support two compiler packages, I decided to use option #2 since gcc comes on the Solaris installation CDs.

* If you edit Config.pm, you should be aware that Solaris Perl patches will overwrite this file.

Tuning Apache for performance

I recently came across Colm MacCarthaigh’s Apache tuning presentation and technical white paper:

Tuning Apache and Linux for performance presentation:
http://www.stdlib.net/~colmmacc/Apachecon-EU2005/scaling-apache-presentation.pdf

Tuning Apache and Linux for performance paper:
http://www.stdlib.net/~colmmacc/Apachecon-EU2005/scaling-apache-handout.pdf

Colm is an admin at heanet, which runs some of the busiest web servers in the world. The presentation and white paper cover the entire software stack, which includes kernel, file system and of course Apache web server tuning. These are must reads for website administrators.