Blog O’ Matty

I recently got sick of the Safari spinning ball, and decided to conduct some research to speed up my favorite web browser. After reading numerous posts on the Apple discussion board, I made the following changes to significantly boost page rendering time:

1. Add Mike’s Ad Blocking Hosts file to /etc/hosts and restart lookupd

2. Purge the Safari cache from Safari -> Clear Cache

3. Remove old RSS articles by clicking Preferences -> RSS -> Remove Now

Once these three actions were complete, Safari was back to post install speeds! Nice!

I just had another awful experience with a large corporation’s customer service, and this got me thinking about how crappy customer service has gotten. There seems to be a complete lack of problem ownership, representatives are paid to get you off the phone, and in most cases the problem is pawned off on the individual who is trying to get help. When I was a wee lad, I remember customer support representatives being friendly and trying to assist with actual problem resolution regardless of the circumstances. Has capitolism truly eroded the desire to care about people when assisting them over the phone?

I came across my new favorite error message last night while debugging some issues with the Solaris printstack() function:

$ gdb -q /var/tmp/apache2/bin/httpd

(gdb) shell ps -ef | grep gdb
matty 9216 8963 1 20:30:34 pts/1 0:00 gdb -q /var/tmp/apache2/bin/httpd
matty 9217 9216 0 20:30:41 pts/1 0:00 bash -c ps -ef | grep gdb

(gdb) attach 9216
Attaching GDB to itself is not a good idea…

You gotta love some geek humor. :)

If you have ever done a long listing of /usr/lib on a Linux system, you probably choked and asked yourself what the f$%^ is this mess? After reading through Peter Seebach’s article Dissecting Shared Libraries, things don’t seem so bad, and the large number of files actually starts to make sense. Step one in sorting out the library madness requires making sense of the digits (the major and minor revision numbers) that appear in the shared library file name. Peter’s article clarifies this with the following description:

“One of the potential advantages of dynamic linking, however, is in fixing bugs. It’d be nice if you could fix a bug in the library and not have to recompile a thousand programs to take advantage of that fix. So sometimes, you want to link to a newer version. Unfortunately, that creates some cases where you want to link to the newer version and some cases where you’d rather stick with an older version. There is a solution, though — two kinds of version numbers:

So under most circumstances, it is safe to load a library with the same major number and a higher minor number; consider it an unsafe practice to load a library with a higher major number.”

This makes sense, and seems like a good thing for systems that don’t want to break applications. Now what about all those links in /usr/lib? Peter provides the following description:

To prevent users (and programmers) from needing to track library numbers and updates, the system comes with a large number of symbolic links. In general, the pattern is that

If you haven’t checked out IBM’s redbook or Linux developerworks sites, I highly recommend doing so. There are numerous awesome pieces of documentation spanning numerous technologies.

I installed Spam Karma 2 a few weeks back, and am amazed at how well it works! Not only has it captured every piece of SPAM posted to my BLOG, it automatically purges them so I don’t have to do anything myself (the “purging” feature is a configuration option). Thanks Dr. Dave!

This past week I wanted to use the Solaris cfgadm utility to unconfigure a few LUNs. When I ran ‘cfgadm -al’, I noticed that the FC adaptors were not visible in the cfgadm output:

$ cfgadm -al

Ap_Id                          Type         Receptacle   Occupant     Condition
c2                             scsi-bus     connected    configured   unknown
c2::dsk/c2t0d0                 CD-ROM       connected    configured   unknown
usb0/1                         unknown      empty        unconfigured ok
usb0/2                         unknown      empty        unconfigured ok

This seemed odd, since I could see the controllers in the vxdmpadm output:

$ vxdmpadm listctlr all

CTLR-NAME       ENCLR-TYPE      STATE      ENCLR-NAME
=====================================================
c1              Disk            ENABLED      Disk
c0              Disk            ENABLED      Disk
c4              EMC             ENABLED      EMC0
c3              EMC             ENABLED      EMC0
c4              EMC_CLARiiON    DISABLED     EMC_CLARiiON0
c3              EMC_CLARiiON    DISABLED     EMC_CLARiiON0

Since the controllers in question were Emulex adaptors, I read through the Emulex admin guide and found that the platform and FC adaptor had to be DR aware to support configure/unconfigure operations. Since I couldn’t locate a “DR aware” label in our vendors documentation, I decided to open a ticket to see if the servers supported cfgadm. After a week or two of chatting with support, our vendor indicated that we would need to use the Sun Leadville drivers to configure and unconfigure LUNs with Emulex adaptors in Solaris systems. This was awesome news, and I am super happy that the lpfc driver will now be installed in /kernel/drv by default! Niiiiiiice.

While reviewing some system information last week , I came across a discrepancy (which I would think is a bug — at least according to the idle time description in the w manual pages) in the Solaris ‘w’ output:

$ date
Wed Dec 21 10:19:59 EST 2005

$ w

10:19am  up 5 day(s), 19:52,  2 users,  load average: 0.02, 0.03, 0.04
User     tty           login@  idle   JCPU   PCPU  what
user1234 pts/1        10:19am    51                myapp

$ who -u

user1234   pts/1        Dec 21 10:19   .    10503      (1.2.3.4)

The user “user1234″ logged in at 10:19, and according to w has been idle for 51 minutes (since they have been logged in for one minute or less, they obviously cannot be idle for 51 minutes). A quick check of w.c reveals that idle time is calculated using the atime from a stat of /dev/pts/[1-9]+:

if (stat(ttyname, &stbuf) != -1) {
    lastaction = stbuf.st_atime;
    diff = now - lastaction;
    diff = DIV60(diff);

While who.c uses the mtime from a stat of /dev/pts/[1-9]+:

if ((rc = stat(path, stbufp)) != -1) {
    idle = timnow - stbufp->st_mtime;

To understand when atime and mtime are updated, I read through the Solaris stat(2) manual page:

st_atime        Time  when  file  data  was  last  accessed.
                Changed by the following functions: creat(),
                mknod(), pipe(), utime(2), and read(2).

st_mtime        Time when data was last modified. Changed by
                the  following  functions: creat(), mknod(),
                pipe(), utime(), and write(2).

Since there seems to be some inconsistency (atime in w and mtime in who) in the source code, I thought I would look at my favorite OS (FreeBSD / Darwin) to see how they implemented w:

# From FreeBSD source code
touched = stp->st_atime;
if (touched < ep->utmp.ut_time) {
   /* tty untouched since before login */
   touched = ep->utmp.ut_time;
}
if ((ep->idle = now - touched) < 0)
   ep->idle = 0;

So this explains it. Since the user logged in and didn’t do anything, the pseudo-terminal would have been written to but not necessarily read from (I assume since the pseudo-terminal is setup by the terminal driver after the user is authenticated). FreeBSD fixes this problem by adding an additional check to compare idle time to the time the user logged in, which seems to address this issue. I posted this issue to the Solaris discussion forums, but unfortunately no one replied. Since the putback model with opensolaris is still somewhat new and requires an excessive amount of work on the contributor’s behalf, I think I will just chock this up to fun and move my code analysis efforts back to Apache and FreeBSD (which is a super neat OS).

I just purchase The Smashing Pumpkin’s rendition of “Christmastime,” and truly dig this tune. If you have access to iTunes, check it out and let me know what you think.