Blog O’ Matty

I submitted my DSL order to Speakeast last week, and have been receiving daily status e-mails similar to the following:

BROADBAND ORDER IN PROCESS

Broadband Installation Status:

Stage 1: Order Received
Your order has been received by Speakeasy.
Stage 1: COMPLETED.

Stage 2: Order Placed
We have placed your order with the vendor.
Stage 2: COMPLETED.

Stage 3: Install Confirmation
Stage 3: PENDING.

Stage 4: IP Address & Circuit Configured
We have configured our router for your impending circuit.
Stage 4: COMPLETED.

Stage 5: Configuration Information Sent
We have sent you machine configuration information.
Stage 5: COMPLETED.

Stage 6: Install Confirmed
Stage 6: PENDING.

Stage 7: Set Up Fee Received
Stage 7: PENDING.

Stage 8: Installation Complete
Stage 8: PENDING.

This is amazingly cool, and is super valuable for understanding what is happening with your order. I imagine this saves them a good deal of support calls, and keeps the customer in the loop along the way. Way to go Speakeasy!

To address the recent vulnerabilities with fetchmail, I wanted to upgrade to the latest secure version. Having performed this operation numerous times in the past, I used the following process to download and compile the source code:

$ wget http://download.berlios.de/fetchmail/fetchmail-6.2.5.2.tar.gz

$ configure –with-ssl=/usr/local/ssl

$ make
[ …… ]
cd intl; make
make: Fatal error in reader: Makefile, line 210: Badly formed macro assignment
Current working directory /tmp/fetchmail-6.2.5.2/intl
*** Error code 1
make: Fatal error: Command failed for target `@INTLDEPS@’

Gak! The make process failed when attempting to include the NLS (Native Language Support) dependencies. I checked through the fetchmail FAQ, and they recommend disabling native language support with the “–disable-nls” option to fix this issue. Armed with this information I once again attempted to build fetchmail from source:

$ make clean

$ configure –with-ssl=/usr/local/ssl –disable-nls

$ make

[ …… ]
cd intl; make
make: Fatal error in reader: Makefile, line 210: Badly formed macro assignment
Current working directory /tmp/fetchmail-6.2.5.2/intl
*** Error code 1
make: Fatal error: Command failed for target `@INTLDEPS@’

Oh good grief! Not being one to give up, I read through the Makefile and located the NLS dependencies referenced by INTLDEPS. Since I don’t need NLS support, I removed* the NLS dependencies from the Makefile, and once again attempted to compile from source code:

$ make clean

$ configure –with-ssl=/usr/local/ssl –disable-nls

$ grep INTLDEPS Makefile
fetchmail: $(objs) @INTLDEPS@

$ vi Makefile

$ grep INTLDEPS Makefile
fetchmail: $(objs)

$ make
[ ….. ]
gcc -DHAVE_CONFIG_H -DLOCALEDIR=”/usr/local/share/locale” -c -I. -I. -I./intl -I./intl -I/usr/local/ssl/include -I/usr/ kerberos/include -O md5c.c
gcc -L/usr/local/ssl/lib rcfile_y.o rcfile_l.o socket.o getpass.o pop2.o pop3.o imap.o etrn.o odmr.o fetchmail.o idle.o env.o options.o daemon.o driver.o transact.o sink.o rfc822.o smtp.o xmalloc.o uid.o mxget.o md5ify.o cram.o kerberos.o gssapi.o opie.o rpa.o interface.o netrc.o base64.o report.o unmime.o conf.o checkalias.o smbdes.o smbencrypt.o smbmd4.o smbutil.o ipv6-connect.o lock.o md5c.o -lnsl -lsocket -lintl -lresolv -lssl -lcrypto -ll -o fetchmail
cd po; make

Success! Don’t you love it when a software package comes together! :)

* The fetchmail archives indicated that this issue *may* be attributable to a bug in the native Solaris make utility, and *might* be fixed by installing GNU make. I don’t want to support another software package, so disabling NLS was my best option.

In continuing on with my commitment to describe my favorite network utilities, I bring to you mtr:

$ mtr -r -c 10 mail.prefetch.net

HOST: me                           Loss%   Snt   Last   Avg  Best  Wrst StDev
  1. 10.238.4.1                    0.0%    10    9.2   8.2   5.4  12.6   2.1
  2. 68.86.108.13                  0.0%    10    9.2   9.4   6.3  14.9   2.8
  3. 68.86.106.45                  0.0%    10   15.1  10.1   6.9  17.5   3.5
  4. 68.86.106.13                  0.0%    10   14.0  11.0   5.7  22.1   4.8
  5. 68.86.106.9                   0.0%    10   14.6  11.4   7.2  18.5   3.6
  6. 12.118.120.89                 0.0%    10   10.2  10.3   8.2  14.6   2.2
  7. tbr1-p012201.attga.ip.att.ne  0.0%    10   23.8  25.8  21.8  33.9   3.6
  8. tbr2-cl1.wswdc.ip.att.net     0.0%    10   27.3  26.9  23.7  33.4   3.0
  9. ggr2-p390.wswdc.ip.att.net    0.0%    10   23.0  27.6  22.6  41.0   5.6
 10. so6-3-0-2488M.ar1.DCA3.gblx.  0.0%    10   26.0  30.4  23.9  43.3   6.3
 11. so0-0-0-622M.ar2.CLE1.gblx.n  0.0%    10  130.3  49.8  37.7 130.3  28.4
 12. Enet-Inc-Internet.so-0-3-3.a  0.0%    10   51.2  45.9  42.5  54.9   4.0
 13. extreme.xlhost.com            0.0%    10   44.2  67.1  41.4 261.4  68.3
 14. mail.prefetch.net            0.0%    10   48.3  56.6  41.5 172.9  40.9

mtr allows you to view hop-by-hop latency metrics, and is invaluable for finding busy devices between two endpoints. Some routers will place low QOS priorities on ICMP traffic, so it is important to capture traffic at various intervals to see if a device is truly overloaded.

While reading through my daily mailing lists, I came across the following post. At first I was a bit nervous about the widespread chaos this would cause to OpenBSD users. After further analysis, it dawned on me that this will actually FIX more than it will break. Since software is getting worse by the day, it is nice to see that an Operating System takes pride in protecting users from crappy software.

While analzying some throughput issues this weekend, it came to my attention that I have never written about my favorite network analysis tools. I have 10 - 12 utilities that I cannot live without, and plan to cover each utility over the next few weeks (stay tuned!). Tonight, you are in luck! I present to you nicstat:

$ nicstat 5

    Time   Int   rKb/s   wKb/s   rPk/s   wPk/s    rAvs    wAvs   %Util     Sat
00:27:34  hme0    1.50   70.79   21.96   57.29   69.93 1265.28    0.59    0.00
00:27:39  hme0    1.38   68.24   20.15   55.46   70.28 1259.90    0.57    0.00
00:27:44  hme0    1.33   72.09   19.37   58.51   70.45 1261.69    0.60    0.00
00:27:49  hme0    1.57   70.89   22.36   58.29   71.73 1245.39    0.59    0.00
00:27:54  hme0    1.40   69.54   20.36   56.48   70.24 1260.69    0.58    0.00
00:27:59  hme0    1.32   69.52   18.96   56.29   71.05 1264.75    0.58    0.00
00:28:04  hme0    1.51   70.82   22.16   57.09   69.89 1270.31    0.59    0.00

This nifty utility reports utilization metrics for each Ethernet interface in a Solaris server. nicstat is written in C and Perl, and the program header contains the following descriptions of each field that is displayed:

Int            Interface
rKb/s          read Kbytes/s
wKb/s          write Kbytes/s
rPk/s          read Packets/s
wPk/s          write Packets/s
rAvs read      Average size, bytes
wAvs write     Average size, bytes
%Util          %Utilisation (r+w/ifspeed)
Sat            Saturation (defer, nocanput, norecvbuf, noxmtbuf)

Brendan Gregg creates some amazing tools, and I highly recommend sending him lots of beer and chocolate (or thank you emails if your cheap)!

When mirrors and RAID5 devices are created with Veritas Volume Manager, the volume contents need to be synchronized to an initial consistent state (e.g., parity and mirrors need to be synchronized). Veritas assigns a unique task id to each operation, and allows tasks to be monitored and displayed with the vxtask(1m) utility:

$ vxtask list

TASKID  PTID TYPE/STATE    PCT   PROGRESS
   172           ATCOPY/R 05.57% 0/209698816/11689984 PLXATT oravol04 oravol04-02 oradg

Since synchronization can be extremely I/O intensive, you might want to pause a synchronization task until a later time. This is easily accomplished by passing the task id to vxtask(1m)’s pause option:

$ vxtask pause 172

$ vxtask list

TASKID  PTID TYPE/STATE    PCT   PROGRESS
   172           ATCOPY/P 05.76% 0/209698816/12077056 PLXATT oravol04 oravol04-02 oradg

Once you are ready to resume the task, you can use vxtask(1m)’s resume option:

$ vxtask resume 172

$ vxtask list

TASKID  PTID TYPE/STATE    PCT   PROGRESS
   172           ATCOPY/R 06.02% 0/209698816/12613632 PLXATT oravol04 oravol04-02 oradg

This is super useful, and it looks to have caught on.

While cleaning up some bash scripts tonight, I started to wonder which of the following two substitutions is clearer:

$ echo ${foo=$(ls)}
Desktop Documents Library Movies Music Pictures Public SUMS Sites ae play

or

$ echo ${foo=`ls`}
Desktop Documents Library Movies Music Pictures Public SUMS Sites ae play

I really think the second form is a bit more clear, though it could throw off people who get ` and ‘ mixed up! Choices, choices!

Veritas Volume Manager comes with a wide variety of command line utilities, which can be used to create, delete and maintain Veritas objects. When operations are performed with the CLI and no disk group is passed to the “-g” (disk group to use) option, the command will default to using the value assigned to defaultdg. This value of defaultdg is stored in the /etc/vx/volboot file:

$ grep defaultdg /etc/vx/volboot
defaultdg oradg

If you would like to change the default disk group, you can use vxdctl(1m)’s “defaultdg” option:

$ vxdctl defaultdg oof

To verify that the value was changed, you can run vxdg(1m) with the “defaultdg” option:

$ vxdg defaultdg
oof

This can save a lot of typing when creating new Veritas objects!