Running an SSH client inside your Firefox web browser

I recently came across FireSSH, which is an SSH client that runs inside Firefox. The FireSSH plug-in allows you to create an SSH connection to a remote host using just a web browser, and I can see all kinds of uses for this! The plug-in is written entirely in javascript, and uses a couple of features that require Firefox 4 (Firefox 4 rocks, so upgrading to it should be a no brainer). To access the plug-in, you will first need to surf over to the mozilla plug-in site and install it using your choice of installation options. Once installed, you can run FireSSH by opening Firefox and running Tools->FireSSH. This will present the following screen:

null

Once you fill in the connection attributes you will we logged in and presented with an interactive SSH window similar to this:

null

FireSSH also supports public-key authentication, so you don’t need to rely on passwords to get into your hosts. This is a killer app, and I LOVE Firefox 4. It loads quickly, it’s super fast and the UI layout rocks. Niiiice!

Configuring NSCD to cache DNS host lookups

I haven’t really spent that much time configuring nscd, so I thought I would take a crack at it this morning while sipping my cup of joe.

Looking at one of my production hosts, I queried for the “host” cache statistics. This is the nscd cache which keeps DNS lookups. With the nscd daemon running, you can query the size / performance of the caches with the -g flag.


$ nscd -g   
CACHE: hosts

         CONFIG:
         enabled: yes
         per user cache: no
         avoid name service: no
         check file: yes
         check file interval: 0
         positive ttl: 0
         negative ttl: 0
         keep hot count: 20
         hint size: 2048
         max entries: 0 (unlimited)

         STATISTICS:
         positive hits: 0
         negative hits: 0
         positive misses: 0
         negative misses: 0
         total entries: 0
         queries queued: 0
         queries dropped: 0
         cache invalidations: 0
         cache hit rate:        0.0

Ugh! No bueno! So, out of the box, nscd isn’t configured to cache anything. This means that every request this machines does is hitting a DNS server in /etc/resolv.conf. This adds overhead to our DNS servers, and increases the time the applications running on this box have to wait to do something useful. Looking at the configuration options for the “host” cache…


$ grep hosts /etc/nscd.conf 
        enable-cache            hosts           yes
        positive-time-to-live   hosts           0
        negative-time-to-live   hosts           0
        keep-hot-count          hosts           20
        check-files             hosts           yes

Hm. So positive-time-to-live is set to zero. Looking at the man page for /etc/nscd.conf…

positive-time-to-live cachename value
Sets the time-to-live for positive entries (successful
queries) in the specified cache. value is in integer
seconds. Larger values increase cache hit rates and
reduce mean response times, but increase problems with
cache coherence. Note that sites that push (update) NIS
maps nightly can set the value to be the equivalent of
12 hours or more with very good performance implica-
tions.

Ok, so lets set the cache age here to 60 seconds. It seems like a decent starting value… After making this change, and restarting the daemon, here are some performance statistics of the host cache.


CACHE: hosts

         CONFIG:
         enabled: yes
         per user cache: no
         avoid name service: no
         check file: yes
         check file interval: 0
         positive ttl: 60
         negative ttl: 0
         keep hot count: 20
         hint size: 2048
         max entries: 0 (unlimited)

        STATISTICS:
         positive hits: 143
         negative hits: 1
         positive misses: 20
         negative misses: 41
         total entries: 20
         queries queued: 0
         queries dropped: 0
         cache invalidations: 0
         cache hit rate:       70.2

Crazy. Enabling only a 60s cache, we are now performing 70% less DNS lookups. This is going to have a significant performance improvement. By default, the setting keep-hot-count is set to 20. This is the number of objects allowed in the “hosts” cache. Looking at the man page for nscd.conf…


keep-hot-count cachename value

This attribute allows the administrator to set the
number of entries nscd(1M) is to keep current in the
specified cache. value is an integer number which should
approximate the number of entries frequently used during
the day.

So, raising positive-time-to-live to say, 5 minutes wont have much value unless keep-hot-count is also raised. The cache age, and the number of objects within the cache both need to be increased. Doing so will help keep your DNS servers idle, and applications happy.

Continuing to tail a file after it’s been rotated

I use xtail and tail quite a bit to debug problems. Sometimes when I’m up late at night troubleshooting issues, the log rotation jobs will kick off and my tail session will stop updating (tail will follow the file descriptor associated with the file by default, not the name of the file). Since this can make debugging an issue a bit confusing, I’ve made it a habit to alias tail to something like this:

$ alias tailer=”tail –follow=name –retry”

The “–follow=name” option will cause tail to follow the name of the file and not the file descriptor, and the “–retry” option will cause tail to try to re-open the file if it’s not accessible (log rotation events will cause the file to change). It’s become second nature to run tailer now, and I don’t get that “WTF?!?” moment at 2am now. ;)

Using the Linux arping utility to send out gratuitious ARPs

I managed a number of Redhat and Heartbeat clusters. On a couple of occassions the services that manage the virtual IPs have misbehaved, and the storage has ended up on one node and the virtual IP on another. To fix this I need to manually move the virtual IP to the host it belongs on, and then issue a gratuitous ARP so other hosts on the network clear their ARP cache and use the MAC address associated with the device the virtual IP now resides on.

The Linux arping utility can be used to send out a gratuitious ARP (an “ARP Request” or “ARP Response” is the actual item sent) to update hosts on your network . To use arping to update the ARP cache on all of the devices on the local layer-3 network, you can run arping with the -U option (unsolicited ARP mode), the “-I” option (interface to send the gratuitious ARP out on) and the IP that is assigned to the interface you want the ARP cache entry updated for:

$ arping -U -I eth0 186.168.86.100

Once executed, the arping command will cause a layer-2 broadcast message to be sent, which should cause the ARP caches to be updated on all of the hosts on the local network (ARP replies are sent to the layer-2 broadcast address, so all hosts on your network should receive these). This assumes that your hosts aren’t configured to ignore unsolicited ARP requests. ;) Arping can also be used to detect duplicate IPs when run with the “-D” (duplicate address detection mode) option. This is a handy tool that everyone should be aware of.

A simple and easy way to restart dropped SSH sessions on Linux servers

I frequently use OpenSSH port forwards to move around my various networks, and there is nothing worse than an SSH connection dropping when you make heavy use of them. Recently I came across the autossh utility, which provides a crazy easy way to monitor ssh sessions and restart them when they are reset or dropped. To use this awesome little tool, you can invoke autossh with the options you would normally pass to your ssh client:

$ AUTOSSH_LOGFILE=$HOME/autossh.log

$ AUTOSSH_GATETIME=60

$ autossh -D -L 8000 foo@bar.com

In the example above I am creating a dynamic local port forward that will accept connections on localhost:8000. To change how autossh manages the sessions and logs it creates, you can set one of more environment variables. The variables are documented in the autossh(1) manual page, and allow you control how data is logged, connection attributes and how keepalives operate. If autossh is in use and your ssh connection dies for any reason, it will be restarted and your port forwards will be transparently restored. Bada boom, bada bing!

New zone features in Solaris 10 update 9

I just finished reading through Frank Batschulat’s slide deck of new zone features in Solaris 10 update 9. The new update on attach enhancements look extremely useful, and I have a few systems I will have to try to p2v using the tools that now ship with Solaris. If you are using zones, you should check out his presentation. Good stuff!