Fun times with the bash read function and subshells

There are a few shellisms that have bitten me over the years. One issue that has bitten me more than once is the interation of variable assignments when a pipe is used to pass data to a subshell. This annoyance can be easily illustrated with an example:

$ cat test

#!/bin/bash

grep MemTotal /proc/meminfo | read stat total size
echo $total

$ ./test

On first glance you would think that the echo statement would display the total amount of memory in the system. But alas, it produces nothing. The reason this occurs is because the grep output is piped to read which is run in a subshell. Read assigns the values passed from grep to the variables, but once read is finished the subshell it is running inside of will exit and the contents of stat, total and size will be lost.

To work around this we can implement one of the solutions proposed in the bash FAQ. Here is my favorite for cases similar to this:

$ cat test

#!/bin/bash

foo=$(grep MemTotal /proc/meminfo)
set -- $foo
total=$2
echo $total

$ ./test
8128692

This works because the output is stored in the variable foo and processed inside the current shell. No susbshells are created so there is no way for the variables to get nuked. If you haven’t read the bash FAQ, gotchas pages or Chris Johnson’s book you are definitely missing out. I still encounter goofy shell-related issues but am now able to immediately identify most of them since I’ve flooded myself with shell-related information. :) So what is your biggest annoyance with the various shells?

Converting time since the epoch to a human readable string

I was parsing some Netbackup logs today, and needed a way to convert the time since the epoch into a human readable string. A while back I read about the various forms of input that can be passed to the GNU date’s “-d” option, one of these being the time since the epoch:

$ date -d @1263408025
Wed Jan 13 13:40:25 EST 2010

This solved my issue, though unfortunately it’s not super portable. Food for thought.

Serve out content over HTTP from your cwd immediatly

Ever want to immediatly serve content from a specific directory over HTTP, but didn’t want to bother messing with httpd.conf or other webserver configiurations?

If you’ve got Python installed, this is a snap.  Execute python with the SimpleHTTPServer module, using port 8080 so there isn’t a need to elevate privs to root.

$ python -m SimpleHTTPServer 8080
Serving HTTP on 0.0.0.0 port 8080 …

Sure enough, pointing a browser to the IP address :8080 of the box hits my home directory listing.  Super easy, super fast, super simple!

I use this to serve content to my PS3.  The PS3 doesn’t support NFS or CIFS, so to download content to the hard drive, the best method is by pulling it over HTTP with the embedded web brower.   On my MacBook, I change into the directory containing whatever media I want to transfer, fire up HTTP, and suck it down to the hard drive on the PS3.  Nice!

Awesome use of read-only variables in bash scripts

I was reading through Jim Perrin’s CentOS hardening article, and saw one super interesting use of read-only bourne shell variables. If you have users that are frequently logging in and staying idle for days and or weeks, you can add a readonly TMOUT variable to /etc/profile:

$ echo “readonly TMOUT=3600” >> /etc/profile

The TMOUT variable controls the amount of time a user can be idle before the system logs them out. Since the variables in /etc/profile will be applied to the environment before a users .bash* and .profile files, you can be sure that users can’t override (this doesn’t address users who use C shells, but that can be addresses similarly) the read-only TMOUT variable and stay idle for longer periods of time. This also works well for HISTFILE environment variable, which is mentioned in the article. Great article Jim!

Implementing locks in shell scripts

I have been working on a shell script that manages lxc-containers, and came across a use case last where it is possible for two yum processes to interfere with each other. To ensure that only one yum process is run at a single point in time, I implemented file based locks using flock(1). Flock makes this super easy, since it has a “-x” option to create an exclusive lock (this is the default), and a “-n” option which causes flock to exit with a return code of 1 if it can’t obtain the lock. This allow code similar to the following to be used to protect sensitive areas:

 
(
    flock -n -x 200

    if [ $? != "0" ]; then
        echo "ERROR: Unable to acquire the yum lock. Is another yum running?"
        exit 1
    fi    
    
    # Do yum stuff here

) 200>${TMP_DIR}/yum.lck



In my case this tiny bit of code ensures that only one yum process is able to run, which helps keep my package database in a sane state.