There are a few shellisms that have bitten me over the years. One issue that has bitten me more than once is the interation of variable assignments when a pipe is used to pass data to a subshell. This annoyance can be easily illustrated with an example:
$ cat test
grep MemTotal /proc/meminfo | read stat total size
On first glance you would think that the echo statement would display the total amount of memory in the system. But alas, it produces nothing. The reason this occurs is because the grep output is piped to read which is run in a subshell. Read assigns the values passed from grep to the variables, but once read is finished the subshell it is running inside of will exit and the contents of stat, total and size will be lost.
To work around this we can implement one of the solutions proposed in the bash FAQ. Here is my favorite for cases similar to this:
$ cat test
foo=$(grep MemTotal /proc/meminfo)
set -- $foo
This works because the output is stored in the variable foo and processed inside the current shell. No susbshells are created so there is no way for the variables to get nuked. If you haven’t read the bash FAQ, gotchas pages or Chris Johnson’s book you are definitely missing out. I still encounter goofy shell-related issues but am now able to immediately identify most of them since I’ve flooded myself with shell-related information. :) So what is your biggest annoyance with the various shells?
I was parsing some Netbackup logs today, and needed a way to convert the time since the epoch into a human readable string. A while back I read about the various forms of input that can be passed to the GNU date’s “-d” option, one of these being the time since the epoch:
$ date -d @1263408025
Wed Jan 13 13:40:25 EST 2010
This solved my issue, though unfortunately it’s not super portable. Food for thought.
I came across the Advanced Bash Scripting guide while checking through my RSS feeds this morning. It has a ton of great examples and goes pretty in-depth on Bash scripting features. A good read for sure. =)
Ever want to immediatly serve content from a specific directory over HTTP, but didn’t want to bother messing with httpd.conf or other webserver configiurations?
If you’ve got Python installed, this is a snap. Execute python with the SimpleHTTPServer module, using port 8080 so there isn’t a need to elevate privs to root.
$ python -m SimpleHTTPServer 8080
Serving HTTP on 0.0.0.0 port 8080 …
Sure enough, pointing a browser to the IP address :8080 of the box hits my home directory listing. Super easy, super fast, super simple!
I use this to serve content to my PS3. The PS3 doesn’t support NFS or CIFS, so to download content to the hard drive, the best method is by pulling it over HTTP with the embedded web brower. On my MacBook, I change into the directory containing whatever media I want to transfer, fire up HTTP, and suck it down to the hard drive on the PS3. Nice!
I was reading through Jim Perrin’s CentOS hardening article, and saw one super interesting use of read-only bourne shell variables. If you have users that are frequently logging in and staying idle for days and or weeks, you can add a readonly TMOUT variable to /etc/profile:
$ echo “readonly TMOUT=3600” >> /etc/profile
The TMOUT variable controls the amount of time a user can be idle before the system logs them out. Since the variables in /etc/profile will be applied to the environment before a users .bash* and .profile files, you can be sure that users can’t override (this doesn’t address users who use C shells, but that can be addresses similarly) the read-only TMOUT variable and stay idle for longer periods of time. This also works well for HISTFILE environment variable, which is mentioned in the article. Great article Jim!
I have been working on a shell script that manages lxc-containers, and came across a use case last where it is possible for two yum processes to interfere with each other. To ensure that only one yum process is run at a single point in time, I implemented file based locks using flock(1). Flock makes this super easy, since it has a “-x” option to create an exclusive lock (this is the default), and a “-n” option which causes flock to exit with a return code of 1 if it can’t obtain the lock. This allow code similar to the following to be used to protect sensitive areas:
flock -n -x 200
if [ $? != "0" ]; then
echo "ERROR: Unable to acquire the yum lock. Is another yum running?"
# Do yum stuff here
In my case this tiny bit of code ensures that only one yum process is able to run, which helps keep my package database in a sane state.