Fun times with the bash read function and subshells

There are a few shellisms that have bitten me over the years. One issue that has bitten me more than once is the interation of variable assignments when a pipe is used to pass data to a subshell. This annoyance can be easily illustrated with an example:

$ cat test

#!/bin/bash

grep MemTotal /proc/meminfo | read stat total size
echo $total

$ ./test

On first glance you would think that the echo statement would display the total amount of memory in the system. But alas, it produces nothing. The reason this occurs is because the grep output is piped to read which is run in a subshell. Read assigns the values passed from grep to the variables, but once read is finished the subshell it is running inside of will exit and the contents of stat, total and size will be lost.

To work around this we can implement one of the solutions proposed in the bash FAQ. Here is my favorite for cases similar to this:

$ cat test

#!/bin/bash

foo=$(grep MemTotal /proc/meminfo)
set -- $foo
total=$2
echo $total

$ ./test
8128692

This works because the output is stored in the variable foo and processed inside the current shell. No susbshells are created so there is no way for the variables to get nuked. If you haven’t read the bash FAQ, gotchas pages or Chris Johnson’s book you are definitely missing out. I still encounter goofy shell-related issues but am now able to immediately identify most of them since I’ve flooded myself with shell-related information. :) So what is your biggest annoyance with the various shells?

4 thoughts on “Fun times with the bash read function and subshells”

  1. I got bit the other day when assigning variables the output of commands and then attempting to concatenate them together by echo’ing them on the same line, like this (not full accuracy, going by memory here):

    NETWORKINFO=`ifconfig eth0| tr “\n” ” “`
    IPADDR=`echo $NETWORKINFO | awk ‘{print $2}’`
    MAC=`echo $NETWORKINFO | awk ‘{print $10}’`
    echo -e “eth0:\t$IPADDR\t$MAC”

    Which resulted in something like:

    192.168.0.1-22-22

    Needless to say, this made me nuts until I finally added -x to my #!/bin/bash she-bang, which showed me that $IPADDR and $MAC had ‘\r’ characters at the end of them, which led to this crazy echo statement. I added a piped sed ‘s/\r//g’ after each variable definition to clear this up. I’m sure there’s a better way, but a recent example to contribute.

    Thanks for the always handy tips!

  2. Or you could use the bash single line HERE document (which I only learnt of a couple of weeks ago; though it is of course mentioned in the BashFAQ referenced)

    read stat total size <<< $(grep MemTotal /proc/meminfo)
    echo $total

  3. > The reason this occurs is because the grep output is piped to read which is run in a subshell.

    That’s true, but not generally for shells. SUSv3 doesn’t specify that the last statement in pipe needs to be executed in a subshell. zsh executes the last statement in current shell environment, so read works as expected.

    @Tony: what’s the point of tr in your script, when you don’t use quoting later in echo?

  4. Something noteworthy: the subshell pitfall not only lurks on the right hand side of the pipe, but also on the left hand side.

    #!/bin/bash
    #
    # subshells.sh:
    # Demonstrates subshell behavior upon pipes. What’s to be expected, is
    # that the RHS of the pipe will be run in a subshell and can’t modify
    # global state. But, surprise, that’s also the case for the LHS.
    #

    function lhs() { STATE_LHS=1; }
    function rhs() { STATE_RHS=1; cat; }

    lhs | rhs
    echo “\$STATE_LHS: $STATE_LHS, \$STATE_RHS: $STATE_RHS”

    Can for example get you, when you have a function that needs to access global state and you start to pipe its output into a logging function. All of a sudden, the global state changes won’t be there any more.

Leave a Reply

Your email address will not be published. Required fields are marked *