Archive
Posts in Shell
Creating a set of random numbers from the command line
This past weekend I was doing some database testing and needed to generate some random numbers to populate a table. My typical go-to utility for generating one random number is head piped to od and tr: This works well and can be aded to a loop to get more than one number. But I was curious if there was a native Linux utility available to do this work. A quick poke through the Linux man pages turned up the coreutils shuf utility: This was exactly what I was after…
$ read more →Which file descriptor (STDOUT, STDERR, etc.) is my application writing to?
When developing ansible playbooks a common pattern is to run a command and use the output in a future task. Here is a simple example: In the first task dnf will run and the output from the command will be placed in either STDOUT or STDERR. But how do you know which one? One way is to add a debug statement to your playbook: Once the task runs you can view the stderr and stdout fields to see which of the two is populated: In the output above we can see that stderr is empty and stdout contains the output from the command…
$ read more →Using xargs and lscpu to spawn one process per CPU core
One of my friends reached out to me earlier this week to ask if there was an easy way to run multiple Linux processes in parallel. There are several ways to approach this problem but most of them don't take into account hardware cores and threads. My preferred solution for CPU intensive operations is to use the xargs parallel option ("-P") along with the CPU cores listed in lscpu. This allows me to run one process per core which is ideal for CPU intensive applications…
$ read more →Using awk character classes to simplify parsing complex strings
This week I was reading a shell script in a github repository to see if it would be good candidate to automate a task. As I was digging through the code I noticed a lengthy shell pipeline to parse a string similar to this: Here is the code they were using to extract the string "gorp": After my eyes recovered I thought this would be a good candidate to simplify with awk character classes. These are incredibly useful for applying numerous field separators to a given line of input. I took what the original author had and simplified it to this: The argument passed to the field separated option (-F) contains a list of characters to use as delimiters…
$ read more →What are bash return codes > 128?
I like to keep PS1 pretty simple. I like to see the host I'm working on, the directory I'm in, the user I'm currently logged in as and the return code from the last command I executed. This is easy to cobble together with bash escape s equences: The other day I was testing some new ansible playbooks and saw a return code of 130. Several years ago I read learning the bash shell and recalled something about the magic number 128…
$ read more →