Creating happy little block devices with UDEV

I’m a long time admirer of Bob Ross and the amazing paintings he produced on his hit tv show the joy of painting. I’m equally a fan of udev and the power it places in administrators hands. While Bob painted amazing clouds, seascapes and mountains with a swipe of his brush I’m able to make /dev just as beautiful with a few keyboard strokes (I guess that makes my keyboard my easel).

This past weekend I decided to clean up and automate the device creation process on several of my database servers. These servers had hundreds of mapper devices which were defined in multipath {} sections in /etc/multipath.conf. The aliases used in these blocks had upper and lower case names so the udev rules file had become a bit of a mess. Here is a snippet to illustrate:

multipath {
    wwid  09876543210
    alias happydev_lun001

multipath {
    wwid  01234567890
    alias Happydev_lun002

…. more devs ….

By default udev creates /dev/dm-[0-9]+ entries in /dev. I prefer to have the friendly names (happydev_lunXXX in this case) added as well so I don’t have to cross reference the two when I’m dealing with storage issues. To ensure that the friendly names in /dev were consistently named I created the following udev rule:

$ cd /etc/udev.rules.d && cat 90-happydevs.rules

ENV{DM_NAME}=="[Hh]appydev*", NAME="%c" OWNER:="bob", GROUP:="ross", MODE:="660", PROGRAM="/usr/local/bin/ $env{DM_NAME}"

The rule above checks the DM_NAME to see if it starts with the string [Hh]appydev. If this check succeeds UDEV will call the /usr/local/bin/ helper script which in turn calls tr to lower() the name passed as an argument. Here are the contents of the script:

$ cat /usr/local/bin/


echo ${1} | tr '[A-Z]' '[a-z]'

The STDOUT from gets assigned to the %c variable which I reference in the NAME key to create a friendly named device in /dev. We can run udevadm test to make sure this works as expected:

$ udevadm test /block/dm-2 2>&1 | grep DEVNAME
udevadm_test: DEVNAME=/dev/happydev_lun001

In the spirit of Bob Ross I udev’ed some pretty little block devices. :)

Using docker to build software testing environments

I’ve been on the docker train for quite some time. While the benefits of running production workloads in containers is well known, I find docker just as valuable for evaluating and testing new software on my laptop. I’ll use this blog post to walk through how I build transient test environments for software evaluation.

Docker is based around images (Fedora, CentOS, Ubuntu, etc.), and these images can be created and customized through the use of a Dockerfile. The Dockerfile contains statements to control the OS that is used, the software that is installed and post configuration. Here is a Dockerfile I like to use for building test environments:

$ cat Dockerfile

FROM centos:7

RUN yum -y update
RUN yum -y install openssh-server openldap-servers openldap-clients openldap
RUN sed -i 's/PermitRootLogin without-password/PermitRootLogin yes/' /etc/ssh/sshd_config
RUN echo 'root:XXXXXXXX' | chpasswd

RUN /usr/bin/ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -C '' -N ''
RUN /usr/bin/ssh-keygen -t rsa -f /etc/ssh/ssh_host_dsa_key -C '' -N ''

CMD ["/usr/sbin/sshd", "-D"]

To create an image from this Dockerfile you can use docker build:

$ docker build -t centos:7 .

The “-t” option assigns a tag to the image which can be referenced when a new container is instantiated. To view the new image you can run docker images:

$ docker images centos
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
centos              7                   4f798f95cfe1        8 minutes ago       414.8 MB    6                   f07f6ca555a5        3 weeks ago         194.6 MB    7                   980e0e4c79ec        3 weeks ago         196.7 MB    latest              980e0e4c79ec        3 weeks ago         196.7 MB

Not to have some fun! To create a new container we can use docker run:

$ docker run -d -P -h foo --name foo --publish 2222:22 centos:7

This instantiates a new CentOS container with the name (–name) foo, the hostname (-h) foo and uses the centos:7 image I created earlier. It also maps (–publish) port 22 in the container to port 2222 on my local PC. To access the container you can fire up SSH and connect to port 2222 as root (this is a test container so /dev/null the hate mail):

$ ssh root@localhost -p 2222
root@localhost's password: 
[root@foo ~]# 

Now I can install software, configure it, break it and debug issues all in an isolated environment. Once I’m satisfied with my testing I can stop the container and delete it:

$ docker stop foo

$ docker rm foo

I find that running an SSH daemon in my test containers is super valuable. For production I would take Jérôme’s advice and look into other methods for getting into your containers.