Archive
Posts from 2017
Forwarding the systemd journal to syslog
I am a big fan of the ELK stack and use it daily for various parts of my job. All of the logs from my physical systems, VMs and containers get funneled into elasticsearch, indexed and are available for me to slice and dice with Kibana. In addition to syslog data I also like to funnel the systemd journal into elasticsearch. This is easily accomplished by changing the journald.conf ForwardToSyslog configuration directive to yes: This small change will cause all journal entries to get routed to the local syslog daemon…
$ read more →Becoming a GDB power user is a truly valuable thing
I've been around Linux and UNIX for quite some time and one thing that has always piqued my interests is debugging broken software. Bryan Cantrill made some excellent points on why postmortem debugging is needed at DOCKERCON and the following video is a must watch: His points on restarting a broken container w/o root causing the source of the failure is SPOT ON! I also love his mad cow analogy. I've had the same mind set since I started managing infrastructure and I find the whole root cause process exciting and fun…
$ read more →Automating cron jobs with the ansible cron module
Over the past month I have been rewriting some cron scripts to enhance monitoring and observability. I've also been refactoring my ansible playbooks to handle deploying these scripts in a consistent fashion. Ansible ships with the cron module which makes this process a breeze. The cron module has all of the familiar cron attributes (hour, minute, second, program to run, etc.) and takes the following form: When I first played around with this module I noticed that each playbook run would result in a cron entry being added…
$ read more →Using xargs and lscpu to spawn one process per CPU core
One of my friends reached out to me earlier this week to ask if there was an easy way to run multiple Linux processes in parallel. There are several ways to approach this problem but most of them don't take into account hardware cores and threads. My preferred solution for CPU intensive operations is to use the xargs parallel option ("-P") along with the CPU cores listed in lscpu. This allows me to run one process per core which is ideal for CPU intensive applications…
$ read more →Using the python SMTP DebuggingServer to test SMTP communications
During the development of the dns-domain-expiration-checker script I needed a way to test SMTP mail delivery w/o relying on an actual mail exchanger. While reading through the smtplib and snmpd documentation I came across the SMTP debugging server. This nifty module allows you to run a local mail relay which will print the messages it receives to standard out. To enable it you can load the smtpd module and instruct it to run the DebuggingServer command on the IP and port passed as arguments: This will fire up a local mail server on localhost:8025 and each messaged received will be printed to STDOUT…
$ read more →