Archive for 'Linux Debugging'

Riemann and the case of the unhappy system call

This past weekend I spent a good deal of time playing with riemann. Riemann is a powerful stream processor and I’m hoping to use it to correlated and analyze metrics from disparate data sources. After downloading and installing it I received the following error: $ ./riemann ERROR [2016-10-10 12:21:36,614] main – riemann.bin – Couldn’t start […]

Ganglia never met a quote it didn’t like. Wait it did …

This week I encountered a weird issue while developing a new Ganglia plug-in. After moving my ganglia processes to a docker container I noticed that the grid overview images weren’t displaying. This was a new ganglia installation so I figured I typo’ed something in the gmetad.conf configuration file. I reviewed the file in my git […]

Summarizing system call activity on Linux systems

Linux has a guadzillion debugging utilities available. One of my favorite tools for debugging problems is strace, which allows you to observe the system calls a process is making in realtime. Strace also has a “-c” option to summarize system call activity: $ strace -c -p 28009 Process 28009 attached Process 28009 detached % time […]

Using the automated bug-reporting tool (abrt) to generate core dumps when a Linux process fails

Software fails, and it often occurs at the wrong time. When failures occur I want to understand why, and will usually start putting together the events that lead up to the issue. Some application issues can be root caused by reviewing logs, but catastrophic crashes will often require the admin to sit down with gdb […]

Dealing with xauth “error in locking authority file” errors

I recently logged into one of my servers and received the following error: $ ssh foo matty@foo’s password: Last login: Tue Nov 1 13:42:52 2011 from /usr/bin/xauth: error in locking authority file /home/matty/.Xauthority I haven’t seen this one before, but based on previous “locking issues” I’ve encountered in the past I ran strace against […]

One way to avoid tcpdump “packets dropped by kernel” messages

I have been knee deep this week debugging a rather complex DNS issue. I’ll do a full write up on that next week. While I was debugging the issue I needed to fire up tcpdump to watch the DNS queries from one of my authoritative servers to various servers on the Internet. What I noticed […]

« Older Entries