Analyzing I/O activity on Linux hosts with iotop
There have been a number of folks in the SysAdmin blogosphere posting about iotop, which is a Linux utility for viewing I/O activity in a a top-like display. To see just how useful iotop was, I installed it and ran it with the "-o" option (only report processes that are actively doing I/O): This is cool stuff, and I am stoked that I can now easily view I/O activity on my Linux and Solaris hosts. Niiiice!
$ read more →Chroot'ing users with openssh
I recently learned about the new ChrootDirectory in OpenSSH 5.2, and wanted to play around with it to see what it was capable of. To begin my quest, I started off by creating a couple of users that would be chroot'ed to their home directories when they logged into the server with sftp. Once the users were created, I added the following configuration stanza to my sshd_config file to chroot these users when they logged in with their sftp client: Once these directives where added, I started up the daemon in debug mode: Debug mode will cause the daemon to log verbosely to stdout, which is extremely useful for locating problems with new configuration directives. Now that the daemon was running, I tried to login with the user u1: The first attempt was a no go, but luckily verbose logging made debugging this issue a snap: After changing /home/u1 to be owned by root, I was able to login and poke around: This is super useful, though building chroot jails for normal SSH sessions will require a bit more work (i.e., you need to populate the chroot directory with all the config files and binaries needed to run a typical shell session)…
$ read more →Monitoring network bandwidth with bwm-ng
There are a bunch of utilities available to monitor bandwidth utilization on Linux hosts, and I've touched on a few in previous posts. I recently came across bwm-ng while perusing the Debian package repository, and decided to try it out. When bwm-ng is executed without any arguments, it provides a relatively simple curses interface with throughput statistics for each interface in the system: But the simplicity of the tool stops there, since there are a SLEW of options to control the output format, and whether or not sampled data is written to a file. This is a nifty utility, but I think I will stick with iftop.
$ read more →Using paste to create columns from input data
I periodically need to take input data from various utilities and convert it to columnar data. There are a million ways to do this, but I have come to rely on the paste utility to perform this task: In the output above, paste will take the input given to it and print the data in 3 columns (you can add more hyphens to get more columns of data). If anyone has some interesting little tidbits such as this, feel free to add them to the comments section. Thanks!
$ read more →Parallelizing shell tasks with project middleman and xargs
While poking around the Internet, I came across a link to project middleman. The project provides an easy way for administrators to parallelize tasks inside shell scripts, and is described rather nicely in the README file that comes with the source code: "The philosophy behind mdm is that users should benefit from their multi-core systems without making drastic changes to their shell scripts. With mdm, you annotate your scripts to specify which commands might benefit from parallelization, and then you run it under the supervision of the mdm system. At runtime, the mdm system dynamically discovers parallelization opportunities and run the annotated commands in parallel as appropriate." And when they mention annotating a shell script, it really is as simple as placing the "mdm-run" binary in front of tasks that can be parallelized (you can also define an I/O profile if tasks will interfere with each others I/O streams): This is pretty sweet, and I need to play around with this a bit more on my quad core desktop…
$ read more →