I mentioned a few posts back that I was playing with a bunch of Linux utilities. One of these utilities is dstat, which allows you to view a number of system statistics (disk utilization, network utilization, page activity, etc.) from the command line:
$ dstat
----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
0 0 100 0 0 0|1591B 134k| 0 0 | 0 0.6B| 1 4
1 7 90 0 1 1| 0 0 |2939k 69k| 0 0 |2784 3190
2 6 88 0 2 2| 0 0 |2882k 68k| 0 0 |2738 3079
0 9 90 0 0 1| 0 552k|2724k 64k| 0 0 |2591 2859
1 10 71 13 2 4| 0 22M|2782k 67k| 0 0 |2693 2773
0 4 92 0 2 2| 0 0 |2646k 63k| 0 0 |2487 2781
0 4 95 0 0 1| 0 0 |1712k 42k| 0 0 |1696 2069
1 7 91 0 0 1| 0 0 |2593k 63k| 0 0 |2489 2781
2 6 89 0 2 1| 0 2896k|2695k 64k| 0 0 |2555 2918
1 6 65 23 2 3| 0 21M|2760k 65k| 0 0 |2688 2942
The default output contains a number of useful statistics, but additional stats can be displayed if you pass a few options to dstat:
$ dstat -tcdgimnsy
-----time----- ----total-cpu-usage---- -dsk/total- ---paging-- -interrupts ------memory-usage----- -net/total- -swp/total- ---system--
date/time |usr sys idl wai hiq siq| read writ| in out | 17 18 | used buff cach free| recv send| used free| int csw
08-03 09:43:42| 0 0 100 0 0 0|1607B 134k| 0 0.6B| 1 1 | 19M 12M 104M 116M| 0 0 | 92k 729M| 1 5
08-03 09:43:43| 0 0 100 0 0 0| 0 0 | 0 0 | 0 5 | 19M 12M 104M 116M| 198B 580B| 92k 729M| 5 14
08-03 09:43:44| 0 1 99 0 0 0| 0 0 | 0 0 | 0 4 | 19M 12M 104M 116M| 132B 580B| 92k 729M| 4 8
08-03 09:43:45| 0 0 100 0 0 0| 0 0 | 0 0 | 0 4 | 19M 12M 104M 116M| 132B 580B| 92k 729M| 4 8
08-03 09:43:46| 0 0 100 0 0 0| 0 0 | 0 0 | 0 4 | 19M 12M 104M 116M| 132B 580B| 92k 729M| 4 14
08-03 09:43:47| 0 1 99 0 0 0| 0 0 | 0 0 | 0 4 | 19M 12M 104M 116M| 132B 580B| 92k 729M| 4 7
08-03 09:43:48| 0 0 100 0 0 0| 0 240k| 0 0 | 2 4 | 19M 12M 104M 116M| 132B 580B| 92k 729M| 6 13
08-03 09:43:49| 0 0 100 0 0 0| 0 0 | 0 0 | 0 4 | 19M 12M 104M 116M| 132B 580B| 92k 729M| 4 8
08-03 09:43:50| 0 0 100 0 0 0| 0 0 | 0 0 | 0 4 | 19M 12M 104M 116M| 132B 580B| 92k 729M| 4 8
The output above will overflow a standard 80 character wide display, so you may need to adjust the size of your terminal window to see everything. Dstat is an amazing utility, and yet another tool I plan to add to my performance analysis toolkit.
While playing around with my Debian 5.0 host last week, I came across the pidstat utility. Pidstat allows you to display a number of statistics for processes running on a system (and threads inside that process), which can be incredibly useful for troubleshooting performance problems. To use pidstat to get a breakdown of how a given process is using the processors on a system, pidstat can be run with the “-p” option and the process ID to interrogate:
$ pidstat -p 3297 2
Linux 2.6.26-1-686 (disarm) 03/07/2009 i686
08:07:18 AM PID %user %system %CPU CPU Command 08:07:20 AM 3297 11.06 74.37 85.43 0 dd 08:07:22 AM 3297 13.00 70.00 83.00 0 dd 08:07:24 AM 3297 10.95 72.64 83.58 0 dd
If you are looking to see how much I/O a specific process is responsible for, you can run pidstat with the “-d” option:
$ pidstat -d -p 3288 2
Linux 2.6.26-1-686 (disarm) 03/07/2009 i686
08:06:35 AM PID kB_rd/s kB_wr/s kB_ccwr/s Command 08:06:37 AM 3288 3.98 113319.40 0.00 dd 08:06:39 AM 3288 4.00 112514.00 0.00 dd 08:06:41 AM 3288 4.00 81454.00 0.00 dd
And finally, to view paging activity per process, you can run pidstat with the “-r” option (and optionally the “-t” flag if you want to see thread activity):
$ pidstat -r -t -p 3338 2
Linux 2.6.26-1-686 (disarm) 03/07/2009 i686
08:10:20 AM PID TID minflt/s majflt/s VSZ RSS %MEM Command 08:10:22 AM 3338 - 0.00 0.00 3180 596 0.23 dd 08:10:22 AM - 3338 0.00 0.00 3180 596 0.23 |__dd
08:10:22 AM PID TID minflt/s majflt/s VSZ RSS %MEM Command 08:10:24 AM 3338 - 0.00 0.00 3180 596 0.23 dd 08:10:24 AM - 3338 0.00 0.00 3180 596 0.23 |__dd
08:10:24 AM PID TID minflt/s majflt/s VSZ RSS %MEM Command 08:10:26 AM 3338 - 0.00 0.00 3180 596 0.23 dd 08:10:26 AM - 3338 0.00 0.00 3180 596 0.23 |__dd
Pidstat is pretty darn cool, and I will definitely be using this in the future! Nice!
I recently needed to configure a CentOS Linux host to use 802.1Q tagged queuing. This turned out to be super easy, and allowed me place the physical server on multiple layer-2 networks. To configure an 802.1Q interface, I first checked the ifcfg-eth0 script for the interface that was planning to participate in 802.1Q:
$ cd /etc/sysconfig/network-scripts
$ cat ifcfg-eth0
DEVICE=eth0
BOOTPROTO=dhcp
HWADDR=00:1B:24:5C:2C:DD
ONBOOT=yes
TYPE=Ethernet
Once I had the interface information, I created a ifcfg-vlan301 interface file with the VLAN number as well as the VLAN options:
$ cat ifcfg-vlan301
VLAN=yes
VLAN_NAME_TYPE=VLAN_PLUS_VID_NO_PAD
DEVICE=vlan301
PHYSDEV=eth0
BOOTPROTO=static
ONBOOT=yes
TYPE=Ethernet
After the files were created, I used service to restart networking and everything came up smoothly. There are much better references on the Internet for Linux 802.1Q configuration, but I’m posting this here so I have a handy reference (and one I know works!).
While reviewing one of my Linux hosts, I noticed that a bunch of write activity was occurring to one specific file system. I was curious to see what this write activity was, so I started reading up on the Linux inotify framework. Inotify allows you to monitor file system events, and the super useful iwatch utility provides a command line interface to the inotify framework. When iwatch is executed and passed the name of a directory on the command line, it will print all of the events that are applicable to that directory:
$ iwatch -v /tmp
This command line will cause a line to be printed to STDOUT each time an event is triggered due to operations taking place in /tmp. If you would prefer to get an e-mail when events occur, you can add the “-m” option to the command line:
$ iwatch -v -m matty@prefetch.net /tmp
This utility rocks, and I hope Solaris will provide a similar utility to take advantage of their notification framework.
Picked up this neat little trick on command-line-fu.
Read any UNIX man page in OS X as a pdf in preview
$ man -t <command> | open -f -a preview