The DTraceToolkit comes with two super useful scripts to observe the “randonmess” or “sequentialness” of an application. The first script is iopattern, which provides a system-wide view of random and sequential I/O, the total amount of I/O generated, and an I/O size distribution:
$ iopattern 5
%RAN %SEQ COUNT MIN MAX AVG KR KW
100 0 9 512 4096 1706 0 15
100 0 19 512 1024 592 0 11
100 0 4 512 512 512 0 2
100 0 8 512 4096 1856 0 14
The second script is seeksize.d. Seeksize.d provides a histogram of the number of sectors traversed between I/O operations for each process on the system:
$ seeksize.d
1615 /usr/lib/ssh/sshd
value ------------- Distribution ------------- count
2048 | 0
4096 |@@@@@@@ 2
8192 |@@@@@@@@@@@ 3
16384 |@@@@@@@@@@@ 3
32768 |@@@@@@@@@@@ 3
65536 | 0
[ ..... ]
While discussing these scripts last night, I didn’t provide many details on how they actually work. Each script uses the io provider to detect when an I/O occurs, and the value of “b_blkno” to determine which block is being read into memory or written to disk. Since DTrace comes with a nifty type called an aggregation, the previous block read can be compared with the current block read (both stored as thread local variables), and the result stored in the aggregation. This is pretty interesting stuff, and I apologize for not providing further details during my talk.
Tonight I gave a talk at the local opensolaris users group titled “DTrace for SysAdmins: An introduction to the DTraceToolkit.” I would like to thank everyone for coming out, and for putting up with my broken voice (I am currently getting over a cold). I put the presentation slides up on prefetch.net, and I hope everyone will snag the latest version of the toolkit and test it out!
I got sick last Friday, and have been stuck at home all weekend recovering. While I was resting in bed yesterday, I set iTunes to play random music from my CD collection. iTunes was playing musical artists who I haven’t listened to in several years (I can’t recall the last time I actually listened to Magic Carpet Ride by Steppenwolf), and it got me thinking about who the best guitar players of all time were. There are TONS of awesome guitarists, and after pondering this I came up with my ten favorite guitarists:
1. Jimmy Hendrix
2. Jimmy Page from Led Zepplin
3. Eric Clapton
4. Slash from Guns N’ Roses
5. John Frusciante from The Red Hot Chili Peppers
6. Mick Mars from Motley Crue
7. Dave Matthews from the Dave Matthews band
8. James Hetfiel, Kirk Hammett and Dave Mustaine (when he was with the band) from Metallica
9. Bill Corgan from The Smashing Pumpkins
10. Jerry Cantrell from Alice In Chains
My list is definitely oriented towards the music I listen to regularly, and also takes into account that the artists typically wrote music. If I am missing someone important, feeel free to post a comment. I digs me some music!
I recently spent some time reading through the grub source code and the real mode assembly that resides in the MBR (master boot record). As I was reading though an explanation of the MBR real mode asembly code, I was amazed at how much cool stuff was packed into the 512-bytes that makes up the MBR (and of that 512-bytes, only 448-bytes are dedicated to storing machine code and error strings). The link above does a great job describing the real mode assembly, and in a nutshell the MBR code in that link does the following four things:
The MBR real mode assembly is pretty neat, and the following 448-bytes of real mode assembly from the OpenBSD MBR are amazingly cool (I am a geek at heart, so this kinda stuff interests me):
$ objdump -D -b binary -mi386 -Maddr16,data16 /usr/mdec/mbr
/usr/mdec/mbr: file format binary
Disassembly of section .data:
00000000 <.data>:
0: ea 05 00 c0 07 ljmp $0x7c0,$0x5
5: 8c c8 mov %cs,%ax
7: 8e d0 mov %ax,%ss
9: bc fc ff mov $0xfffc,%sp
c: 8e d8 mov %ax,%ds
e: b8 a0 07 mov $0x7a0,%ax
11: 8e c0 mov %ax,%es
13: 31 f6 xor %si,%si
15: 31 ff xor %di,%di
17: b9 00 02 mov $0x200,%cx
1a: fc cld
1b: f3 a4 repz movsb %ds:(%si),%es:(%di)
1d: ea 22 00 a0 07 ljmp $0x7a0,$0x22
22: 1e push %ds
23: 07 pop %es
24: 0e push %cs
25: 1f pop %ds
26: b4 02 mov $0x2,%ah
28: cd 16 int $0x16
2a: a8 03 test $0x3,%al
2c: 74 0a je 0x38
2e: b0 07 mov $0x7,%al
30: e8 cb 00 call 0xfe
33: 80 0e b4 01 01 orb $0x1,436
38: f6 c2 80 test $0x80,%dl
3b: 75 08 jne 0x45
3d: be 36 01 mov $0x136,%si
40: e8 af 00 call 0xf2
43: b2 80 mov $0x80,%dl
45: be be 01 mov $0x1be,%si
48: b9 04 00 mov $0x4,%cx
4b: 8a 04 mov (%si),%al
4d: 3c 80 cmp $0x80,%al
4f: 74 0f je 0x60
51: 83 c6 10 add $0x10,%si
54: e2 f5 loop 0x4b
56: be 6a 01 mov $0x16a,%si
59: e8 96 00 call 0xf2
5c: fb sti
5d: f4 hlt
5e: eb fc jmp 0x5c
60: 88 d0 mov %dl,%al
62: 24 0f and $0xf,%al
64: 04 30 add $0x30,%al
66: a2 27 01 mov %al,0x127
69: b0 34 mov $0x34,%al
6b: 28 c8 sub %cl,%al
6d: a2 34 01 mov %al,0x134
70: 56 push %si
71: be 1a 01 mov $0x11a,%si
74: f6 06 b4 01 01 testb $0x1,436
79: 75 01 jne 0x7c
7b: 46 inc %si
7c: e8 73 00 call 0xf2
7f: 5e pop %si
80: 26 c7 06 fe 01 00 00 movw $0x0,%es:510
87: f6 06 b4 01 01 testb $0x1,436
8c: 75 31 jne 0xbf
8e: 88 14 mov %dl,(%si)
90: bb aa 55 mov $0x55aa,%bx
93: b4 41 mov $0x41,%ah
95: cd 13 int $0x13
97: 8a 14 mov (%si),%dl
99: 72 24 jb 0xbf
9b: 81 fb 55 aa cmp $0xaa55,%bx
9f: 75 1e jne 0xbf
a1: f6 c1 01 test $0x1,%cl
a4: 74 19 je 0xbf
a6: b0 2e mov $0x2e,%al
a8: e8 53 00 call 0xfe
ab: 66 8b 4c 08 mov 8(%si),%ecx
af: 66 89 0e 12 01 mov %ecx,274
b4: 56 push %si
b5: b4 42 mov $0x42,%ah
b7: be 0a 01 mov $0x10a,%si
ba: cd 13 int $0x13
bc: 5e pop %si
bd: 73 19 jae 0xd8
bf: b0 3b mov $0x3b,%al
c1: e8 3a 00 call 0xfe
c4: 8a 74 01 mov 1(%si),%dh
c7: 8b 4c 02 mov 2(%si),%cx
ca: b8 01 02 mov $0x201,%ax
cd: 31 db xor %bx,%bx
cf: cd 13 int $0x13
d1: 73 05 jae 0xd8
d3: be 52 01 mov $0x152,%si
d6: eb 81 jmp 0x59
d8: be 7d 01 mov $0x17d,%si
db: e8 14 00 call 0xf2
de: 26 81 3e fe 01 55 aa cmpw $0xaa55,%es:510
e5: 75 05 jne 0xec
e7: ea 00 7c 00 00 ljmp $0x0,$0x7c00
ec: be 61 01 mov $0x161,%si
ef: e9 67 ff jmp 0x59
f2: 50 push %ax
f3: fc cld
f4: ac lods %ds:(%si),%al
f5: 84 c0 test %al,%al
f7: 74 0f je 0x108
f9: e8 02 00 call 0xfe
fc: eb f6 jmp 0xf4
fe: 50 push %ax
ff: 53 push %bx
100: b4 0e mov $0xe,%ah
102: bb 01 00 mov $0x1,%bx
105: cd 10 int $0x10
107: 5b pop %bx
108: 58 pop %ax
109: c3 ret
10a: 10 00 adc %al,(%bx,%si)
10c: 01 00 add %ax,(%bx,%si)
10e: 00 00 add %al,(%bx,%si)
110: c0 07 00 rolb $0x0,(%bx)
113: 00 00 add %al,(%bx,%si)
115: 00 00 add %al,(%bx,%si)
117: 00 00 add %al,(%bx,%si)
119: 00 21 add %ah,(%bx,%di)
11b: 55 push %bp
11c: 73 69 jae 0x187
11e: 6e outsb %ds:(%si),(%dx)
11f: 67 20 64 72 69 addr32 and %ah,0x69(%edx,%esi,2)
124: 76 65 jbe 0x18b
126: 20 58 2c and %bl,44(%bx,%si)
129: 20 70 61 and %dh,97(%bx,%si)
12c: 72 74 jb 0x1a2
12e: 69 74 69 6f 6e imul $0x6e6f,105(%si),%si
133: 20 59 00 and %bl,0(%bx,%di)
136: 4d dec %bp
137: 42 inc %dx
138: 52 push %dx
139: 20 6f 6e and %ch,110(%bx)
13c: 20 66 6c and %ah,108(%bp)
13f: 6f outsw %ds:(%si),(%dx)
140: 70 70 jo 0x1b2
142: 79 20 jns 0x164
144: 6f outsw %ds:(%si),(%dx)
145: 72 20 jb 0x167
147: 6f outsw %ds:(%si),(%dx)
148: 6c insb (%dx),%es:(%di)
149: 64 20 42 49 and %al,%fs:73(%bp,%si)
14d: 4f dec %di
14e: 53 push %bx
14f: 0d 0a 00 or $0xa,%ax
152: 0d 0a 52 or $0x520a,%ax
155: 65 gs
156: 61 popa
157: 64 20 65 72 and %ah,%fs:114(%di)
15b: 72 6f jb 0x1cc
15d: 72 0d jb 0x16c
15f: 0a 00 or (%bx,%si),%al
161: 4e dec %si
162: 6f outsw %ds:(%si),(%dx)
163: 20 4f 2f and %cl,47(%bx)
166: 53 push %bx
167: 0d 0a 00 or $0xa,%ax
16a: 4e dec %si
16b: 6f outsw %ds:(%si),(%dx)
16c: 20 61 63 and %ah,99(%bx,%di)
16f: 74 69 je 0x1da
171: 76 65 jbe 0x1d8
173: 20 70 61 and %dh,97(%bx,%si)
176: 72 74 jb 0x1ec
178: 69 74 69 6f 6e imul $0x6e6f,105(%si),%si
17d: 0d 0a 00 or $0xa,%ax
180: 90 nop
...
1b5: 00 4f 78 add %cl,120(%bx)
...
1ec: 00 00 add %al,(%bx,%si)
1ee: 80 00 01 addb $0x1,(%bx,%si)
1f1: 00 a6 ff ff add %ah,-1(%bp)
1f5: ff 00 incw (%bx,%si)
1f7: 00 00 add %al,(%bx,%si)
1f9: 00 ff add %bh,%bh
1fb: ff (bad)
1fc: ff (bad)
1fd: 7f 55 jg 0x254
1ff: aa stos %al,%es:(%di)
I have a few Redhat Linux servers that log application data to one or more logfiles in /var/log. For some reason Redhat Linux doesn’t compress the logfiles in /var/log by default, which can be a problem if your logging a lot of information, or if you need to keep logfiles around for historical reasons. To change this behavior and compress the logfiles that are stored in /var/log, you can uncomment the “compress” line in /etc/logadm.conf. This is a simple one line change, and the result should look similar to this:
# uncomment this if you want your log files compressed
compress
Once that change is made, the logfiles in /var/log will be compressed each time they are rotated.