While catching up on the DTrace discussion list tonight, I came across a cool DTrace debugging recommendation from Adam Leventhal. Adam mentioned using the DTRACE_DEBUG flag, which will cause libdtrace to spew tons of debugging data each time dtrace is invoked:
$ export DTRACE_DEBUG=1
$ dtrace -n 'syscall:::entry { @calls[execname] = count(); }'
libdtrace DEBUG: opened provider fasttrap
libdtrace DEBUG: opened provider fbt
libdtrace DEBUG: opened provider lockstat
libdtrace DEBUG: opened provider profile
libdtrace DEBUG: opened provider sdt
libdtrace DEBUG: opened provider systrace
libdtrace DEBUG: opened 64-bit module unix (/platform/sun4u/kernel/sparcv9/unix) [0]
libdtrace DEBUG: opened 64-bit module krtld (misc/sparcv9/krtld) [1]
libdtrace DEBUG: opened 64-bit module genunix (/platform/sun4u/kernel/sparcv9/genunix) [2]
libdtrace DEBUG: opened 64-bit module platmod (/platform/SUNW,Ultra-5_10/kernel/misc/sparcv9/platmod) [3]
libdtrace DEBUG: opened 64-bit module SUNW,UltraSPARC-IIi (/platform/sun4u/kernel/cpu/sparcv9/SUNW,UltraSPARC-IIi) [4]
libdtrace DEBUG: opened 64-bit module specfs (/kernel/fs/sparcv9/specfs) [6]
libdtrace DEBUG: opened 64-bit module dtrace (/kernel/drv/sparcv9/dtrace) [7]
[ ...... ]
dtrace: description 'syscall:::entry ' matched 225 probes
^C
utmpd 2
fmd 4
inetd 4
svc.configd 5
svc.startd 5
sshd 16
nscd 77
ipmon 94
mysqld 149
dtrace 1784
libdtrace DEBUG: dt_buf_destroy(section headers): size=1024 resizes=1
libdtrace DEBUG: dt_buf_destroy(string table): size=512 resizes=0
libdtrace DEBUG: dt_buf_destroy(loadable data): size=512 resizes=0
libdtrace DEBUG: dt_buf_destroy(unloadable data): size=512 resizes=0
libdtrace DEBUG: dt_buf_destroy(probe data): size=512 resizes=0
libdtrace DEBUG: dt_buf_destroy(probe args): size=512 resizes=0
libdtrace DEBUG: dt_buf_destroy(probe offs): size=512 resizes=0
libdtrace DEBUG: dt_buf_destroy(probe rels): size=512 resizes=0
This is super cool, and useful for tracking down problems, or learning more about what DTrace does behind the scenes.