While debugging Apache this week, I wanted to get verbose output from libtool. After digging around on the web, I found that setting LTFLAGS to “–debug” will cause libtool to display numerous useful pieces of information:
$ export LTFLAGS="--debug"
$ apxs -c mod_dtrace.c
Link command: /tmp/apache/build/libtool --debug --mode=link gcc -o
mod_dtrace.la -rpath /tmp/apache/modules -module -avoid-version
mod_dtrace.lo
/tmp/apache/build/libtool --debug --mode=compile gcc -prefer-pic
-DAP_HAVE_DESIGNATED_INITIALIZER -DSOLARIS2=10
-D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -g -O2 -I/tmp/apache/include
-I/tmp/apache/include -I/tmp/apache/include -c -o mod_dtrace.lo
mod_dtrace.c && touch mod_dtrace.slo
libtool: enabling shell trace mode
+ preserve_args=' --debug'
+ test 16 -gt 0
+ arg=--mode=compile
+ shift
+ case $arg in
++ echo X--mode=compile
++ /bin/sed -e '1s/^X//' -e 's/[-_a-zA-Z0-9]*=//'
+ optarg=compile
[ ... ]
If you are trying to figure out how things are built with libtool, this will definitely helpful!
I am a huge fan of awk, and find myself constantly using it to parse simple data steams. awk contains numerous string-related functions, which are an invaluable resource for awk script developers. To illustrate some of the cool awk functions, I created a single-line text file with the string “String of string”:
$ cat text
string of string
To get the length of each line in the file text, the length() function can be used:
$ awk '{ i = length(0); print i }' text
16
To see if the string “ing” is present in the file text, the index() function can be used:
$ awk '{ i = index(0,"ing"); print i}' text
4
index() will return the location of the first occurence of “ing,” which can then be used to facilitate further string processing. To retrieve a range of characters in a string, a beginning and ending offset can be passed to the awk substr() function:
$ awk '{ i = substr(0,5,10); print i }' text
ng of stri
And finally, to tokenize (split a line into word-length pieces) a string, the split() function can be used:
$ awk 'BEGIN { i = 1 } { n = split(0,a," "); while (i <= n) {print a[i]; i++;} }' text
string
of
string
I dig awk!
I came across Scott Kaiser’s defrag.plscript a while back, and have found it useful for determining if the VxFS free extent map is fragmented. The script takes a file system as an option, and prints a one-line string to indicate if the file system should be defragmented:
$ defrag.pl /u01
/u01 is badly fragmented. Defragmentation is recommended.
If the script determines that the free extent map (the script doesn’t report on directory entry fragmentation) is fragmented, you can perform an online defragmentation by invoking fsadm with the “-e” (reorganize extents) and “-d” (reorganize directory entries) options:
$ /usr/lib/fs/vxfs/fsadm -v -e -d /u01
UX:vxfs fsadm: INFO: V-3-20287: using device /dev/vx/rdsk/oradg/oravol01
UX:vxfs fsadm: INFO: V-3-20223: directory reorganization complete
UX:vxfs fsadm: INFO: V-3-20261: extent reorg pass 1
AU: aun = 16, tfree = 32064, sfree = 384
aun = 16, seg = 0, nfrag = 95, fblks = 1690, devid = 0 start = 524288, len = 2048
req[0] fset 999 ino 32 blocks 1 off 0x0 len 1
req[0] fset 999 ino 33 blocks 1 off 0x0 len 1
UX:vxfs fsadm: ERROR: V-3-24364: reorg failed for fset 999 ino 33
req[0] fset 999 ino 34 blocks 1 off 0x0 len 1
req[0] fset 999 ino 35 blocks 1 off 0x0 len 1
req[0] fset 999 ino 36 blocks 1 off 0x0 len 1
req[0] fset 999 ino 37 blocks 1 off 0x0 len 1
req[0] fset 999 ino 38 blocks 1 off 0x0 len 1
[ ... ]
aun = 16, seg = 0, nfrag = 86, fblks = 5397, devid = 0 start = 524288, len = 6144
UX:vxfs fsadm: INFO: V-3-20262: extent reorg complete
I prefer to avoid fragmentation, and try to preallocate all files to avoid this issue (for dynamic applications this isn’t always possible). For further information please refer to the fsadm_vxfs(1m) manual page.
I use the ORCA utility to graph various system and application metrics, and have recently run into a few problems. The application periodically crashes for no apparaent reason, and I haven’t had time to debug the issue (once I get a core file I will figure this out). Since I rely on the graphs to trend server and application capacity, I want to ensure that the application gets restarted each time a failure occurs. Since ORCA is running on a Solaris 10 server, I decided to convert the existing start/stop scripts to Solaris 10 SMF manifests. To begin the conversion process, I first created a shell script that would be able to start up ORCA and clear any lockfiles that are present:
$ cat /usr/local/bin/orca.start
#!/bin/sh
if [ -d /var/orca/configs/orcallator.cfg.lock ]
then
logger -p daemon.notice "Removing orca lockfile"
rm -rf /var/orca/configs/orcallator.cfg.lock
fi
logger -p daemon.notice "Starting orca in daemon mode"
/usr/local/bin/orca -logfile /var/logs/orcallator.log
-daemon /var/orca/configs/orcallator.cfg
Once I verified that the script worked correctly, I created an SMF manifest with a stop and start method and no dependencies:
$ cat orca.xml
<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">
<tservice_bundle type='manifest' name='orca'>
<service
name="application/orca"
type="service"
version="1">
<create_default_instance enabled="true"/>
<exec_method
type='method'
name='start'
exec='/usr/local/bin/orca.start'
timeout_seconds='0'>
</exec_method>
<exec_method
type='method'
name='stop'
exec=':kill -15'
timeout_seconds='3'>
</exec_method>
</service>
</service_bundle>
After the manifest was created, I used the svccfg ‘validate’ option to verify the structure of the XML document:
$ svccfg validate orca.xml
$ echo ?
0
If svccfg encounters an error, it will display an error on the console, and return a non-zero return code. If the XML document validates, the svccfg ‘import’ option can be used to import the manifest into the SMF repository:
$ svccfg import orca.xml
Once the manifest has been imported into the SMF repository, the svccfg ‘listprop’ option can be used to display the service’s properties:
$ svccfg -s application/orca listprop
start method
start/exec astring /opt/data/orca/scripts/orca.start
start/timeout_seconds count 0
start/type astring method
stop method
stop/exec astring ":kill -15"
stop/timeout_seconds count 3
stop/type astring method
All of this took me about 15 minutes, and now when ORCA crashes SMF restarts the process, which generates the following messages in the system logfile:
Nov 9 11:56:03 winnie root: [ID 702911 daemon.notice] Removing orca lockfile
Nov 9 11:56:03 winnie root: [ID 702911 daemon.notice] Starting orca in daemon mode
The SMF team did an awesome job with this!!
While messing around with DTrace, I devised a way to extract the Apache version with a simple DTrace script (this of course can be acquired with telnet and netcat, but it’s fun doing things with DTrace):
$ cat version.d
#pragma D option quiet
pid$target::ap_get_server_version:return
{
printf("Apache web Server Version: %sn",copyinstr(arg1));
}
$ for i inpgrep httpd; do dtrace -p i -s version.d & done
[1] 24581
[2] 24582
[3] 24583
Apache web Server Version: Apache/2.0.54 (Unix)
Apache web Server Version: Apache/2.0.54 (Unix)
Apache web Server Version: Apache/2.0.54 (Unix)
Apache web Server Version: Apache/2.0.54 (Unix)
$ pkill dtrace
This example utilizes the pid provider to catch returns from ap_get_server_version, and prints the function return value as a string (the return value is a pointer to a global variable named server_version, which contains the token definition from the ServerTokens command). I looooooooooooooves me some DTrace!