Generating statistics from the Sun directory server access logs

I have been managing Sun’s directory server for close to four years, and it is one of the few products I use that has all the bells and whistles out of the box. One extremely useful feature is the ability to generate statistics from the server access logs. This capability is built into the logconv.pl perl script, which is part of the Sun directory server resource kit (as a side note, logconv.pl is what motivated me to write ldap-stats.pl to analyze OpenLDAP log files).

To use the logconv.pl statistics program to analyze your server access logs, you will first need to grab the resource kit from the Sun download page. Once you download and extract the resource kit, you can cd into the $RESOURCE_KIT_HOME/perl directory, and run logconv.pl with the optional “-V” (verbose) option, and the names of the log(s) to analyze:

$ logconv.pl -V access

This will produce a report similar to the following:

SunOne Access Log Analyzer 4.71

Initializing Variables...
Processing 1 Access Log(s)...

  /home/matty/access (Total Lines: 135987) 
       1000 Lines Processed
       2000 Lines Processed
          < ..... >
     134000 Lines Processed
     135000 Lines Processed
*    135987 Lines Processed                     Total Lines Processed:       135987

* Total Lines Analyzed:  135987


----------- Access Log Output ------------

Start of Log:  15/May/2007:16:13:30
End of Log:    16/May/2007:16:13:55

Restarts:                     0

Opened Connections:           16967
Closed Connections:           0
Total Operations:             34183
Total Results:                34183
Overall Performance:          100.0%
Most Pending Operations:      3

Searches:                     16552
Modifications:                13
Adds:                         4
Deletes:                      1
Mod RDNs:                     0
Compares:                     0

5.x Stats 
Persistent Searches:          0
Internal Operations:          0
Entry Operations:             0
Extended Operations:          658
Abandoned Requests:           0
Smart Referrals Received:     0

VLV Operations:               0
VLV Unindexed Searches:       0
SORT Operations:              0
SSL Connections:              0

Entire Search Base Queries:   1
Unindexed Searches:           0

FDs Taken:                    16967
FDs Returned:                 0
Highest FD Taken:             472

Broken Pipes:                 0
Connections Reset By Peer:    0
Resource Unavailable:         0

Binds:                        16955
Unbinds:                      16738

 LDAP v2 Binds:               2
 LDAP v3 Binds:               16953
 Expired Password Logins:     0
 SSL Client Binds:            0
 Failed SSL Client Binds:     0
 SASL Binds:                  0

 Directory Manager Binds:     0
 Anonymous Binds:             2
 Other Binds:                 16953



----- Errors -----

err=0                 33985    Successful Operations   
err=49                  148    Invalid Credentials (Bad Password)
err=32                   50    No Such Object          


----- Top 20 Failed Logins ------

9           uid=foo,ou=people,dc=prefetch,dc=net
9           uid=bar,ou=people,dc=prefetch,dc=net

 < ..... >


----- Total Connection Codes -----

U1                    16738    Cleanly Closed Connections              
B1                      220    Bad Ber Tag Encountered                 

 < ..... >

----- Top 20 Clients -----

Number of Clients:  9

7204   1.2.3.4  
                   7124 -  U1   Cleanly Closed Connections
                     25 -  B1   Bad Ber Tag Encountered

5340   1.2.3.5      
                   5322 -  U1   Cleanly Closed Connections
                     18 -  B1   Bad Ber Tag Encountered

 < ..... >


----- Top 20 Bind DN's -----

Number of Unique Bind DN's: 398

3857            uid=foo,ou=people,dc=prefetch,dc=net
2761            uid=bar,ou=people,dc=prefetch,dc=net

  < ..... >


----- Top 20 Search Bases -----

Number of Unique Search Bases: 25

7716            ou=people,dc=prefetch,dc=net
5302            ou=groups,dc=prefetch,dc=net

 < ..... >


----- Top 20 Search Filters -----

Number of Unique Search Filters: 619

5324            (&(objectclass=organizationalperson)(uid=foo))
2761            (&(objectclass=organizationalperson)(entrydn=uid=bar,ou=people,dc=prefetch,dc=net))

 < ..... >

----- Top 20 Most Frequent etimes -----

34049           etime=0     
134             etime=1     


----- Top 20 Longest etimes -----

etime=1         134       
etime=0         34049     


----- Top 20 Largest nentries -----

nentries=5                      1
nentries=1                  16537
nentries=0                     14


----- Top 20 Most returned nentries -----

16537           nentries=1    
14              nentries=0    
1               nentries=5    



----- 5.x Extended Operations -----

302         1.3.6.1.4.1.42.2.27.9.6.1     Other                                                       
302         1.3.6.1.4.1.42.2.27.9.6.6     Other                                                       
18          1.3.6.1.4.1.42.2.27.9.6.5     Other                                                       
18          1.3.6.1.4.1.42.2.27.9.6.8     Other                                                       
18          1.3.6.1.4.1.42.2.27.9.6.4     Other                                                       


----- Top 20 Most Requested Attributes -----

8127        nsRoleDN           
8123        displayName        

 < ..... >

----- Recommendations -----

None.

As you can see from the output above, there are numerous useful statistics, including the number of unindexed searches, a list of errors, operations by client, etc. These statistics are an invaluable tool, and extremely useful for proactively finding problems in your infrastructure.

Generating passwords for LDAP

When I generate LDIF entries for new LDAP users, I need to populate the userPassword field with a crypt password or an SHA1 or MD5 hash. Sun’s directory server comes with the getpwenc utility to assist with this:

$ getpwenc CRYPT password
{crypt}g72FJndhHyuiob

In addition to generating crypt passwords, it can also create MD5 and SHA1 hashes.

Debugging directory server issues with DTrace

Today while installing Sun’s directory server, I kept getting the error “No components have been selected for installation in the selected directory” after selecting numerous components for installation:

Directory Server components showing a checked box will be installed.

 [X]  1   Sun Java(TM) System Directory Suite    0 bytes
 [X]  2     Sun Java(TM) System Directory Server    0 bytes
 [ ]  3     Sun Java(TM) System Directory Console Support    0 bytes
 [ ]  4   Sun Java(TM) System Administration Services    0 bytes
 [ ]  5     Sun Java(TM) System Administration Server    0 bytes
 [ ]  6     Sun Java(TM) System Administration Console    0 bytes
 [ ]  7   Sun Java(TM) System Server Console    0 bytes
 [ ]  8     Sun Java(TM) System Server Console Core    0 bytes
 [ ]  9     Java Runtime Environment    0 bytes
 [ ]  10  Sun Java(TM) System Server Basic Libraries    0 bytes

   To check a particular component, enter its number, or 0 when you are
   finished [0] {"<" goes back, "!" exits}: 0

No components have been selected for installation in the selected directory.

Since I had previously installed the directory server on the machine, I started to wonder if the directory server had placed a package repository somewhere on the file system. To see if this was the case, I fired up opensnoop from the DTraceToolkit:

$ opensnoop

  UID    PID COMM          FD PATH
                 < ..... >
    0    657 java          16 /var/sadm/install/productregistry.access
    0    657 java          16 /var/sadm/install/productregistry.access.lock
    0    657 java          16 /var/sadm/install/productregistry.access
    0    657 java          18 /var/sadm/install/productregistry.access.tmp
    0    657 java          16 /var/sadm/install/productregistry.access
    0    657 java          16 /var/sadm/install/productregistry.access.lock

Sure enough, there is a product registry file in /var/sadm/install. Once I removed it with rm:

$ rm -f /var/sadm/install/productregistry

The installation went off without a hitch. DTrace truly does bring a lot to the table, and I feel like a fish out of water when debugging difficult issues on Solaris 9, AIX and Linux hosts.