Blog O’ Matty

The netstat utility provides a command line interface to retrieve system routing tables, connection states, and network statistics. Two of the available statisticss, “tcpPassiveOpens” and “tcpActiveOpens,” provide data on the number of new connections to a server (tcpPassiveOpens), and the number of connections initiated by the server (tcpActiveOpens). Both statistics counters can be retrieved with netstat’s “-s” option, and a simple while loop allows a SysAdmin to get a high level view of TCP connections to and from a server:

$ while :
> do
>        netstat -s | egrep '(tcpPassiveOpens|tcpActiveOpens)'
>       sleep 10
> done
        tcpActiveOpens      =  6228     tcpPassiveOpens     =    75
        tcpActiveOpens      =  6228     tcpPassiveOpens     =    75
        tcpActiveOpens      =  6228     tcpPassiveOpens     =    76
        tcpActiveOpens      =  6228     tcpPassiveOpens     =    85
        tcpActiveOpens      =  6228     tcpPassiveOpens     =   140
        tcpActiveOpens      =  6228     tcpPassiveOpens     =   197
        tcpActiveOpens      =  6228     tcpPassiveOpens     =   255

This is kinda fun to run on super busy web servers!!!

When I get deploy new Fujitsu and Sun hardware, I always run VTS (Validation Test Suite) on the hardware platforma. VTS performs rigorous hardware testing, and **usually** finds faults in components that are faulty out of the box. The VTS commands are available in “/opt/SUNWvts/bin.”

The VTS tools rely on the RPC framework ( *grumble* ), so you need to make sure rpcbind is running prior to invoking the various utilities. To start the VTS GUI, you can execute the “sunvts” or “fjvts” utility:

$ sunvts &

To get a curses based display, you can pass the “-t” option to the “sunvts” or “fjvts” utility:

$ sunvts -t

Once the display appears, I usually navigate to “set_options” -> “Test_Execution” and enable the “Verbose” and “Run On Error” options. The “Run On Error” option allows the validation test suite to continue operating if it finds an error. The “Verbose” option causes a plethora of information to be printed to the console display. Once these items are enabled, you can hit “start,” and watch the test suite leep into action. I find it useful to tail(1) /var/adm/messages while VTS is running. This will allow you to see hardware error discovery as it happen.

Name-based virtual hosts allow a web server to host multiple domain names (www.daemons.net, mail.daemons.net, blatch.daemons.net) from one IP address. This allows a web hosting infrastructure to conserve IP address space, and simplify namespace management.

Apache name-based virtual hosts are configured with the “NameVirtualHost” and “VirtualHost” directives, and rely on the HTTP “Host:” header attribute. This attribute is required in HTTP 1.1, and should be present with every request. The following example grabs /index.html using the HTTP/1.1 protocol:

$ telnet www.daemons.net 80
Trying 66.36.244.105...
Connected to www.daemons.net.
Escape character is '^]'.
GET / HTTP/1.1
Host: www.daemons.net

HTTP/1.1 200 OK
Date: Thu, 24 Feb 2005 16:33:23 GMT
Server: Apache/2.0.52
Last-Modified: Sun, 20 Jun 2004 14:39:21 GMT
ETag: "d54a2-912-c108d840"
Accept-Ranges: bytes
Content-Length: 2322
Content-Type: text/html

[ ... ]

Based on this output, it looks like my friend Clay needs to obscure his “Server:” header. Server identification is controlled with the “ServerTokens” directives.

The Solaris package commands (e.g., pkgproto, pkgadd, pkgtrans ) operate on two package formats. The first format is the “datastream” format. Packages created as datastream formatted packages use a single self contained file. This file includes the binary contents, application configuration files, and metadata to describe the package and installation process. The second format is the “file system format.” File system formatted packages contain hierarchical directory structures with all of the binaries, configuration, and metadata to describe the packages.

Both package formats can be installed with the pkgadd(1m) utility, and serve a unique purpose. Datastream formatted files are usually easier to distribute, since an archiving tool is removed from the installation/bundling process. File system formatted packages are nice to use with Solaris Jumpstart post install scripts, and make locating individual files within a package much easier.

The Solaris pkgtrans utility allows you to convert between both formats relatively easily. The following example takes a datastream formatted package, and converts it to a file system formatted package:

$ pkgtrans -o /tmp/RICHPse /var/tmp RICHPse

When pkgtrans is invoked with the “-s” option, packages can be converted from file system to datastream format:

$ pkgtrans -s /tmp /var/tmp/RICHPse.pkg RICHPse

A lot of people complain about Sun packages, but I find them easy to build, manage, and support.

Ever needed to grab a password protected page from the command line? This can be accomplished with curl’s “-u” option:

$ curl -k -i https://prefetch.net/secret -u me:somethingstrong |more

The username and password can be passed as an argument to the “-u” option. If you are paranoid about your password being visible on the command line, you can omit the password, and curl will prompt you for it:

$ curl -k -i https://prefetch.net/secret -u me
password:

In case you are curious, the “-k” option forces curl to dump the HTTP headers. I use both options to debug web server issues.

When the OpenBSD packet filter (PF) is configured to log traffic, each packet is logged to the OpenBSD “pflog” pseudo-device. This device can be queried with several tools, including tcpdump:

oscar# tcpdump -i pflog0 -ttt -e -o
tcpdump: WARNING: pflog0: no IPv4 address assigned
tcpdump: listening on pflog0
Jan 23 21:27:33.361173 rule 4/0(match): block in on tun0: 12.144.129.35 > adsl-19-10-38.asm.bellsouth.net: icmp: echo request
Jan 23 21:28:01.505716 rule 4/0(match): block in on tun0: 217.20.209.217.34777 > adsl-19-10-38.asm.bellsouth.net.socks: S (src OS: short-pkt) 3962893738:3962893738(0) win 5840 (DF)

If you are running a busy firewall, you are probably using pflogd to archive this information to a file on your FFS file system. I occassionally like to monitor pflog0 when I am testing new services, especially ones that don’t play nicely with firewalls.

Ever wonder how you can tunnel web and AIM traffic securely from one location to another? This can be accomplished with ssh’s “-D” option. This allows traffic to be sent securely over a SSH session, and routed out through a remote endpoint. This looks like:

Firefox/GAIM < -- HTTP/AIM--> loopback:PORT < -- SSH --> REMOTE END < -- HTTP/AIM --> Internet

To create a local proxy on TCP port 8000, we can pass the value 8000 to the “-D” option:

$ ssh -C -D 8000 -p 443 ick@ick.net

Once the SSH connection is established, you need to configure your client (e.g., firefox, gaim) to proxy connections to the loopback interface on TCP port 8000. Once your clients are configured to use the localhost.8000 listener, all application traffic will be sent securely through your ssh session, and routed through the Internet connection on the remote end.

Since most web proxies tunnel secure connections, you can setup your remote endpoint to accept SSH connections on TCP port 443. This is amazingly useful for routing around corporate firewalls and proxies. You don’t want to get caught looking for jobs while your at work, right? ;)

The FTP protocol uses a control channel to send commands to a server, and a data channel to send and receive files. The control channel by default uses TCP port 21, and the data channel is negotiated with the FTP PORT and PASV comands. When ACTIVE mode FTP is in use, the client chooses the port to use for data transfers. When PASSIVE mode FTP is used, the server is responsible for picking the data port.

With ACTIVE mode FTP, the client picks a high numbered port to use for the data transfer, and instructs the server to use this port by issuing a PORT command. For non application aware firewalls, these connections are usually problematic.

With PASSIVE mode FTP, the client issues a PASV FTP command to the server, and the server picks a port for the client to connect back on. All data is then transfered over this channel. This method works with most stateful firewalls, and is supported in most mainstream FTP clients.

The Solaris “ftp” command defaults to ACTIVE mode FTP, but supports PASSIVE mode FTP when invoked with the “-p” option:

$ ftp -p sunsite.unc.edu

Connected to sunsite.unc.edu.
220 ProFTPD Server (Bring it on...)
Name (sunsite.unc.edu:matty): anonymous
331 Anonymous login ok, send your complete email address as your password.
Password:
230-
Welcome to ftp.ibiblio.org, the public ftp server of ibiblio.org. We
hope you find what you're looking for.

If you have any problems or questions, please send email to

ftpkeeper@ibiblio.org

Thanks! 

230 Anonymous access granted, restrictions apply.
Remote system type is UNIX.
Using binary mode to transfer files.

For a super detailed explanation of ACTIVE and PASSIVE FTP, check out Slacksite:

http://slacksite.com/other/ftp.html

This is a super useful resource.