Configuring OpenBSD to support cable modems

I recently switched from DSL to cable, and had to learn how to use the ISC DHCP software with OpenBSD and my cable provider (Comcast). This was relatively easy, and the dhclient(8) manpage has lots of useful information and examples. Once I read through the man page, I created a file called/etc/dhcpclient.conf with the following information:

interface "hme0" {
send host-name "OBSD1";
request subnet-mask, broadcast-address, time-offset, routers;
}

script "/etc/dhclient-end";

This configuration file tells dhclient(8)to request DHCP services on the hme0 interface. The services are defined between curly braces, and include default routers, DNS servers, and the time a lease is valid ( I personally only grab netmask, broadcast, lease time, IP addresses and default routers from the Comcast DHCP servers). I also created the following shell script to flush the PF stable table, and add a new Pf policy during lease negotiation:

#!/bin/sh

# Startup PF
/sbin/pfctl -e -F all -f /etc/pf.conf

Once the configuration is in place, the dhclient(8) utility can be invoked to grab a lease on an interface defined in dhclient.conf:

$ dhclient hme0

When you are attempting to get DHCP working, you can run the dhclient(8) utility with the “-d” option. This will print tons of debugging information, and help you clear up issues in a timely manner. Now that the DHCP connection is working, I added the following to /etc/rc.conf to grab a lease when the server boots:

# Bringup hme0
echo -n "Bringing hme0 online..."
/sbin/ifconfig hme0 up
echo "Done"

# Startup the cable connection 
echo -n "Startup Cable connection..."
/sbin/dhclient hme0
echo "Done"

If you want to review the leases and expiration times, you can view the contents of
“/var/db/dhclient.leases”:

$ ls -al /var/db/dhclient.leases
———- 1 root wheel 796 Mar 27 13:48 /var/db/dhclient.leases

You should enable pf in /etc/rc.conf, and ensure a default policy is in place prior to grabbing the initial lease. There is a period of time when you will be sitting on the Internet unprotected. It might be paranoia, but I like to have a DROP ALL but DHCP policy in place prior to dhclient starting.

Printing the entries in a sendmail queue

Since I began managing sendmail installations 8-years ago, I have used the mailq command to print the entries in the sendmail queue. While reading through the bat book to find the root cause of my sendmail DNS problems, I came across sendmail’s “-bp” option:

$ /usr/sbin/sendmail -bp

/var/spool/mqueue is empty
                Total requests: 0

This provides the same results as the mailq command:

$ mailq

/var/spool/mqueue is empty
                Total requests: 0

I am still amazed at how complex the sendmail program is. Rule 0, Rule 3, Rule 5, 8000 macros … ge gads!

OpenBSD kernel memory statistics

The OpenBSD platform uses mbuf(9) data structures, macros, and function calls to manage the kernel memory used by the various networking protocols. An adequate supply of mbuf(9)’s is required for proper operation, and mbuf(9) starvation can lead to kernel panics. You can monitor mbuf(9) usage with the netstat utilities “-m” option:

$ netstat -m
136 mbufs in use:
        129 mbufs allocated to data
        3 mbufs allocated to packet headers
        4 mbufs allocated to socket names and addresses
33/56/256 mbuf cluster pages in use (current/peak/max)
576 Kbytes allocated to network (50% in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines

If you run out of mbufs, you can increase the NMBCLUSTERS option to add more.

Optimizing sendmail smart hosts

While debugging an issue last week, I noticed some odd behavior when the Sendmail Smart host configuration directive (DS) was used. Smart hosts allow nodes to delegate mail delivery services to upstream devices, alleviating the need to run a full blown mail relay on each host. Smart hosts are configured with the Sendmail MC SMART_HOST directive, or through the sendmail.cf/submit.cf DS variable:

$ grep DS /etc/mail/submit.cf
DSsmtp-relay.dns.net

While I was performing a packet capture to isolate a problem with mail delivery, I noticed that each outbound message triggered several odd DNS lookups:

$ snoop -d eri0 port 53

s1 -> dnsserver.dns.net DNS C smtp-relay.dns.net. Internet MX ?
dnsserver.dns.net -> s1 DNS R

s1 -> dnsserver.dns.net DNS C smtp-relay.dns.net.internal.dns.net. Internet MX ?
dnsserver.dns.net -> s1 DNS R Error: 3(Name Error)

s1 -> dnsserver.dns.net DNS C smtp-relay.dns.net.corp.dns.net. Internet MX ?
dnsserver.dns.net -> s1 DNS R Error: 3(Name Error) 

s1 -> dnsserver.dns.net DNS C smtp-relay.dns.net.dns.net. Internet MX ?
dnsserver.dns.net -> s1 DNS R Error: 3(Name Error)

s1 -> dnsserver.dns.net DNS C smtp-relay.dns.net. Internet Unknown (28) ?
dnsserver.dns.net -> s1 DNS R

s1 -> dnsserver.dns.net DNS C smtp-relay.dns.net. Internet Addr ?
dnsserver.dns.net -> s1 DNS R smtp-relay.dns.net. Internet Addr 1.2.3.4 

Upon closer inspection, I noticed that sendmail (actually the resolution libraries) were attempting to resolve the SMTP relay along with each entry in the search path. To fix this issue ( I think it’s an issue, since it’s wasting bandwidth and CPU cycles), I fully qualified the smart host in my submit.cf:

$ grep DS /etc/mail/submit.cf
DSsmtp-relay.dns.net.

Once this was changed, and sendmail was restarted, the resolver only requested entries for the FQDN:

s1 -> dnsserver.dns.net DNS C smtp-relay.dns.com. Internet MX ?
dnsserver.dns.net -> s1 DNS R

s1 -> dnsserver.dns.net DNS C smtp-relay.dns.com. Internet Unknown (28) ?
dnsserver.dns.net -> s1 DNS R

s1 -> dnsserver.dns.net DNS C smtp-relay.dns.com. Internet Addr ?
dnsserver.dns.net -> s1 DNS R smtp-relay.dns.com. Internet Addr 1.2.3.4

I just dusted off my bat book, and hope to find an MC setting to disable MX lookups for the smart host. That should kill four more UDP datagrams. :)

IP filter logging

In my previous post, I described how to setup IP filter to protect a desktop workstation. The IP filter policy allowed SSH connections, and dropped and logged all other inbound traffic. When a packet is dropped and logged, IP filter writes the packet headers to the /dev/ipl pseudo-device. This device is monitored by ipmon(1m), which is started by the IP filter initialization scripts. When ipmon(1m) detects that a new entry has been logged to /dev/ipl, the message is routed to syslogd’s local0 facility ( you can also configure IP filter to log directly to a file), using one of several priorities listed in the ipmon(1m) manual page:

LOG_INFO
  Packets logged using the log keyword as the action
  rather than pass or block.

LOG_NOTICE
  Packets logged that are also passed.

LOG_WARNING
  Packets logged that are also blocked.
 
LOG_ERR
  Packets that have been logged and that can  be  con-
  sidered "short".

To get my IP filter installation to log everything, I added the following entry to /etc/syslog.conf:

$ grep local0 /etc/syslog.conf
local0.debug /var/log/ipflog

Once the entry is added to the syslog.conf configuration file (remember to use tabs to delimit the entries), syslogd needs to be restarted. This can be accomplished with the svcadm(1m) utility:

$ /usr/sbin/svcadm restart system-log

When traffic is blocked, you should see an entry in “/var/log/ipflog”:

$ ls -la /var/log/ipflog
-rw-r–r– 1 root root 33765 Mar 13 21:08 /var/log/ipflog

I recommend adding /var/log/ipflog to /etc/logadm.conf (or newsyslog.conf if you are using the BSD Operating Systems). This will ensure that your logs get rotated periodically ( on busy MSFT networks, it will grow by leaps and bounds!).

How does nohup work?

I have used nohup(1) for years to startup processes, and to ensure they keep running when my shell exits. When a shell exits, each child process will receive a SIGHUP signal, which causes the process to exit if a signal handler is not installed to deal with the SIGHUP signal. When a command is invoked with the nohup(1) utility, the signal disposition for SIGHUP is set to ignored, allowing the process to continue executing when the shell exits. We can see this with the Solaris “psig” command:

$ ssh oscar &
[1] 958

$ psig 958 | grep HUP
HUP default

The psig utility indicates that the SIGHUP disposition is set to the default, which will cause the process to terminate when we exit the shell. When the same command is invoked with the nohup utility, we can see that the signal disposition for SIGHUP is set to ignored:

$ nohup ssh oscar &
[2] 967

$ psig 967 | grep HUP
HUP ignored

Solaris is an amazing Operating system, and allows the signal dispositions of running processes (and process groups!!) to be set on the fly. This is accomplished with nohup’s “-p” and “-g” options:

$ ssh -p 443 oscar &
[1] 1081

$ psig 1081 | grep HUP
HUP default

$ nohup -p 1081
Sending output to nohup.out

$ psig 1081 | grep HUP
HUP ignored

While this isn’t the best example, hopefully you get the point. Sessions, process groups, process group leaders and controlling terminals are really neat concepts, and explained on pages 677 – 700 of Solaris Systems Programming (ISBN: 0201750392). This is an INCREDIBLE book, and sits next to my lazy boy for easy reference.