Using TCP Wrappers to protect Linux and Solaris services

I have been using tcp wrappers for years, and it’s a very simple way to allow and deny network access to applications. TCP wrapper functionality is built into the system libwrap.so module, which various applications are linked against. To see if a given application supports tcp wrappers, you can use the ldd utility:

$ ldd `which sshd` | grep wrap
libwrap.so.0 => /lib64/libwrap.so.0 (0x00002ac16fe0f000)

TCP wrappers is configured through the /etc/hosts.allow and /etc/hosts.deny files. The hosts.allow file allows you to control which services will be accepted, and the hosts.deny file allows you to control which services will be denied. Both files use a format similar to the following:

DAEMON_LIST : CLIENT_LIST [ : SHELL_COMMAND ]

The DAEMON_LIST contains the name of the executable you are protecting, which could be sshd, sendmail or any other daemon that you are trying to protect. The CLIENT_LIST contains the hosts or domain names you wish to allow or deny access to, and they can take various forms:

ALL — matches everything
.prefetch.net — matches everything in the prefetch.net domain
192.168.0.0/255.255.0.0 — matches everything in the 192.168 /16 IP address space
192.168.1.1 — matches a single IP address

SHELL_COMMAND allows you to run a command when the rule matches. This could be used to run a notification script, block an IP with iptables or to provide some more extensive logging. To put this into action, we can set up our hosts.allow and hosts.deny files to limit access to our SSH daemon. The following hosts.allow will allow connections from the IP 192.168.1.100, and deny access from everyone else:

$ cat /etc/hosts.allow
sshd : 192.168.1.100

$ cat /etc/hosts.deny
ALL : ALL

When libwrap processes these files, it will first look for matches in /etc/hosts.allow by sequentially evaluating the rules. If a match isn’t found, it will then consult the hosts.deny file. If a connection is denied, you should see a message similar to the following in the messages file:

Apr 16 13:16:18 localhost sshd[3628]: refused connect from ::ffff:192.168.1.8 (::ffff:192.168.1.8)

TCP wrappers is an invaluable tool, and provides a simple and intuitive way to secure your services. It’s no substitute for a properly functioning host firewall, but an additional tool that can be used to protect your critical services.

The case of the missing SSH keys

I built a couple of new Solaris 10 hosts today using a stripped down image, and was greeted with the following error when I tried to log in:

$ ssh 192.168.1.20
Unable to negotiate a key exchange method

The server was spitting out “no kex alg” errors, which appear to be due to key exchange issues. I poked around my sshd_config file, and for some reason the host host keys weren’t generated when the ssh service initialized. To fix this, I ran the ssh service with the -c option (this generated the RSA and DSA host keys):

$ /lib/svc/method/sshd -c

added the host keys to my sshd configuration file:

# Paths to host keys
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key

And then ran ‘svcadm refresh ssh’ to restart the service. Once that completed, I was able to login to the host. Nice!

Logging su attempts and failed logins

As a conscientious Solaris administrator, I make every attempt possible to protect my servers from malicious users. This includes disabling all unneeded services, enabling strong password policies, configuring system auditing, enabling strong network defaults, applying system patches and configuring system logging. When I configure system logging, I like to configure the syslogd daemon to log everything to a centralized location. This is typically accomplished by adding an entry similar to the following to /etc/syslog.conf:

*.debug @logserver.prefetch.net

Additionally, I like to log each time a user logs into my systems, as well as all attempts to su to another user. To log all su attempts, the file /var/adm/sulog can be created (in recent releases of Solaris, this file is created by default):

$ touch /var/adm/sulog

To log all successful and unsuccessful logins, you will first need to set the variable SYSLOG_FAILED_LOGINS in /etc/default/login to the value 0. Once the variable is adjusted, you will need to create a log file to store the login attempts:

$ touch /var/adm/loginlog

After the log file is created, the auth priority needs to be added to /etc/syslog.conf:

auth.debug /var/adm/loginlog

With the loginlog and sulog files in place, it is relativley easy to see who accessed a given system at time X, and who tried to become the super user.

LDAP client deficiencies

I have been spending a bit of time lately configuring Solaris and Linux hosts to authenticate against LDAP. Authentication works well on the surface, but the actual client implementations are somewhat lacking. Let’s take the Linux pam_ldap module for instance. To authenticate a single session, the pam_ldap module performs thirty-three operations, which includes 7 TCP connections and a number of redundant searches. Here is what I see in my logfile for each login session that is authenticated through pam_ldap:

1. New connection
  conn=801 
  LDAP connection from 10.10.20.3 to 10.10.20.2

2. Anonymous BIND
  conn=801 
  BIND dn="" method=128 version=3

3. Search
   conn=801
   SRCH base="ou=people,dc=prefetch,dc=net" 
   FILTER="(&(objectClass=posixAccount)(uid=test))" 
   ATTRIBUTES="uid userPassword uidNumber gidNumber cn homeDirectory 
              loginShell gecos description objectClass"
   nentries => 1

4. New connection
   conn=802
    LDAP connection from 10.10.20.3 to 10.10.20.2

5. Anonymous BIND
   conn=802
   BIND dn="" method=128 version=3

6. Search
   conn=802
   SRCH base="ou=people,dc=prefetch,dc=net" 
   FILTER="(&(objectClass=posixAccount)(objectClass=posixAccount)(uid=test))"    
   ATTRIBUTES=ALL
   nentries => 1

7. BIND as user
   conn=802
   BIND dn="uid=test,ou=People,dc=prefetch,dc=net" method=128 version=3

8. Anonymous BIND
   conn=802
   BIND dn="" method=128 version=3

9. Close connection 801

10. New connection
    conn=803
    LDAP connection from 10.10.20.3 to 10.10.20.2

11. Anonymous BIND
    conn=803
    BIND dn="" method=128 version=3

12. Search
    conn=803
    SRCH base="ou=people,dc=prefetch,dc=net" 
    FILTER="(&(objectClass=posixAccount)(uid=test))" 
    ATTRIBUTES=ALL
    nentries => 1
    
13. Search
    conn=803
    SRCH base="ou=people,dc=prefetch,dc=net" 
    FILTER="(&(objectClass=posixGroup)(|(memberUid=test)
           (uniqueMember=uid=test,ou=People,dc=prefetch,dc=net)))" 
    attrs="gidNumber"
    nentries => 0

14. Close connection 802

15. New connection
    conn=804
    LDAP connection from 10.10.20.3 to 10.10.20.2

16. Anonymous BIND
    conn=804
    BIND dn="" method=128 version=3

17. Search
    conn=804
    SRCH base="ou=people,dc=prefetch,dc=net" 
    FILTER="(&(objectClass=posixAccount)(uidNumber=100))" 
    ATTRIBUTES="uid userPassword uidNumber gidNumber cn homeDirectory 
               loginShell gecos description objectClass"
    nentries => 1

18. Search
    conn=804
    SRCH base="ou=people,dc=prefetch,dc=net" 
    FILTER="(&(objectClass=posixAccount)(uid=test))" 
    ATTRIBUTES="uid userPassword uidNumber gidNumber cn homeDirectory 
               loginShell gecos description objectClass"
    nentries => 1

19. New connection
    conn=805
    LDAP connection from 10.10.20.3 to 10.10.20.2

20. Anonymous BIND
    conn=805
    BIND dn="" method=128 version=3

21. Search
    conn=805
    SRCH base="ou=people,dc=prefetch,dc=net" 
    FILTER="(&(objectClass=posixAccount)(uidNumber=100))" 
    ATTRIBUTES="uid userPassword uidNumber gidNumber cn homeDirectory 
               loginShell gecos description objectClass"
    nentries => 1

22. New connection
    conn=806
    LDAP connection from 10.10.20.3 to 10.10.20.2

23. Anonymous BIND
    conn=806
    BIND dn="" method=128 version=3

24. Search
    conn=806
    SRCH base="ou=people,dc=prefetch,dc=net" 
    FILTER="(&(objectClass=posixAccount)(uidNumber=100))" 
    ATTRIBUTES="uid userPassword uidNumber gidNumber cn homeDirectory 
               loginShell gecos description objectClass"
    nentries => 1


25. Close connection 806

26. New connection
    conn=807
    LDAP connection from 10.10.20.3 to 10.10.20.2

27. Anonymous BIND
    conn=807
    BIND dn="" method=128 version=3

28. Search
    conn=807
    SRCH base="ou=people,dc=prefetch,dc=net" 
    FILTER="(&(objectClass=posixAccount)(uidNumber=100))" 
    ATTRIBUTES="uid userPassword uidNumber gidNumber cn homeDirectory 
               loginShell gecos description objectClass"
    nentries => 1

29. Close connection 807

30. Close connection 805

31. Search
    conn=804
    SRCH base="ou=people,dc=prefetch,dc=net" 
    FILTER="(&(objectClass=posixAccount)(uid=test))" 
    ATTRIBUTES="uid userPassword uidNumber gidNumber cn homeDirectory 
               loginShell gecos description objectClass"
    nentries => 1

32. Close connection 803

33. Close connection 804

The Sun native LDAP client is not much better, and I am somewhat curious why the PAM modules couldn’t be written to use persistent connections, and to cache data between PAM phases. If you are using either of these implementations to authenticate users, I sure hope you’re using the name service caching daemon (nscd). :)

Checking the integrity of Solaris binaries

One new feature in Solaris 10 that doesn’t get much press is the basic auditing and reporting tool (bart). Bart allows you to generate integrity checks for one or more files on a server. This allows you to compare two groups of file integrity checks (groups of file integrity checks are referred to as manifests in the bart documentation) to see what changed on a server. Bart is super easy to use, and comes with just two options, “create” and “compare.” The “create” option can be used to create a new manifest, and the “compare” option can be used to compare the contents of two manifests. The following example show how to use the “create” option to generate a file integrity check of every file that resides in a global zone’s* root file system:

$ bart create -R / > bart.manifest.08-14-2006.1

$ bart create -R / > bart.manifest.08-14-2006.2

One two manifests are created, the bart “compare” option can be run to compare the manifests:

$ bart compare bart.manifest.08-14-2006.1 bart.manifest.08-14-2006.2

/var/adm/messages:
  size  control:8866  test:8957
  mtime  control:44e100a3  test:44e1019e
  contents  control:b349f015631c87065842009d87a1a456    
  test:be07b4863f18165fcd154b9f0fce2a64

/var/cron/log:
  size  control:76152  test:76396
  mtime  control:44e10070  test:44e1019d
  contents  control:7cd2f996f0cec248cd5eae4f3e6cce7e  
  test: 29bf6ecbd171ebe1879e641d5b5739f2

/var/log/pool/poold:
  size  control:651159  test:652111
  mtime  control:44e10160  test:44e10232
  contents  control:9339cb8fac19bb9231e35866cd1a2942  test:89880fbd73332cfc770454fdd034cba1

/var/svc/log/network-ssh:default.log:
  size  control:226076  test:226181
  mtime  control:44e10070  test:44e1019d
  contents  control:5a856f39ede7c7528f9405f573eedd5b  
  test:778ebe08677923862b03aec5d41e3c51

As you can see from the output above, several logfiles changed between two consecutive runs. While not a complete file integrity solution, bart is a super useful utility, and should be used after each system installation and patch application.

* The bart manual page states that you shouldn’t run bart on the root file system in a non-global zone.

Solaris secure by default initiative!

While perusing the latest Nevada build notes, I came across the following PSARC case:

PSARC case 2004/368 : Secure By Default
BUG/RFE:4875624 *syslogd* turn off UDP listener by default
BUG/RFE:5004374 Ship with remote services disabled by default
BUG/RFE:5016956 By default rpcbind should not listen for remote requests
BUG/RFE:5016975 By default snmpd/dx should not be enabled.
BUG/RFE:5016998 By default inetd should not listen for remote connections.
BUG/RFE:5017041 By default sendmail should not listen for remote connections
BUG/RFE:5046450 Create a greenline profile for Secure by Default installation
BUG/RFE:6267741 RFE: One-touch knob for outbound-only sendmail
BUG/RFE:6414308 syslogd could use some lint soap

I have been bitching about the number of services that come enabled by default for the past ten years, and am SUPER excited to see that Sun finally fixed this annoyance! Nice!