I just came across Rick Moen’s Preventing Domain Expiration article. Rick did a great job with the article, and it’s cool to see that they took my domain-check shell script and implemented it in Perl. The Perl version supports for TLDS, and contains a bit more functionality than the bash implementation. If I get some time in the next few months, I will have to update the domain-check bash script to support the same TLDs as the Perl implementation. Great job Rick and Ben!!
-
-
Categories
- Apache
- Apple
- Articles, Presentations and Certifications
- DNS & BIND
- Firefox
- FreeBSD Misc
- FreeBSD Networking
- FreeBSD Utilities
- Gnome
- GNU Privacy Guard
- Hardware
- HTTP
- Investing
- Java
- Links
- Linux Debugging
- Linux Installation
- Linux Kernel
- Linux LVM
- Linux Misc
- Linux Networking
- Linux Package Management
- Linux performance
- Linux Recovery
- Linux Security
- Linux Storage
- Linux Utilities
- Linux Virtualization
- Monitoring
- Music
- MySQL
- Networking
- NFSv3
- OpenBSD Kernel
- OpenBSD Misc
- OpenBSD Networking
- OpenBSD Ports
- OpenBSD Security
- OpenBSD Utilities
- OpenLDAP
- OpenSSH
- OpenSSL
- Perl
- Postfix
- Procmail
- Rants
- Samba
- Security
- Sendmail
- Service providers
- Smartmontools
- Snort
- Software
- Solaris Cluster
- Solaris Debugging
- Solaris DTrace
- Solaris Fault Management
- Solaris Kernel
- Solaris Linker
- Solaris Live Upgrade
- Solaris Misc
- Solaris Networking
- Solaris NFS
- Solaris Patching
- Solaris Recovery
- Solaris Resource Management
- Solaris Security
- Solaris Shell
- Solaris SMF
- Solaris Storage
- Solaris Utilities
- Solaris Volume Manager
- Solaris Xen
- Solaris ZFS
- Solaris Zones
- Storage Area Networking
- Sun Directory Server
- Sun Servers
- Sun Web Server
- Uncategorized
- UNIX
- UNIX Shell
- Veritas Cluster Server
- Veritas File System
- Veritas Netbackup
- Veritas Volume Manager
- VMWare ESX Server
- Wireless
- Wordpress
-
Blogroll
Search
While perusing my BIND query logs, I came across the following entry:
Nov 21 12:34:41 dns named[780]: [ID 866145 local0.info] client 1.2.3.4#32773: query: yikes.com IN MX -E
All of the text up to the record type (MX in this case) made sense, but I had no idea what the “-E” meant. Being the curious person I am, I dug through the BIND source code to locate the logging code. After a couple of find statements, I was able to locate the logging code in query.c:
ns_client_log(client, NS_LOGCATEGORY_QUERIES, NS_LOGMODULE_QUERY,
level, "query: %s %s %s %s%s%s", namebuf, classname,
typename, WANTRECURSION(client) ? "+" : "-",
(client->signer != NULL) ? "S": "",
(client->opt != NULL) ? "E" : "");
So a “+” or “-” in a query log entry indicates that a client requested recursion, and the “E” means that the query requested EDNS0. I would like to thank Knobee for his feedback on this post.
While debugging a DNS problem a few weeks back, I needed a way to measure the time it took a name server to respond to a DNS request. After poking around the OpenBSD ports collection, I came across the nsping utility. Nsping queries a DNS server passed on the command line, and reports the time it took the server to resolve a name. The following example shows how to use nsping to measure the time it takes to resolve the name prefetch.net on the name server ns2.dreamhost.com:
$ nsping -t 5 -h prefetch.net ns2.dreamhost.com
NSPING ns2.dreamhost.com (66.201.54.66): Hostname = "prefetch.net", Type = "IN A" + [ 0 ] 46 bytes from 66.201.54.66: 76.224 ms [ 0.000 san-avg ] + [ 1 ] 46 bytes from 66.201.54.66: 79.862 ms [ 78.043 san-avg ] + [ 3 ] 46 bytes from 66.201.54.66: 79.902 ms [ 78.663 san-avg ] + [ 4 ] 46 bytes from 66.201.54.66: 79.912 ms [ 78.975 san-avg ] + [ 6 ] 46 bytes from 66.201.54.66: 79.920 ms [ 79.164 san-avg ] ^C Total Sent: [ 7 ] Total Received: [ 5 ] Missed: [ 2 ] Lagged [ 0 ] Ave/Max/Min: 79.164 / 79.920 / 76.224
Each line contains the size of the response, the time it took to complete the request, and a sequence number. The summary line contains the numer of requests that were sent to the server, the number that were missing, and the average, maximum and minimum response times. If you want to use a resource record type other than “A,” (the default resource record type) you can invoke nsping with the “-T” option the resource record type to use:
$ nsping -t 5 -h prefetch.net -T mx ns1.dreamhost.com
NSPING ns1.dreamhost.com (66.33.206.206): Hostname = "prefetch.net", Type = "IN MX" + [ 0 ] 136 bytes from 66.33.206.206: 73.875 ms [ 0.000 san-avg ] + [ 1 ] 136 bytes from 66.33.206.206: 79.905 ms [ 76.890 san-avg ] + [ 2 ] 136 bytes from 66.33.206.206: 80.476 ms [ 78.085 san-avg ] + [ 3 ] 136 bytes from 66.33.206.206: 80.030 ms [ 78.572 san-avg ] + [ 6 ] 136 bytes from 66.33.206.206: 80.004 ms [ 78.858 san-avg ] ^C Total Sent: [ 7 ] Total Received: [ 5 ] Missed: [ 2 ] Lagged [ 0 ] Ave/Max/Min: 78.858 / 80.476 / 73.875
Now to figure out why DNS responses are missing!
I support BIND on a few servers, and when run as a caching name server, BIND can consume a fair amount of memory if you have lots of clients. There are two ways to restrict the amount of memory BIND uses. The first method, which is described in Pro DNS and BIND, is to set the “datasize” variable to the total amount of memory you want to allocate to BIND. The book provides an awesome description of this variable:
datasize
“The maximum amount of data memory the server may use. The default is default. This is a hard limit on server memory usage. If the server attempts to allocate memory in excess of this limit, the allocation will fail, which may in turn leave the server unable to perform DNS service. Therefore, this option is rarely useful as a way of limiting the amount of memory used by the server, but it can be used to raise an operating system data size limit that is too small by default. If you wish to limit the amount of memory used by the server, use the max-cache-size and recursive-clients options instead.”
The datasize variable is definitely useful in some cases, but can lead to server failures if BIND attempts to allocate memory above the threshold defined by datasize. A better method to limit memory is to use the max-cache-size variable, which will cause BIND to expire entries when it approaches the memory limit defined by max-cache-size. The Pro DNS and BIND book provides the following description of max-cache-size:
max-cache-size
“The maximum amount of memory to use for the server’s cache, in bytes. When the amount of data in the cache reaches this limit, the server will cause records to expire prematurely so that the limit is not exceeded. In a server with multiple views, the limit applies separately to the cache of each view. The default is unlimited, meaning that records are purged from the cache only when their TTLs expire.”
If you manage servers running BIND, I highly recommend picking up a copy of Pro DNS and BIND. It is an AWESOME book, and should be on every DNS admins bookshelf.
20 Aug
Monitoring DNS servers
I recently started supporting several DNS servers running BIND 9. To ensure that these server are up and operational at all times, I wrote a small shell script named dns-check to test the operational state of each server. The script takes a file as an argument, and each line in the file contains the IP address of a DNS server (names will also work), a name to resolve, and the record type that should be requested. If the script is unable to resolve the name for one reason or another (any return code > 0 is a failure), the script will log a message to syslog, and send E-mail to the address listed in the $ADMIN variable, or an address passed to the “-e” option. Here is sample run:
$ cat dns-check-sites
ns1.fooby.net mail.fooby.net A
ns2.fooby.net mail.fooby.net A
$ dns-check -e dns-admin@prefetch.net -f dns-check-sites
The script is nothing special, but might be useful to folks running DNS servers.
If you run DNS and SMTP servers, you probably know how important it is to validate the configurations used by your SMTP relays and DNS servers. Broken configurations can lead to clients not being able to find your website, open mail relays, unroutable mail, and your domain being blackholed by the Internet. Luckily there are three awesome services that can be used to sanity check the DNS and SMTP servers for a domain. The first service is the Mail abuse website, which checks to see if your mail server is acting as an open relay:
The second service is DNS report, which validates the DNS server configuration for a specific domain:
The last service is the open SPF website, which allows you to generate SPF (Sender Policy Framework) TXT records for a domain:
I use all three services, and can’t begin to describe how useful they are.
Internet advertising has become big business, and we see the effects of it in almost every page we view. The ad content typically comes from one or more well known ad servers, and some folks have come up with some clever ways (e.g., hosts files, DNS integration, etc.) to minimize the “ad effect” in the content we view. I have been using Mike’s host file for quiet some time, but for some reason OS X (actually lookupd) doesn’t handle large hosts files real well. Since OS X would get bogged down during DNS resolution, I decided to merge all of the ad domains into DNS to centrally fix the problem for the clients I support.
This was super easy to do, and only required two steps (assuming you are already running bind). The first step is to add one “zone” statement to named.conf for each ad domain you want to nix. The following example shows the named.conf entry you would add for the ad domain adservers.com:
zone "adservers.com"
{
type master; notify no; file "master/null.zone";
};
You can get a comprehensive list of the well known ad server domains from the ad blocking website. Once you retrieve the list, you can merge the domains into the named.conf using a combination of shell utilities, or you can download the Perl script (updateads.pl) I wrote to automate this process. The Perl script grabs the latest host file from the ad blocking website, formats the data, and spits out several lines that can be appended to named.conf:
$ updateads.pl |more
// *** Added domains on Thu May 25 13:53:34 2006 *** //
zone "ad1.com" { type master; notify no; file "master/null.zone"; };
zone "ad2.com" { type master; notify no; file "master/null.zone"; };
zone "ad3.com" { type master; notify no; file "master/null.zone"; };
[ ..... ]
Once you add all of the domains to named.conf, you need to create a zone file with one wildcard A record (this record is what is used to remove the ad servers, since the wildcard record will translate all entries in a given domain to 127.0.0.1). I am currently using the following zone file (with different domain names) to implement my ad blocking solution:
; File: null.zone
; Last modified: 07-10-2005
$TTL 86400 ; one day
@ IN SOA ns.mydomain.com hostmaster.mydomain.com. (
2005071005 ; serial number YYYYMMDDNN
28800 ; refresh 8 hours
7200 ; retry 2 hours
864000 ; expire 10 days
86400 ) ; min ttl 1 day
NS ns.mydomain.com.
A 127.0.0.1
* IN A 127.0.0.1
I have found that using this technique speeds up the time it takes to render a page, enhances privacy, and will also cut down on the amount of traffic consumed by your site. Tis good stuff!
I have used nslookup for years to access information in the Domain Name Systems (DNS), and was sad to see that it was moved to the “deprecated” state when dig (domain information groper) was released. Since I knew nslookup(1m) inside and out, I put off learning about dig, and instead focused on learning other interesting technologies. I recently wanted to add TSIG support to a DNS server, and needed to be able to test signing keys. I didn’t see support in nslookup, so I decided to sit down and learn everything there was to know about dig.
To get started with dig, you can execute dig with the “-h” (help) option:
$ dig -h
Usage: dig [@global-server] [domain] [q-type] [q-class] {q-opt}
{global-d-opt} host [@local-server] {local-d-opt}
[ host [@local-server] {local-d-opt} [...]]
Where: domain are in the Domain Name System
q-class is one of (in,hs,ch,...) [default: in]
q-type is one of (a,any,mx,ns,soa,hinfo,axfr,txt,...) [default:a]
(Use ixfr=version for type ixfr)
q-opt is one of:
-x dot-notation (shortcut for in-addr lookups)
-n (nibble form for reverse IPv6 lookups)
-f filename (batch mode)
-b address (bind to source address)
-p port (specify port number)
-t type (specify query type)
-c class (specify query class)
-k keyfile (specify tsig key file)
-y name:key (specify named base64 tsig key)
d-opt is of the form +keyword[=value], where keyword is:
+[no]vc (TCP mode)
+[no]tcp (TCP mode, alternate syntax)
+time=### (Set query timeout) [5]
+tries=### (Set number of UDP attempts) [3]
+domain=### (Set default domainname)
+bufsize=### (Set EDNS0 Max UDP packet size)
+ndots=### (Set NDOTS value)
+[no]search (Set whether to use searchlist)
+[no]defname (Ditto)
+[no]recursive (Recursive mode)
+[no]ignore (Don't revert to TCP for TC responses.)
+[no]fail (Don't try next server on SERVFAIL)
+[no]besteffort (Try to parse even illegal messages)
+[no]aaonly (Set AA flag in query)
+[no]adflag (Set AD flag in query)
+[no]cdflag (Set CD flag in query)
+[no]cmd (Control display of command line)
+[no]comments (Control display of comment lines)
+[no]question (Control display of question)
+[no]answer (Control display of answer)
+[no]authority (Control display of authority)
+[no]additional (Control display of additional)
+[no]stats (Control display of statistics)
+[no]short (Disable everything except short
form of answer)
+[no]all (Set or clear all display flags)
+[no]qr (Print question before sending)
+[no]nssearch (Search all authoritative nameservers)
+[no]identify (ID responders in short answers)
+[no]trace (Trace delegation down from root)
+[no]dnssec (Request DNSSEC records)
+[no]multiline (Print records in an expanded format)
global d-opts and servers (before host name) affect all queries.
local d-opts and servers (after host name) affect only that lookup.
This will print all of the availble options along with a short description of the option. When used to lookup information in DNS, dig takes the following form:
dig [@name server to use] [domain] [record-type]
“@server” represents the DNS server to query, “domain” represents the domain entry to query (e.g., www.daemons.net), and “record-type” indicates the type of record to retrieve. To lookup the A record for the host www.daemons.net, dig can be executed with the domain entry to resolve:
$ dig www.daemons.net
; <<>> DiG 9.2.2 <<>> www.daemons.net ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19436 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 2 ;; QUESTION SECTION: ;www.daemons.net. IN A ;; ANSWER SECTION: www.daemons.net. 16140 IN A 66.148.84.65 ;; AUTHORITY SECTION: daemons.net. 28078 IN NS olympus.daemons.net. daemons.net. 28078 IN NS elysium.daemons.net. ;; ADDITIONAL SECTION: elysium.daemons.net. 121832 IN A 206.222.17.178 olympus.daemons.net. 35432 IN A 66.148.71.8 ;; Query time: 22 msec ;; SERVER: 192.168.1.1#53(192.168.1.1) ;; WHEN: Thu Oct 20 16:12:51 2005 ;; MSG SIZE rcvd: 125
This will cause dig to retrive the A record (the default record type) from one of the servers defined in the local name server repository (e.g., /etc/resolv.conf). If you would like to specify the DNS server to query, the server name can be appended to the “@” symbol and passed as an argument to dig:
$ dig @192.168.1.1 www.daemons.net A
; <<>> DiG 9.2.2 <<>> @192.168.1.1 www.daemons.net A ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35663 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 2 ;; QUESTION SECTION: ;www.daemons.net. IN A ;; ANSWER SECTION: www.daemons.net. 15949 IN A 66.148.84.65 ;; AUTHORITY SECTION: daemons.net. 27887 IN NS olympus.daemons.net. daemons.net. 27887 IN NS elysium.daemons.net. ;; ADDITIONAL SECTION: elysium.daemons.net. 121641 IN A 206.222.17.178 olympus.daemons.net. 35241 IN A 66.148.71.8 ;; Query time: 11 msec ;; SERVER: 192.168.1.1#53(192.168.1.1) ;; WHEN: Thu Oct 20 16:16:03 2005 ;; MSG SIZE rcvd: 125
As with nslookup, dig can be used to retrieve all of the standard record types (e.g., A, MX, SOA, NS, etc.). The following example show how to retrieve the MX (mail exchangers) records for the domain daemons.net:
$ dig @192.168.1.1 daemons.net MX
; <<>> DiG 9.2.2 <<>> @192.168.1.1 daemons.net MX ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 2, ADDITIONAL: 4 ;; QUESTION SECTION: ;daemons.net. IN MX ;; ANSWER SECTION: daemons.net. 84867 IN MX 20 deimos.daemons.net. daemons.net. 84867 IN MX 10 phobos.daemons.net. ;; AUTHORITY SECTION: daemons.net. 27791 IN NS elysium.daemons.net. daemons.net. 27791 IN NS olympus.daemons.net. ;; ADDITIONAL SECTION: phobos.daemons.net. 2029 IN A 206.222.17.179 deimos.daemons.net. 84867 IN A 66.148.71.29 elysium.daemons.net. 121545 IN A 206.222.17.178 olympus.daemons.net. 35145 IN A 66.148.71.8 ;; Query time: 23 msec ;; SERVER: 192.168.1.1#53(192.168.1.1) ;; WHEN: Thu Oct 20 16:17:38 2005 ;; MSG SIZE rcvd: 183
dig also supports numerous options to control which pieces of the query and response are displayed. To view just the result of a query (e.g., just an IP or FQDN), dig’s “+short” option can be used to disable all extraneous output (perfect for shell scripts):
$ dig +short @192.168.1.1 www.daemons.net a
66.148.84.65
If you need to resolve IP addresses to hostname, you are in luck! dig provides the “-x” option to retrieve the PTR record for in-addr.arpa entries, and performs all the translations for you:
$ dig @192.168.1.1 -x 206.222.17.178
; <<>> DiG 9.2.2 <<>> @192.168.1.1 -x 206.222.17.178 ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36325 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 2 ;; QUESTION SECTION: ;178.17.222.206.in-addr.arpa. IN PTR ;; ANSWER SECTION: 178.17.222.206.in-addr.arpa. 86389 IN PTR elysium.daemons.net. ;; AUTHORITY SECTION: 17.222.206.in-addr.arpa. 86389 IN NS dns3.ee.net. 17.222.206.in-addr.arpa. 86389 IN NS dns1.ee.net. 17.222.206.in-addr.arpa. 86389 IN NS dns2.ee.net. ;; ADDITIONAL SECTION: dns2.ee.net. 172789 IN A 206.222.1.2 dns3.ee.net. 172789 IN A 206.222.1.3 ;; Query time: 20 msec ;; SERVER: 192.168.1.1#53(192.168.1.1) ;; WHEN: Thu Oct 20 20:18:46 2005 ;; MSG SIZE rcvd: 170
Dig can also perform zone file transfers with the AXFR protocol:
$ dig @66.148.71.8 daemons.net AXFR
;; Connection to 66.148.71.8#53(66.148.71.8) for daemons.net failed: connection refused.
The daemons.net DNS adminstrator is limiting zone transfers to pre-defined IP address, so unfortunately the transfer did not complete succesfully. If zone transfers were supported, the AXFR would retrieve the zone file for the domain daemons.net. If you are having problems with DNS reslution, dig provides numerous options to help troubleshoot problems. The “trace” option can be super useful for tracing a query through the entire hierarchical DNS system:
$ dig +trace @192.168.1.1 www.daemons.net a
; <<>> DiG 9.2.2 <<>> +trace @192.168.1.1 www.daemons.net a ;; global options: printcmd . 518210 IN NS L.ROOT-SERVERS.NET. . 518210 IN NS M.ROOT-SERVERS.NET. . 518210 IN NS A.ROOT-SERVERS.NET. . 518210 IN NS B.ROOT-SERVERS.NET. . 518210 IN NS C.ROOT-SERVERS.NET. . 518210 IN NS D.ROOT-SERVERS.NET. . 518210 IN NS E.ROOT-SERVERS.NET. . 518210 IN NS F.ROOT-SERVERS.NET. . 518210 IN NS G.ROOT-SERVERS.NET. . 518210 IN NS H.ROOT-SERVERS.NET. . 518210 IN NS I.ROOT-SERVERS.NET. . 518210 IN NS J.ROOT-SERVERS.NET. . 518210 IN NS K.ROOT-SERVERS.NET. ;; Received 340 bytes from 192.168.1.1#53(192.168.1.1) in 22 ms net. 172800 IN NS A.GTLD-SERVERS.net. net. 172800 IN NS G.GTLD-SERVERS.net. net. 172800 IN NS H.GTLD-SERVERS.net. net. 172800 IN NS C.GTLD-SERVERS.net. net. 172800 IN NS I.GTLD-SERVERS.net. net. 172800 IN NS B.GTLD-SERVERS.net. net. 172800 IN NS D.GTLD-SERVERS.net. net. 172800 IN NS L.GTLD-SERVERS.net. net. 172800 IN NS F.GTLD-SERVERS.net. net. 172800 IN NS J.GTLD-SERVERS.net. net. 172800 IN NS K.GTLD-SERVERS.net. net. 172800 IN NS E.GTLD-SERVERS.net. net. 172800 IN NS M.GTLD-SERVERS.net. ;; Received 502 bytes from 198.32.64.12#53(L.ROOT-SERVERS.NET) in 81 ms daemons.net. 172800 IN NS elysium.daemons.net. daemons.net. 172800 IN NS olympus.daemons.net. ;; Received 109 bytes from 192.5.6.30#53(A.GTLD-SERVERS.net) in 39 ms www.daemons.net. 86400 IN A 66.148.84.65 daemons.net. 86400 IN NS olympus.daemons.net. daemons.net. 86400 IN NS elysium.daemons.net. ;; Received 125 bytes from 206.222.17.178#53(elysium.daemons.net) in 46 ms
The output shows that the first query requested the TLD server responsible (L.ROOT-SERVERS.NET in this case) for the .net domain, the second query requested the name servers registered to daemons.net, and the final query requested the A record for the host www.daemons.net. Dig is a super cool and powerful utility!! I how to cover TSIG and signed DNS queries in a future BLOG post.