Using BIND to reduce ad server content

Internet advertising has become big business, and we see the effects of it in almost every page we view. The ad content typically comes from one or more well known ad servers, and some folks have come up with some clever ways (e.g., hosts files, DNS integration, etc.) to minimize the “ad effect” in the content we view. I have been using Mike’s host file for quiet some time, but for some reason OS X (actually lookupd) doesn’t handle large hosts files real well. Since OS X would get bogged down during DNS resolution, I decided to merge all of the ad domains into DNS to centrally fix the problem for the clients I support.

This was super easy to do, and only required two steps (assuming you are already running bind). The first step is to add one “zone” statement to named.conf for each ad domain you want to nix. The following example shows the named.conf entry you would add for the ad domain adservers.com:

zone "adservers.com" 
{ 
       type master; notify no; file "master/null.zone"; 
};

You can get a comprehensive list of the well known ad server domains from the ad blocking website. Once you retrieve the list, you can merge the domains into the named.conf using a combination of shell utilities, or you can download the Perl script (updateads.pl) I wrote to automate this process. The Perl script grabs the latest host file from the ad blocking website, formats the data, and spits out several lines that can be appended to named.conf:

$ updateads.pl |more

// *** Added domains on Thu May 25 13:53:34 2006 *** //

zone "ad1.com" { type master; notify no; file "master/null.zone"; };
zone "ad2.com" { type master; notify no; file "master/null.zone"; };
zone "ad3.com" { type master; notify no; file "master/null.zone"; };
[ ..... ]

Once you add all of the domains to named.conf, you need to create a zone file with one wildcard A record (this record is what is used to remove the ad servers, since the wildcard record will translate all entries in a given domain to 127.0.0.1). I am currently using the following zone file (with different domain names) to implement my ad blocking solution:

; File: null.zone
; Last modified: 07-10-2005

$TTL    86400   ; one day

@       IN      SOA     ns.mydomain.com      hostmaster.mydomain.com. (
                        2005071005       ; serial number YYYYMMDDNN
                        28800   ; refresh  8 hours
                        7200    ; retry    2 hours
                        864000  ; expire  10 days
                        86400 ) ; min ttl  1 day
                NS      ns.mydomain.com.

                A       127.0.0.1

*               IN      A       127.0.0.1

I have found that using this technique speeds up the time it takes to render a page, enhances privacy, and will also cut down on the amount of traffic consumed by your site. Tis good stuff!

6 thoughts on “Using BIND to reduce ad server content”

  1. Hi Peter,

    Your site has so much cool information, I must have missed the link to the named.conf generator. Since I want to generate a complete named.conf given a base config, I plan to continue using the Perl script to ease my life. Thanks for the awesome work!

    – Ryan

  2. A lot of people miss it for some reason – I’ll have to redesign the site one of these days, it’s too messy to find anything. Anyway, I’m glad you found it useful!

    cheers,

    – Peter

  3. Just set up Privoxy on some host on your network, make it listen on the appropriate IP and port 8080 (or its standard 8118, whichever) and then just use it as a normal HTTP proxy in the web browser.

    Cuts virtually every ad and tons of other nasty webbugs, javascripts, etc etc.

    I have it installed on my workstation and use it just for myself, which is also a possibility (there are win32 binaries with an installer no less, totally simple to take into use.) No need to tinker with esoteric DNS settings when there is software out there that does it much better and is specialized to the task.

    It’s at http://www.privoxy.org and there are binaries for many OS’s and of course source for compilation.

    To further tighten up ad filtering, visit http://www.neilvandyke.org/privoxy-rules/ and add that actions file as well; optional but quite useful.

  4. Awesome docs, btw.

    Here is a tip for something that I implemented. Instead of redirecting traffic to 127.0.0.1, setup a web server that simply hosts ‘Blocked’ as the page. This keeps the web browser from trying multiple times to connect to a non-existent server and makes the pages look a little more clean if there are multiple ads on one site. In my case, ads get redirected to another internal address on a virtual apache server.

    Additionally, this helps my wife who is not computer savy. If she is trying to visit a ‘bad’ site that I have auto-blocked, she will get the standard ‘Blocked’ message instead of a browser error page. She then knows not to bother me about a page that won’t load because the browser appears to be ‘broken’.

Leave a Reply

Your email address will not be published. Required fields are marked *