DNS Block Lists Just Work!

By Gerry Patterson

DNS Block Lists just work. It's something that mail administrators have known for years.

The main threat to the effectiveness of DNS Block Lists are Microsoft Spam Zombies.

Even so, a DNS block list can reduce spam by at least 90%. But it should be implemented correctly.

The Mail Agent Menagerie.

DNS-block lists are one of the least understood technologies around The Internet. Often when I talk to many users, who have quite a good understand of computing, they fail to grasp the concept of a DNS block list or confuse This is due to the experience of most Internet users, who perceive of e-mail as something that occurs only in the inbox of their MUA. The DNS-block list is a technology that only applies to the MTA.

However, many Internet users don't understand the difference between an MTA and an MUA.

To add to the confusion, what are now usally referred to as a DNS-block lists, used to be referred to as "black holes".

This may be the reason that many people confuse DNS-block lists with client-side black lists. It is also the reason for some of the criticisms made about blocking technology. In order to understand DNS-block lists, it is crucial to understand the difference between two important components of the Internet mail system. These are:

Mail User Agent (MUA). This is what most people think of when discussing email. It is a program that allows users to interact with Internet mail. The MUA runs on the client side. For many people it is a program like Mozilla Mail, Thunderbird or Microsoft Outlook. And unfortunately many people think that the MUA is all there is to email.
Mail Transport Agent (MTA). This is the program that dispatches email. It runs on the host side. Traditionally it was the Unix program sendmail, although there are many different variants. Lately, the MTA postfix has gained considerably, thanks largely to the fact that it is the default MTA in many Linux distributions. Postfix is very similar to sendmail. The Microsoft MTA, Exchange Server has gained in popularity over the last ten years, but not nearly as much as Microsoft would like us to believe.

These two components utlilise SMTP (Simple Mail Transport Protocol) to move email around the Internet. SMTP was devised to replace the use of FTP (File Transport Protocol) for this purpose. SMPT remains the backbone of all that has come since.

History of DNSBL -- The Abridged Version.

The first well-known DNS black list, the Real Time Black Hole List (RBL), was developed as part of the Mail Abuse Prevention System (MAPS) in the late nineties. It proved very effective, and was soon incorporated in sendmail as the DNSBL (DNS Block List) feature. At the time sendmail was the fundamental building block of the global SMTP MTA.

The DNSBL feature became an almost instant success.

In its' infancy MAPS appeared to be largely a volunteer organisation. One of the names often associated with MAPS was Paul Vixie. However, it did not have the appearance of a one-man show. It seemed that there were many dedictated and knowledgeable people also working with him on the project.

As the Internet bubble really started to swell, and was just about to pop, MAPS decided to turn commercial. They became a subscription service in 2001. According to a redirection on the old MAPS site this service is now part of the Kelkea Anti-spam Service (see http://www.mail-abuse.com/).

I have not really kept up with what has happened to MAPS since they became a subscription service, however the technology developed during the initial project certainly lives on. Many other free DNS-based block lists have stepped up to fill the rather large gap left by MAPS, when they vacated their pre-eminent position to pursue the pot of gold at the end of the corporate rainbow. And who knows, if they had made their run a little earlier, they would all have become multi-guzzillionaires. Certainly there many less worthy than them who found fortune on the yellow brick road.

The DNSBL feature initially available in sendmail has been adopted by many other MTAs.

There are now many DNS block lists that use the DNSBL feature, initially developed for sendmail. Most of the effective ones are still staffed largely by volunteers. However, this may be changing. There does appear to be need for some professionalisation in the whole arena of mail abuse. Spammers have turned pro, and whether or not mail abuse prevention also becomes fully professional depends on who is willing to pay for the service. The problem of trying to make an effective professional service out of mail abuse prevention is that the Internet has been built on Open Standards that still require a certain degree of co-operation and good will on behalf of the stakeholders (all of us). An effective mail abuse prevention system will require the same inputs and dedication that made the original MAPS a success.

Lately even Microsoft have "discovered" DNS block lists. Recent versions of MS Exchange support DNSBL.

Using DNS Block Lists with sendmail

Setting up an MTA to use DNS-block lists is straight forward. With sendmail this involves changing the sendmail.mc configuration file. The location of this file varies according to the type of operating system and the version of sendmail (Note: For security reasons, it is recommended that older versions of sendmail, especially versions prior to version 8, be upgraded).

For example on a FreeBSD system, if the standard conventions are observed, the configuration file would be called /etc/mail/myhost.mydomain.com.mc (substitute appropriate values for myhost and mydomain). This would have been copied from the blank template (probably called freebsd.mc) when the system was built. And after changes are made to the macro file it would be used to build the configuration file (/etc/mail/myhost.mydomain.com.cf).

These macros are processed with the m4 macro utility.

If you have the standard configuration file there should be a sample line with the DNSBL feature. The default line may still point to the MAPS DNSBL. This will look like:

	dnl FEATURE(dnsbl)

In the past, if you wanted to enable the DNSBL feature, all you had to do was remove the "dnl" word from the start of the line. The word "dnl" is actually a macro that removes everything up to and including the next carriage return. It is often used to insert comment in an m4 file.

Many other DNS block lists can also be accessed with the DNSBL feature. These can be called by inserting the appropriate parameters for other lists. This is outlined in the documentation, and there are some helpful comments in the default configuration file. Look for the sample lines that come after the first DNSBL feature. They may look like this:

	dnl Alternatively, you can provide your own rejection message for the RBL:
	dnl FEATURE(dnsbl, `blackholes.mail-abuse.org', `"550 Mail from " $&{client_addr} " rejected, see http://mail-abuse.org/cgi-bin/lookup?" $&{client_addr}')

	dnl Other DNS based black hole lists
	dnl --------------------------------
	dnl MAPS Relay Spam Stopper (RSS): http://mail-abuse.org/rss/
	dnl NOTE: This is a subscription service as of July 31, 2001
	dnl FEATURE(dnsbl, `relays.mail-abuse.org', `"550 Mail from " $&{client_addr} " rejected; see http://mail-abuse.org/cgi-bin/nph-rss?" $&{client_addr}')

You will need to include the appropriate message for the DNS-block list that you choose for your MTA. For example if you wish to enable the dsbl list and the spamcop list, you would put the following two lines in the m4 macro file:

	FEATURE(`dnsbl', `multihop.dsbl.org', `"550 rejected - see http://dsbl.org/faq-listed"')
	FEATURE(`dnsbl', `bl.spamcop.net', `"550 rejected - see http://www.spamcop.net/reported.shtml"')

Then you should build the configuration file. In most cases it will be necessary to include some flags on the m4 command line.

However, there should be a script or a Makefile in the configuration directory that does this. On my system I simply have to enter "make".

After running the m4 macro, you should be able to find the expanded commands in the configuration file. For example, on my mailhub, the two lines above expand into the following:

	# DNS based IP address spam list list.dsbl.org
	R$*                     $: $&{client_addr}
	R::ffff:$-.$-.$-.$-     $: <?> $(host $4.$3.$2.$1.list.dsbl.org. $: OK $)
	R$-.$-.$-.$-            $: <?> $(host $4.$3.$2.$1.list.dsbl.org. $: OK $)
	R<?>OK                  $: OKSOFAR
	R<?>$+                  $#error $@ 5.7.1 $: "550 rejected - see http://dsbl.org/faq-listed"

	# DNS based IP address spam list bl.spamcop.net
	R$*                     $: $&{client_addr}
	R::ffff:$-.$-.$-.$-     $: <?> $(host $4.$3.$2.$1.bl.spamcop.net. $: OK $)
	R$-.$-.$-.$-            $: <?> $(host $4.$3.$2.$1.bl.spamcop.net. $: OK $)
	R<?>OK                  $: OKSOFAR
	R<?>$+                  $#error $@ 5.7.1 $: "550 rejected - see http://www.spamcop.net/reported.shtml"

Then you must to restart the MTA, to activate the blocklist.

Choosing a DNS Block List

We had to destroy the village in order to save it.

- U.S. Army Officer (quoted by Peter Arnett)

Netiquette is no more than the basic courtesy that should be observed when using the Internet. This is not a new concept. Such courtesy is required when you are writing a letter, making a phone call or conversing with someone in the street, or in a business establishment. Just because computers can carry out some of these functions very quickly and at high volume, it does not exempt individuals from these basic principles of respect for "the other".

An unfortunate consequence of the spam wars has been a general reduction in the observance of "netiquette".

For that reason, some care should be taken to observe netiquette in both the choice of block list, and your implementation. Otherwise you may be "destroying the village in order to save it".

If you are using your block list for the purpose of blocking, you should not silently discard emails. Some major ISPs, who should know better (e.g. Yahoo) seem to be doing this. You should not add to the general deterioration in standards of netiquette, by adopting such a practice.

Also it is important to understand that sending a reject message is not the same as sending an email to the (alleged) offending sender.

This is where one of the major sources of confusion exists in the minds of general users. It is due to the confusion between MTA and MUA. It is important to understand that configuring an MTA to respond with a 550 reject message is quite different from sending an email back to the sender (which you definitely should not do). When your MTA responds with a "550 reject", the message goes to the other MTA (the one trying to send your MTA a message). It is up to the other MTA to decide whether to send an email to the original sender (which in the case of its' own users, it should).

This is why some people (wrongly) accuse blocking technology of flooding the Internet with spurious emails. It does not. It is only when someone mis-configures an MTA to use a content filter and send an email to the (alleged) sender of the email that this problem arises. This is something that a responsible adminstrator should not do.

In fact such mis-configured MTAs are often used by spammers. They get a free-kick, so to speak. All they have to do is send their spam, many thousands of copies - but each with a forged but valid return address, to the MTA in question, which dutifuly sends the email on to the (alleged) return address with a little note saying that this message has not been delivered because it seems to be spam!

My own response to a annoying mis-configured MTAs that do this is to add them to my private block list, usually with the message:

550 Email from mis-configured MTA rejected - send a message via http://http://www.pgts.com.au/page04.php

(perhaps I should put a link to this article)

Fortunately such mis-configured MTAs appear to be becoming rare.

Still, it is important to understand the distinction between sending a message to the other MTA and sending an email to the (alleged) sender. You can never know for sure if the sender is genuine. The MTA reject message however, goes directly to the party on the other end. If it is a spammer, too bad. If it is a genuine MTA, it assists them diagnose the problem.

Also you should include in the message that the block list recommends for your MTA.

And you should choose block lists that allow users (usually administrators) to de-list their own MTA ... Sometimes an MTA might become a "collateral listing".

That allows administrators to take appropriate action, (hopefully) remedy the situation and apply for de-listing.

And genuine a mail administrator will usually attempt to remedy the situation before applying for de-listing, because he knows that the host will be promptly listed again unless the problem is fixed.

A genuine spammer, on the other hand, would not bother applying for de-listing (unless he owns the MTA -- in which case he will soon be identified and denied de-listing).

Spammers usually move on looking for fresh ground to spoil.

Generally you should do the following:

Choose lists that allow MTAs to request de-listing (and that honour genuine requests).
Set up your MTA to respond with a standard 550 reject and the suggested message from the list.
Do not under any circumstances, use the block list (or any list) to generate an email message back to the sender.

Microsoft Spam Zombies

Of course there is the problem of dynamic addresses, and what to do about them. One of the biggest threats to block lists is Microsoft Spam Zombies, which are proliferating at an alarming rate.

For this reason, many adminstrators are adding dynamic (or portable) addresses to block lists. There are some lists that list only portable addresses. The problem with these lists is that as addresses are delegated they can change from portable to non-portable. Maintaining such lists is quite difficult.

Nevertheless, leaving aside the problems of Spam Zombies, the existing DNS-blocking technology, by itself should reduce the spam load by at least 90%.

You can add other measures to reduce the load of the remaining spam. But this will entail some content filtering.

BIBLIOGRAPHY:


Eric Allman	cf/README for sendmail. Straight from the horse's mouth, although it is best to read the documentation that actually ships with your distribution
DNSbl.info	DSNBL List. Very large list of DNSBLs.