blog
MTA developers: allow use of domain DNSBLs at the SMTP level
In this Blog
Jump to
Blocking by domain, rather than IP address, is arguably the most effective strategy to protect against snowshoe and hailstorm spam. However, we often find that users are failing to use domain block lists during the initial SMTP negotiation, before the body of the message is transmitted. Some overlook this aspect completely, meanwhile others have difficulties in configuring some of these checks in their mail systems. Here’s why it’s important to utilize our domain block lists at the SMTP level, along with an issue you may experience in trying to follow this best practice.
Before we get deep into the technical detail let’s take a brief look at both snowshoe & hailstorm spam:
Snowshoe spam
Snowshoe spammers constantly purchase swathes of domains and IP addresses because they burn through them at astounding rates. For their spam to have the highest inbox delivery rate, their messages need to appear as legitimate as possible, by following some of the same rules legitimate senders work by:
- No forgeries
- Reverse DNS in place
- Authenticate the sending IP addresses with Sender Policy Framework (SPF)
- Sign the mails using Domain Keys Identified Mail (DKIM)
- Sent using Transport Layer Security (TLS).
But above all, their connecting IP addresses and domains should not have been classified as ‘spammy’ by various mail filtering and reputation services.
This whiter than white appearance only lasts for a limited amount of time. Before long snowshoe spammers’ IPs and domains typically end up on block lists such as ours. At this point, the snowshoe spammers are forced to move to different IPs and domains.
Hailstorm spam
In hailstorm spam this technique is brought to an extreme: suddenly resources appear out of the blue with tremendously intense spam emissions that last for just a few minutes or even seconds, terminating just before blocking lists come into action.
At the Spamhaus Project we constantly work to reduce the delays between detection and listing deployment; this battle is now played out in seconds, rather than minutes, as was once the case.
Is it better to protect against hailstorm & snowshoe spam with an IP address or by domain name?
The answer is: both should be used, but by domain is usually more effective.
IP addresses are purely numbers and often there is no reason to list an IP until it is observed to emit spam. Conversely, domains are strings of characters, they need to be registered before use, and often it is possible to detect correlations between the different domains of a specific spammer and therefore discover "domain clusters" that can be listed as a unit. This can enable a spam domain to become listed before it has actually been used (something that’s extremely useful if you’re fighting against a hailstorm attack, which is over in the blink of an eye.) An example can be found later in this article.
Furthermore, the registration, administrative and technical costs of handling a new domain make it a significantly more expensive resource than a single IP address. Even if the cost of a domain is just a few dollars, if its useful lifetime in spam is only a few minutes and spammers need thousands of them, costs can quickly escalate. As a result, it’s not unusual for a domain to be associated with multiple IP addresses, so as soon as that single domain becomes listed emissions from several IPs are blocked at the same time.
Considering the above points it’s evident why blocking by domain is probably more effective than blocking by IP address in the fight against this type of spam.
An example: How a domain name may block snowshoe spam when an IP address won’t
Below is a (slightly redacted) example extracted from our inbound spam flow. There is nothing special in this example: it simply evidences the pivotal role played by domains in snowshoe spam, and illustrates how domains can be clustered:
Message details:
Return-Path: <alerts@siteslibrary.com>
Received: from pythagorean.siteslibrary.com (pythagorean.siteslibrary.com [162.244.13.46])
by (redacted) (Postfix) with ESMTPS id (redacted)
for <(redacted)>; Thu, 6 Dec 2018 (redacted) +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siteslibrary.com;
s=dkim; t=(redacted);
bh=(redacted)=;
h=to:from:subject:date:mime-version:message-id:content-type;
b=(redacted)=
To: <(redacted)>
From: "Breaking News" <alerts@siteslibrary.com>
Subject: Putin threatens arms race if U.S. dumps treaty
Date: Thu, 06 Dec 2018 (redacted) -0500
Precedence: bulk
MIME-Version: 1.0
Feedback-ID: (redacted)
List-Unsubscribe: <mailto:(redacted)@siteslibrary.com>, <http://siteslibrary.com/(redacted)/U/>
X-campaignID: (redacted)
Content-type: text/html
X-Original-Message-ID: <(redacted)@siteslibrary.com>
While the subject line refers to a current topic in the news, the message body is full of pharmacy product adverts along with other items.
SMTP details: Let’s now focus on the SMTP aspects of this message that come into play before it is delivered:
- The reverse DNS record is perfectly correct and matching the forward record:
46.13.244.162.in-addr.arpa. 14000 IN PTR pythagorean.siteslibrary.com.
pythagorean.siteslibrary.com. 3600 IN A 162.244.13.46
- The domain used has a valid SPF record, pointing to the IP addresses that actually emit the messages:
"v=spf1 ip4:162.244.13.45 ip4:162.244.13.46 -all"
- The server presented itself (in the HELO/EHLO stage of the SMTP protocol) with the name pythagorean.siteslibrary.com
- The envelope sender address (Return-Path in the headers above) is alerts@siteslibrary.com
- The message is DKIM signed and it validated on receiving.
These five points tell us that all the usual checks performed by mail servers to detect obviously falsified sender informations, or sending systems that are not really mail servers but compromised systems, produce a PASS response. Everything is perfect here; all the domain information is valid. Namely, the sending system is not a compromised system and is certainly operated by the people that are in control of the domain siteslibrary.com.
As previously mentioned, snowshoe spammers work hard to pass all these checks and ensure that their messages are similar to those sent by legit senders, to convince the receiving servers to accept them.
Blocking at the SMTP level
Despite this, if we know in advance that siteslibrary.com is a spammer controlled domain, we have three independent ways to block this message during the initial SMTP negotiation, before contents are actually transmitted:
- Reverse DNS (PTR) checks
- HELO checks
- MAIL FROM (envelope sender) checks
In this particular case, any of these checks would trigger; in other cases, only one or two of them could succeed.
IP details: Now let us delve a little deeper and examine the IP surroundings to this message. A reverse DNS scan of this IP range shows a clear pattern:
162.244.13.35 sinister.healthypupil.com
162.244.13.36 cliquishness.healthypupil.com
162.244.13.37 clock.launchcentro.com
162.244.13.38 interconnectable.launchcentro.com
162.244.13.39 flocculate.vidyabooster.com
162.244.13.40 uncatholicizes.vidyabooster.com
162.244.13.41 regionalistic.gingerlender.com
162.244.13.42 sizing.gingerlender.com
162.244.13.43 ganga.kickoverflow.com
162.244.13.44 taint.kickoverflow.com
162.244.13.45 corella.siteslibrary.com
162.244.13.46 pythagorean.siteslibrary.com
162.244.13.47 chagres.technogroovy.com
162.244.13.48 welsh.technogroovy.com
162.244.13.49 philomela.digitalzoid.com
162.244.13.50 mallet.digitalzoid.com
162.244.13.51 bankable.circlepanda.com
162.244.13.52 condiments.circlepanda.com
162.244.13.53 inhaler.ballchasers.com
162.244.13.54 whamming.ballchasers.com
162.244.13.55 personae.dawt.info
162.244.13.56 blackish.dawt.info
162.244.13.57 frederiksberg.dern.info
162.244.13.58 endogenous.dern.info
162.244.13.59 antigenically.cusk.info
162.244.13.60 enriches.cusk.info
162.244.13.61 isogram.derv.info
162.244.13.62 inception.derv.info
Our spam comes from a server that is part of a family. When this sample was caught the spammer was using just 10 of these IP addresses (35, 36, 45, 46, 47, 48, 49, 50, 53, 54), burning out 5 of his domains. The other domains were obviously ready to be burnt at a different time, as in fact happened.
A ‘whois’ lookup shows that siteslibrary.com was registered less than one month before the spam run:
% whois -h whois.namecheap.com siteslibrary.com
Domain name: siteslibrary.com
Registry Domain ID: 2334321018_DOMAIN_COM-VRSN
Registrar WHOIS Server: whois.namecheap.com
Registrar URL: http://www.namecheap.com
Updated Date: 2018-11-19T16:12:15.00Z
Creation Date: 2018-11-19T16:12:15.00Z
Registrar Registration Expiration Date: 2019-11-19T16:12:15.00Z
Registrar: NAMECHEAP INC
Registrar IANA ID: 1068
Registrar Abuse Contact Email: abuse@namecheap.com
Registrar Abuse Contact Phone: +1.6613102107
Reseller: NAMECHEAP INC
Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited
What’s really interesting to note is that all the domains listed above have been registered at Namecheap in less than one minute for each of the two domain groups (.com and .info):
Domain Creation Date
healthypupil.com 2018-11-19T16:12:50.00Z
launchcentro.com 2018-11-19T16:12:29.00Z
vidyabooster.com 2018-11-19T16:12:25.00Z
gingerlender.com 2018-11-19T16:12:24.00Z
kickoverflow.com 2018-11-19T16:12:16.00Z
siteslibrary.com 2018-11-19T16:12:15.00Z
technogroovy.com 2018-11-19T16:12:07.00Z
digitalzoid.com 2018-11-19T16:12:06.00Z
circlepanda.com 2018-11-19T16:11:57.00Z
ballchasers.com 2018-11-19T16:11:47.00Z
dawt.info 2018-11-08T15:00:55.00Z
dern.info 2018-11-08T15:00:55.00Z
cusk.info 2018-11-08T15:00:47.00Z
derv.info 2018-11-08T15:00:44.00Z
Blocking a domain before it’s been used: On the basis of this data, we can safely conclude that these 14 domains are part of the same family, belonging to the same actor. They can be blocked as a preventive measure, despite the fact that some may not have been used yet.
We routinely do this kind of analysis, involving varying degrees of automation. This means that these spam domains may be included in the DBL before they’ve even been used. As a result, our users have a chance to block these senders even if the sending IP address is not listed on any block list.
Remember; there is often a good reason for that IP address not to be listed, namely it may have started emitting spam only a handful of seconds before it made the connection with your mail server. Many spammers use each IP address for a very short time and open a tremendous number of simultaneous SMTP connections towards their targets to hit them at approximately the same time.
Having explained why there’s a significant benefit to block using a domain block list at the SMTP level, here’s the challenge we’re facing:
Some MTAs do not allow use of domain DNSBLs at the SMTP level
In theory, the three DNSBL-based domain checks that can be carried out during the initial SMTP negotiation -- HELO, MAIL FROM and rDNS -- should not be too difficult to implement in Mail Transfer Agents (MTAs). Particularly when compared to the code required to extract URLs from message bodies. Development teams should be able to implement this feature with little effort.
Unfortunately, despite various domain DNSBLs being available in the market place for years, including the Spamhaus DBL which was first available in March 2010, a large number of MTAs still do not allow these checks at the SMTP level, they only support IP-based block lists.
Some known exceptions: Besides some carrier grade MTAs that have had this functionality for a long time, Postfix and Exim spring to mind as exceptions. Therefore users of these MTAs get better protection against snowshoe and hailstorm spam. Having said this, add-on control panels, supposed to make life easier for users, such as the well known Plesk, often do not allow for this functionality even when the underlying MTA supports it.
A plea to developers of MTAs & MTA user interfaces: Please make it possible to for your users to deploy domain-based block lists before the SMTP DATA stage, checking the HELO string, the MAIL FROM domain and the reverse DNS hostname (when available).
Please put this issue on your agenda before the end of the year! Thanks, and have a great summer!