|"Role Accounts" & "Feedback Loops"
Spam reports are an important mechanism to detect abuse occurring on or from your network. Reporting spam is a long and venerable tradition on the internet. Most ISPs consider well-crafted spam reports to be a favor to them; they want to know about the problem so they can fix it, before it becomes worse or listed in SBL. There are even specific abuse role accounts required by the RFC documents which are the blueprints of the internet.
A role account is an e-mail address which serves a particular function, not an individual person, for example "sales@" or "info@". Postmaster@domain (RFC2821) and abuse@domain (RFC2142) are the two role accounts which every ISP, webhost, mail service and DNS host must have in order to promptly identify spam and abuse related problems on their network. Networks which do not tolerate spammers are careful to look at their abuse mail every day and take effective actions to stop any problems which arise.
A Feedback Loop (FBL) is an automated stream of spam reports sent by prior agreement between individual receiving and sending networks, often based on a "This Is Spam" button in the user interface. FBLs are intended to help streamline and automate the spam reporting process with specific machine-readable parts. A standard Abuse Reporting Format (ARF) is specified and implemented for FBLs. ARF follows existing RFC2045 MIME standards for e-mail (and the earlier RFC1341). More about ARF is here and here.
A single spam report could be a fluke or someone reporting mail they actually signed up for, or it could represent 10,000 or more spam recipients. For wide-scale, pure-spam mailshots, the reporting rate is often even less than one in 10,000 due to filtering and "LART fatigue" of many who used to report spam. In general, hand-crafted reports are more likely to be actual spam than reports from automated http://tinyurl.com/kda37"This Is Spam" (TIS) buttons. Even TIS reports average in excess of 80% spam, though. Evaluate reports as you will; ignore them at the risk of your network's reputation and e-mail deliverability. (Ignoring them includes setting barriers to accepting them, such as imposing requirements like ARF, which is not supported by end-user mail clients, or DKIM or SPF.)
Here are some tools which can help direct spam reports to your proper role account. WARNING: FBLs can produce very high volumes of e-mail. Use a specifically designated e-mail account and allow plenty of disk space and server cycles to accept the full stream of reports. Some networks use a seperate server to accept the flow (for example, email@example.com). Do not apply content-based spam filters to FBL or abuse@ accounts or you will discard the very messages you need to keep your network clean.
1. The Network Abuse Clearinghouse
Not a reporting service or FBL per se, but a database of correct addresses for spam reports based on domain. Registered users can send spam reports via its mail server, or anyone can query it to find reporting addresses for a particular domain. Since reports are not automated, volumes tend to be lower and reports hand-crafted. This may identify some of the more stubborn spam issues on a network, for example HTTP redirectors or DNS. See the Abuse.net website for instructions on using and updating its services.
SpamCop reports the spam source, SMTP relay and spamvertised URLs in the message body based on IP. Registered users can use SpamCop to parse and report spam. A SpamCop feed might be high volume depending on your network size and output. The instructions on this page explain how anyone can receive SpamCop daily or hourly summaries about spam problems in specified IP ranges. For more information see the SpamCop FAQ for abuse-desks and administrators.
3. AOL Feedback Loop: http://postmaster.aol.com/tools/fbl.html
When AOL users click the "This Is Spam" button in their e-mail client, this system generates a "SCOMP" report to you. While it can be high-volume, it offers excellent feedback on proxies, virus infections and spammers on your network. You need to sign up for this free service with AOL. (And same with the MSN, Yahoo! and Outblaze systems, too.) AOL's whitelist info is here.
4. Outblaze (mail.com): Request a feedback loop by contacting firstname.lastname@example.org. ISPs and COI bulk mailers only!
5. Microsoft (msn.com, live.com, hotmail.com) has Feedback Loops and other information for bulk mailers at http://postmaster.msn.com/. Their Smart Network Data Services includes delivery numbers and any Spamhaus Zen listings within your IP ranges at SNDS, "Junk Mail Reporting Program" FBL at JMRPP and other services. Senders may also be interested in this PDF to help their delivery.
6. United Online Trusted List and Feedback Loop (Netzero and Juno): http://www.unitedonline.net/postmaster/whitelisted.html
7. Road Runner FBL: http://feedback.postmaster.rr.com/
Road Runner's postmaster page: http://postmaster.rr.com/
8. Yahoo! FBL: http://feedbackloop.yahoo.net/
Deliverability information: http://postmaster.yahoo.com/
9. "USA.net offers a feedback loop service, operated by Return Path, free of charge, to parties sending large amounts of mail to USA.net members. The feedback loop (FBL) will forward any mail reported as spam originating from the associated IP addresses back to the listed email address. We highly recommend the use of a dedicated e-mail address for this purpose." (Spamhaus: good advice for any FBL!) And, of course, http://postmaster.usa.net/.
10. Comcast Feedback Loop: http://feedback.comcast.net/
Postmaster pages for more info: http://postmaster.comcast.net/
11. Earthlink Feedback Loop: Write to email@example.com with your IP range, domains, your network's contact information including name, contact e-mail and phone, and the e-mail to which the FBL will be sent. ISPs only. [May 2009: reported to be unresponsive; status unknown.]
12. Excite Feedback Loop: http://feedback.bluetie.com/
13. Cox.net Feedback Loop: http://fbl.cox.net/ and Postmaster pages.
14. Mailtrust Feedback Loop: http://fbl.mailtrust.com/ (Rackspace.com)
15. Tucows (OpenSRS) Feedback Loop: http://fbl.hostedemail.com/
16. Synacor Feedback Loop: http://fbl.synacor.com/ Support is at firstname.lastname@example.org.
Spamhaus is happy to update this information at the request of the FBL provider or other authority. Other sites have additional information, and there is a list on Wikipedia. Also, private FBLs can be established between consenting parties to acquire additional evidence of spam, either free or for a price; for example see Abusix.com. Ask your network friends and geek buddies to filter spam from your IP ranges into a forwarding account for you, in exchange for you feeding them data about their ranges.
Tip: Do not put your FBL or abuse@ account behind content-based spam filters. What's the point of filtering away the evidence you need to clean up your network?
Finally, be sure that your network has an enforcible Acceptable Use Policy (AUP) as a part of your contract with each and every customer! More examples are here. Seek legal counsel to ensure that your AUP will cover you in the event of account termination for abuse.
|Identify your IPs so other networks can reach you!
Problems which aren't identified don't get fixed.
Provide roper role accounts in RIR records, including abuse role accounts. Be sure such accounts are read frequently by admins with the authority to fix the problems identified by reports to that address.
Properly identify subnet clients. For example, ARIN requires public identification records for /29 (8 IPs) and larger ranges (section 188.8.131.52.2.).
This FAQ also has information about proper rDNS as well as the role accounts to go with those hostnames.
|We get a lot of abuse mail, how can we handle it?
There are two queue-management tools designed specifically with the needs of an abuse desk in mind. They both come highly recommended and are both used in production environments at large ISP/NSP abuse departments:
Abacus commercial ticketing/tracking system
RT Incident Response open-source ticketing tool
Abuse Pipe Commercial ticketing system.
Also, procmail can be extremely useful for sorting an inbound mail stream. For example, you could flag and sort any mail which had any of your IPs in it. You could bundle it into /24 chunks (or whatever works for you) and triage based on relative volumes. Spammers can even turn themselves in, that way.
Filtering an abuse@ mailbox can be tricky because much of the mail which it should receive looks like spam. Filter out spam reports and you risk not identifying a problem on your network. Spamresource.com offers some thoughts on how to deal with all the spam aimed at the abuse box.
|But we're not seeing any spam reports!?
There are any number of reasons you didn't see spam reports before your IPs were blocked by Spamhaus or other lists or networks. Among them:
You're not signed up for sufficient Feedback Loops. (scroll up for FBLs!)
Your abuse@ account is filtered so spam reports don't get through. It goes over quota, has insufficient retention capacity, inadequate search tools, isn't read frequently or completely, or is otherwise dysfunctional.
Filters are catching spam from your IPs before it hits reporting addresses.
People, users and ISP admins, are tired of reporting spam and simply block it as soon as it's detected, no questions or comments. Sometimes called "LART fatigue".
The use of "disposable e-mail addresses" means those who use them aren't seeing spam in their primary accounts, so it doesn't get reported.
Spammers have improved their listwashing methods to eliminate spam reporting addresses. ISPs aid that process by passing reports along to spammers, but spammers have many other means to wash away the symptoms of their ill-gotten lists while not fixing the basic problem (i.e., no opt in).
|What can we do about accounts which are not breaking the law? (AUP)
Laws regarding the Internet vary by jurisdiction; different countries each have their own laws which cover some abuses, but possibly not all. Besides, police or court intervention is often too slow to be effective at stopping Internet abuse. To effectively stop all abuse in a timely manner, each ISP must enforce a set of rules on all of their customers and users, often called an Acceptable Use
Policy (AUP). Those policies should be written into contracts with every customer of the ISP and apply to the customers-of-customers and all other users of the ISP's network, too. ISPs which want to maintain good relationships with other networks must enforce AUPs which prohibit abuse against those other networks.
Spamhaus is not a law enforcement agency nor does it claim to know all the laws in every country. Instead, we build our anti-spam lists based on our own published policies, policies which all of our users--including very large networks--agree with. Again, any network which wants to exchange e-mail (or other traffic) with Spamhaus users
must enforce policies prohibiting abuse by all users of their network.
If you don't have an AUP yet you can use our free AUP builder to create one. Be sure to have your company lawyer adjust it so that you can effectively enforce the policies you need for your network's traffic to be acceptable to other networks.
|How do I connect with other "abuse desks" in the industry?
The Messaging Anti-Abuse Working Group (MAAWG) is the first step in an effort to align the messaging industry stakeholders along three directives: Collaboration, Technology and Policy. MAAWG provides an excellent opportunity for social networking with other responsible senders and receivers. Another touchstone is the Coalition Against Unsolicited Commercial Email (CAUCE) and its international affiliates. Two open-subscription, discussion-format mailing lists have quite a few abuse desk staff on them, Spam-L and NANOG.
|What can we do to prevent fraudulent signups?
Detecting and preventing fraudulent signups for ISP services can be a complex task. In October 2012 we published a lengthy blogpost with many tips and tricks to deal with this issue: How hosting providers can battle fraudulent sign-ups.
|What can we as an ISP do to prevent spam through compromised accounts?
User account compromises are a growing problem, especially with malware targetting specific services and providers for stealing of user login/password combinations. More information on this subject can be found in our blog article Spam through compromised passwords, can it be stopped?.
|Dynamic IP lists & port 25 blocking - strongly suggested!
Dynamic IP ranges--dialup, cable, and DSL--are a huge source of spam. The users of those ranges are most vulnerable to attacks by viruses and Trojan Horse proxy software installed by virus infections which turn their computers into botnet zombie spam spewers. Dynamic IP users on your network should send their mail out through your network's outbound customer servers, or through other dedicated outbound "smarthosts" accessible via SMTP AUTH (authenticated) on port 587:
The best way to stop the massive amounts of dynamic IP spam is for your network to filter port 25 on dynamic ranges, only allowing port 25 access to your smarthosts (or not even that, if your smarthosts use AUTH). The Messaging Anti-Abuse Working Group (MAAWG) has its recommendations on port 25 blocking here. If you run a NAT gateway, see http://cbl.abuseat.org/nat.html
But even if you cannot block port 25, most other networks would rather not accept any mail from dynamic ranges. To assist them with that, it is polite for your network to list your dynamic, end-user, IP ranges in the Spamhaus PBL.
Besides being a good Internet neighbor and helping other networks avoid spam from your users, listing your dynamic addresses in those lists makes those ranges less attractive to spammers because they know they can't deliver to many networks which use those lists, and you'll get proportionally fewer spam reports.
Please add your dynamic IP ranges to those lists!
|How do users send mail if Port 25 is blocked? (SMTP AUTH)
Users of dynamic ranges with blocked port 25 can connect with SMTP AUTH "SUBMISSION" protocol: ftp://ftp.rfc-editor.org/in-notes/rfc2554.txt. We have more instructions for several other mail programs in the Generic Questions FAQ.
What is needed:
The ISP configures its server to accept SMTP AUTH connections on port 587; all modern mail servers support it. Don't block dynamic users from port 587, and don't use PBL or Zen on 587 connections! Don't block your own users!
Each user configures his or her mail client (MUA) to connect via SMTP AUTH on port 587, with username/password. All current MUAs have that as an option; use the mail client's "help" menu.
In Outlook and Outlook Express, the menus for that are something like this:
- Internet E-mail Settings
- - Outgoing Server
- - - [x] My outgoing server (SMTP) requires authentication
- - Server Port Number
- - - Outgoing server (SMTP): 
- - - [x] This server requires an encrypted connectionThat way, only authorized users can connect to your mail server, you know exactly what account it is, and they bypass "port 25 blocking". Users with "SMTP AUTH" can then send mail from their computer via your server from anywhere they can connect, so it is very good for travelers like business people and students.
You should also consider filtering outbound spam on your mail server, possibly with something like http://spamassassin.apache.org/.
|What is "proxy hijacking"? What do I need to know about proxies?
A proxy is any computer which can perform a function as a surrogate for another computer, under control of a remote operator. When an insecure proxy is used by spammers to transmit spam, that proxy computer is said to be "hijacked." The most common proxies seen in spam are the virus-installed trojans which actually transmit the spam via SMTP (they usually receive their commands on other ports). Other proxies used in spamming include web-cache and promiscuous DNS proxies, explained in this presentation in your choice of .ppt or .pdf formats. Currently, the most popular proxy protocols used by these virus-installed trojans are SOCKS4, SOCKS5 and HTTP-connect, but they may operate on any port. Increasingly, computers controlled by these virus-installed trojans -- "zombies" -- are parts of larger botnets controlled by criminal spam gangs and other Internet criminals.
ISP admins need to be able to identify proxies, secure them and block unauthorized access to them, and remove the software if necessary. Many trojan proxies do not show up with routine port scans, either operating on obscure high-number ports or using evasive mechanisms to avoid detection. Their filenames are not consistent. It may be necessary to do complete forensics, including packet sniffing and system monitoring, to identify the malware.
For many end-user systems, it is simply more effective to wipe the hard disk and reinstall a fresh system, or to pay a professional to clean the system. The "SecCheck" tool at MyNetWatchman.com is an excellent starting point, highly recommended, easy to use and free!
|What is a "honeypot" or "proxypot"? What is a "proxy hijack source" or "C&C"?
[hijack source/C&C]---(SOCKS)--->[zombie PC]---(SMTP)--->[spam rcpt]Even more important than securing proxies is locating the proxy hijack source - the command and control (C&C) server for a botnet composed of many proxies (tens of thousands). These C&C servers are the actual originating source servers which send spam out through compromised proxy systems. The proxies hide the originating IP address from showing up in spam headers, of course, so you probably will not see spam reports, but special honeypots called "proxypots" identify the server motherships. Proxypots mimic a zombie PC, but unlike the zombie, they record the spammer's source IP. Spamhaus uses proxypots -- and other techniques -- to identify those C&C or proxy hijack servers.
To confirm those detections, as well as to monitor your own network routinely, every ISP should have the tools and skills to do traffic flow analysis. When you look at traffic flowing from a C&C server, you will see many connections on high-number ports to many destination IP address around the 'net - thousands and thousands of connections on unusual high-number ports! The destinations are almost always compromised proxies on end-user broadband networks. Be prepared with a good network monitoring tool to check traffic flows from suspected hijack servers. Some of the tools we've heard of include Tippingpoint, Fortinet, Sandvine, Packeteer, Allot NetEnforcer, Cisco's P-cube, and flow-tools for collecting and processing netflow data from Cisco and Juniper routers. Also see Stager for sorting and presenting data collected by those tools.
Network operators can also subscribe to a daily report of reported C&Cs on their network. ISOTF's "Drone Armies" research team posted a summary of their work on the NANOG mailing list and invited interested admins to contact them at email@example.com. Those reports show IP, time, domain, and port.
Spamhaus hears two common excuses given by proxy spammers to trick abuse admins. One excuse is a cover-up for that unusual traffic flow pattern, "We're doing VOIP." A closer look at the packets will disprove that. The other common exuse is "we removed the virus from the computer." Since it wasn't a virus causing the problem, that obviously doesn't stop the spam. Remember, these are dedicated servers with large lists feeding into them. And also remember, if it was a virus problem, you'd see that IP in spam headers. *wink*
|Should an ISP use the XBL to block their own users since it means they have a virus or open proxy?
We're getting a lot of reports of spurious blocking caused by sites using the Spamhaus XBL to block authenticated access to smarthosts / outgoing email servers. The XBL is only designed to be used on incoming email, i.e. on the hosts that your MX records point to.
If you use the same hosts for incoming email and smarthosting / outgoing email, then you should always ensure that you exempt authenticated clients from XBL checks, just as you would for dynamic/dialup blocklists.
As your users are often on dynamic IP addresses, a user may be assigned an IP address from his provider that is in the XBL due to the virus or open proxy situation of a previous user of that IP address.
Another way of putting this is: "Do not use the XBL to block your own users".
Using the XBL to alert an ISP's security department when a user's IP is in the XBL is permissible and a good thing. But remember, that user may not be the one with the virus or open proxy problem.
Note: This also applies to using the XBL to deny access to web-forums, journals or blogs.
|Should an ISP use the PBL to block their own users?
No! Spurious blocking caused by sites using the Spamhaus PBL to block authenticated access to smarthosts or outgoing email servers is very bad. The PBL is only designed to be used on incoming email, i.e. on the hosts that your MX records point to.
If you use the same hosts for incoming email and smarthosting or outgoing email, then you should always ensure that you exempt authenticated clients from PBL checks. As your users are often on dynamic IP addresses, a user may be assigned an IP address from his provider that is in the PBL.
Another way of putting this is: "Do not use the PBL to block your own users".
Note: This also applies to using the PBL to deny access to web-forums, journals or blogs.
|Backscatter from spam firewalls and anti-virus systems
So-called "spam firewalls," software running in front of production servers to process out spam and viruses, can be a problem for other networks if they simply deflect the spam on to other mailboxes. Most spam and all mail-borne viruses use a forged Sender address, and bouncing it back to that address only results in sending unwanted and burdensome mail to innocent third parties. Either reject the SMTP connection with a 5xy message, silently discard it (your firewall identified it as spam, remember?), or file it in a quarantine area for *your* users to glean. Don't make it someone else's responsibility when they are almost certainly not involved.
On the Barracuda Spam Firewall, the option to turn spam bouncing off can be found in the Basic Tab under Spam Scoring. Near the bottom there is a check box for "Send Bounce." This is checked by default and should be unchecked. Instructions with screenshots are shown at http://postmaster.gtcs.com/CudaFix.php. Barracuda 300 may not have this option, but the 400 and 600 versions do have it. Barracuda Networks themselves have now published a document (pdf) on how to shut of this type of bouncing.
When using the amavisd-new content scanner, the configuration file amavisd.conf should contain:
$final_virus_destiny = D_DISCARD; (or D_REJECT)
$final_banned_destiny = D_DISCARD; (or D_REJECT)
$final_spam_destiny = D_DISCARD; (or D_REJECT)
Avoid the D_BOUNCE setting which is the one that causes problems. Note that the D_REJECT seeting has the same effect as a D_BOUNCE in a post queue filter. Thus D_REJECT should be used only for a pre-queue filter.
Note from the above example that the same principle applies to anti-virus notifications and bounces as well. If it's a virus, and it's after the year 2003, 99.9% of the time the return path will be bogus. DO NOT SEND IT THERE!
Other references on spam and virus backscatter:
|My mail server is listed on the SBL as a phish spam source, what should I do ?
Phish spam is sent by criminals gangs usually based in Eastern Europe. They hijack mail servers to get good delivery rates, usually forging the sender address and constantly changing the IP of origin.
If you have this problem, you can not solve it by blocking specific IP addresses or email senders, or in general using filtering/blocking rules, antispam appliances, etc. The spammers exploited a security hole: it should be impossible for anyone external to your organization to use your server to send mail to arbitrary destinations. Your server should accept inbound mail or outbound mail, but it should reject at the SMTP level by default all mail attempting to "pass through".
So, to solve the problem you have to identify and fix this security hole. This should not be too difficult, because the spam was sent by your mail server and therefore evidences and traces have been left in the mail logs. You have to analyze the logs to figure out what the spammers did. If your mail server is Microsoft Exchange, see the FAQ entry below.
In most cases no malware or viruses are involved. Just checking your server and clients for malware is generally insufficient, usually not relevant and does not solve the problem. Obviously, blocking the forged sender address or the IP of origin also does not solve the problem, as those parameters are constantly changed.
These are the most common mechanisms exploited, approximately ordered by number of occurrences, and their typical resolution path:
Do not forget to check the mail queue, and delete all the spam samples that may still be there, waiting to be delivered. Only after all this has been done, the SBL team can process a removal request for a listed phish source.
- (by far the most common) the spam was sent as authenticated mail, injected from an external IP using a valid username and password. Find the involved account from the mail logs and contact its owner.
If for any reason you are unsure about the account involved, then change all passwords, including those of much-abused role accounts such as admin or info. test is also heavily abused.
- if the password was extremely easy to guess, just change it.
- if the password was difficult to guess, it was probably stolen by a keylogger trojan running on a client PC or laptop. Disinfect all the computers used by that person, then change the password.
The logs should always be checked to verify this point, before the following rare possibilities 2, 3, and 4 are even considered.
- the spam was not authenticated, but it came from a PC or laptop internal to the LAN, as shown by the local IP address in the logs: clean that computer, as there is malware on it.
- the spam was not authenticated and it came from a web server in your LAN. In this case, the spammers injected the message via HTTP using a security hole on the web server. Check the web logs for a confirmation, and close the hole.
- the spam was not authenticated and it was injected via SMTP from outside: in this case, you have an "open relay" configuration. Fix it.
For a general perspective about this problem, you can also read this article.
|My Exchange server is sending spam, but it is not an open relay! What should I do?
(See also the previous FAQ entry).
Most likely, the spammers are sending mails by authenticating on your server as an ordinary user. They obtained a valid account/password pair either by guessing (because one of your passwords was simple and so it was discovered through a brute force attack), or through a keylogger program running on a trojaned PC or laptop of somebody using your server to send his/her mail.
Investigate the logs to find the name of the abused account. Follow the indications in the Microsoft Support Article 895853 How to troubleshoot mail relay issues in Exchange Server 2003 and in Exchange 2000 Server, in particular in the section If mail relay occurs from an account on an Exchange computer that is not configured as an open mail relay.
Then, first of all, all the PCs and laptops used by the owner of that account should be checked for malware, as it is possible that the access credentials were stolen by a trojan program running on a client. This is likely if the password was not a very simple one (a very simple password was probably guessed).
Then, the password of that account should be changed to a new and secure one. After that, keep the logs monitored for a few days to make sure that the abusers can not access the server any more.
Another useful reference from Microsoft is Article 324958 How to block open SMTP relaying and clean up Exchange Server SMTP queues in Windows Small Business Server.
|How can we keep 419ers off our webmail?
It's difficult to keep the dedicated 419 scam spammers off of an ISP's webmail, but there are some techniques to make it much harder for them.
Add some systems to minimize the ability to easily and quickly sign up for multiple webmail accounts. Limit new accounts from the same IP address in a small time period. Use CAPTCHA tests. Watch for patterns of abusive usernames (user1, user1, user3, A_user, B_user, etc.) and remove all names within that pattern as soon as detected.
Block IP ranges as needed, often quite aggressively. Afrinic ranges are frequently the source of 419 connections. 419ers will log in from the same Afrinic ranges, or sometimes even RIPE ranges, until they are blocked.
Rate-limit RCPT_TO (419ers load up To, CC and BCC fields...what legit webmail needs more than a dozen or so RCPTs?).
Dump mail with more than 10 lines in the .sig (used by 419ers to avoid other field limitations).
Keep up with abuse@ reports and adapt *quickly* to new attack vectors as they are reported.
Apply outbound filtering with properly adjusted tools such as SpamAssassin. ("Properly adjusted" means you probably need to optimize the parameters for your application.)
|URL shorteners and redirectors
Don't redirect to domains on DBL & SURBL.
Don't redirect to domains with the A RR on SBL and possibly XBL (your call).
Check those blocklists at the time of URL creation and again, later, as traffic on them ramps up (a day or a week).
Don't daisychain redirectors! (Includes both 'don't shorten shorteners' and 'don't accept referrals from shorteners.' Either side can cut the chain.) We are adamant about that.
Don't allow users to change the landing URL.
Fully suspend (404) the offending URL; don't provide an interstitial link on to the spammer's payload.
Role Accounts & Feedback Loops (FBL) - Process those promptly.
|What is "fast flux" hosting?
Fast flux domain hosting involves the use of botnet zombie drones on broadband IPs infected to act as reverse proxies for the spammer's website or nameservers. The spamvertised domain's A-record is pointed at a rapidly changing series of zombie IPs (hence the name) with very short "TTL" values, usually less than five minutes (300s). When both the A and NS RRs use that type of bot hosting it is referred to as double flux. There are typically four or five A records to distribute the load and increase the odds of the website staying up in case a bot crashes or is taken offline. The proxy action hides the IP location of the spammer's dedicated servers. As the very act of hijacking computers is illegal in most jurisdictions, such fast flux hosting is mostly used for further criminal activities such as phishing, malware and child exploitation. These criminals know they could be identified if they used valid whois info so they always use bogus data, so registrars can confidently HOLD (suspend) the domain based on ICANN 184.108.40.206.
13 July 2007: The Honeypot Project has a very informative paper about fast flux networks at http://www.honeynet.org/papers/ff/
The Mannheim Formula paper of 2007 provides fast flux diagnostic formulas:
January 2008: ICANN's Security and Stability Advisory Committee has published an advisory on Fast Flux hosting at sac025.pdf.
Here is an example of the DNS of a fast flux domain (double flux) that was used in child exploitation:
$ dig @NS2.WESTNS.COM wildcard.malaga-53.com a
; <<>> DiG 9.2.4 <<>> @NS2.WESTNS.COM wildcard.malaga-53.com a
; (1 server found)
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13728
;; flags: qr aa rd; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;wildcard.malaga-53.com. IN A
;; ANSWER SECTION:
wildcard.malaga-53.com. 180 IN A 220.127.116.11
wildcard.malaga-53.com. 180 IN A 18.104.22.168
wildcard.malaga-53.com. 180 IN A 22.214.171.124
wildcard.malaga-53.com. 180 IN A 126.96.36.199
wildcard.malaga-53.com. 180 IN A 188.8.131.52
;; Query time: 145 msec
;; SERVER: 184.108.40.206#53(220.127.116.11)
;; WHEN: Sun Sep 3 17:xx:xx 2006
;; MSG SIZE rcvd: 120Here are the hostnames of those A-record addresses. Note that they are all dynamic broadband names typical of botnet zombies:
[18.104.22.168] c-67-190-128-40.hsd1.co.comcast.net.And finally, here are the A-records for the NS hosts, again on botnet zombies:
$ dig @ns2.WESTNS.COM ns2.westns.com a
; <<>> DiG 9.2.4 <<>> @ns2.WESTNS.COM ns2.westns.com a
; (1 server found)
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 53259
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 5, ADDITIONAL: 4
;; QUESTION SECTION:
;ns2.westns.com. IN A
;; ANSWER SECTION:
ns2.westns.com. 180 IN A 22.214.171.124
;; AUTHORITY SECTION:
westns.com. 180 IN NS ns2.westns.com.
westns.com. 180 IN NS ns4.westns.com.
westns.com. 180 IN NS ns3.westns.com.
westns.com. 180 IN NS ns5.westns.com.
westns.com. 180 IN NS ns1.westns.com.
;; ADDITIONAL SECTION:
ns4.westns.com. 180 IN A 126.96.36.199
ns3.westns.com. 180 IN A 188.8.131.52
ns5.westns.com. 180 IN A 184.108.40.206
ns1.westns.com. 180 IN A 220.127.116.11
;; Query time: 2359 msec
;; SERVER: 18.104.22.168#53(22.214.171.124)
;; WHEN: Sun Sep 3 18:xx:xx 2006
;; MSG SIZE rcvd: 198
|What's this ARP (Address Resolution Protocol) daemon spoofing?
It's a tricky way for a spammer to make their spam appear to come from a different IP than their dedicated server, but still on the same VLAN subnet.
To avoid the problem, design your VLAN so that:
- customers can't snoop on each other, even at broadcast level such as arp traffic;
- no customer can send out packets with any source IP other than their assigned one;
- no customer can generate ARP traffic for any IPs other than their own.
As for identifying what server it really is, can you put a port monitor on the switch port of your router and sniff its packets? You should be able to see the MAC address of anything returned from those IP addresses, even if they were injecting forged packets from somewhere else on the Internet. And you should be able to see the gratuitous ARPs.
This technique is in use in the wild. If you detect this on your network, Spamhaus would very much appreciate any details you can provide, including connection information for the spammer's "home base."
|Why should I worry about reverse DNS (rDNS)?
Reverse DNS (rDNS) consists of mapping IP addresses into hostnames. While most Internet applications do not require rDNS to work, there are several reasons why defining rDNS in your network is highly desirable:
- publishing rDNS allows people to associate quickly and easily your IP address with your domain name, and therefore it makes easier to report abuse from your IP address space to your abuse desk
- publishing information about dynamically assigned IPs greatly helps other networks to distinguish the nature of different mail sources in your network (mail servers vs infected end user machines), and to block SMTP connections from dynamic addresses (if they wish to do so) without guessing and risking to inadvertently block mail servers
- publishing proper rDNS for statically assigned IPs - and in particular for those corresponding to proper mail servers - greatly reduces the likelihood that other networks will block those servers by mistake, thinking that their IP belongs to residential dynamic space
- moreover, some networks are refusing mail from IP addresses without rDNS defined
- on the client side, there are security benefits in running applications such as ssh from an IP with rDNS assigned
|How do I get rDNS working?
You need to get a delegation of the reverse DNS zone, and then configure the zone. If your network is, say, 126.96.36.199/16,
"getting a delegation" means having a NS record for 2.1.in-addr.arpa pointing to your nameservers, while "configuring the zone" means inserting the right information in the corresponding zone file in your nameservers.
Getting a delegation
You get a rDNS delegation from the same entity that gave you the IP addresses: it may be the ISP you are taking connectivity from, or the regional Internet registry of your area.
Regional Internet Registries (RIR) have formal procedures for assigning rDNS delegations: follow the links pertinent to your registry.
Configure the zone
For general information about defining a reverse zone in your nameservers, see for instance
the APNIC guide or the RIPE pages linked above.
Note that you do not need to insert all the individual records by hand. If you are using Bind, once you have defined a naming convention for a portion of your space you can use the powerful $GENERATE directive (described in the Bind 9 manual) to define several records with a single line.
For instance, you can write in the 2.1.in-addr.arpa zone file:
$GENERATE 1-254 $.3 PTR adsl-1-2-3-$.dynamic.yourdomain.net.
which is equivalent to 254 lines
188.8.131.52.in-addr.arpa. PTR adsl-1-2-3-1.dynamic.yourdomain.net.
254.3.2.1.in-addr.arpa. PTR adsl-1-2-3-254.dynamic.yourdomain.net.
Do not forget to check that matching forward A records are defined in the zone of yourdomain.net. This is very important because it "validates" the rDNS information. You can define them with another $GENERATE directive, like
$GENERATE 1-254 adsl-1-2-3-$ A 1.2.3.$
which is equivalent to 254 lines
adsl-1-2-3-1.dynamic.yourdomain.net. A 184.108.40.206
adsl-1-2-3-254.dynamic.yourdomain.net. A 220.127.116.11
|What rDNS naming convention should I use?
Make it easier for other networks to identify your dynamic ranges, and thus to filter more appropriately, by using a fixed string like dynamic or dialup as a subdomain immediately below your main domain (also called "right anchored"). Put the geographical information (if it applies to your case) at the next level, and the most variable information (like the last octet of the IP) at the outer, left-most level. Use dots as separators rather than dashes or other symbols so as to create subdomains, and put the most variable information on the left-most part of the name. For instance, this is a good naming scheme:
Clearly the same information could be made available with another scheme like
dadsl7-123-tktu.example.com, but other people would have to define complex regular expressions to parse that, wasting resources and increasing the likelihood of making mistakes.
We also recommend to use similar conventions to identify static IPs and business connections as clearly as possible. For instance, you can use something like:
Such a scheme helps your business customers to maintain good SMTP deliverability to other networks even when neighboring dynamic IP ranges are infected with bots and send spam. Assigning custom rDNS to properly identify mail servers of responsible customers is also a good practice, for example:
|My customer says it's not them; is that true or an excuse?
Some of the excuses commonly used by spammers and signs they are spammers:
- it was a customer of a customer or a reseller;
- it was affiliates (fake and real);
- the server was hacked (Sanford Wallace' "Hacker X" excuse)
- it's opt-in (the list they bought said so!);
- it's Double Opt In! We have IPs! (may come from "web bugs" in spam)
- it's not us, it's just our business model;
- moving servers around to new IPs or "morphing";
- using lots of domains, often with anonymized "whois";
- they turn servers back on as soon as the heat is off;
- spam is sent from a different network than their web hosting;
- using "VoIP" as an excuse to cover up proxy hijacking;
- insisting that proxy hijack software was installed by a virus (it's not; it is large, dedicated software installed intentionally by a server administrator to spam).
Other commonly seen tricks:
- redirector URLs on throwaway pages land at their "bulletproof" site hosted with you;
- VPN tunnels and port forwarding to hide from packet-sniffing (use traffic flow analysis);
- using multiple small blocks of IPs instead of a single aggregated block to camouflage spam sources, also used to game search engine optimization algorithms;
- asymetric routing via dynamic pools (rare);
- scripts installed on the same server as an MTA to send mail without leaving log tracks;
- firewalling the host's corporate IPs so the host can't see the spammer's website.
Steps to keeping your network free of spammers:
1. Review your new customers before you sell them accounts. Treat unusual requests with suspicions. If an offer sounds too good to be true, it probably is.
2. On your IP addresses, use "reverse DNS" which points to your domain.
3. Be sure to read "abuse@your domain" every day, and have your upstream provide you with spam reports sent to them about your IPs.
4. Establish "feedback loops" with SpamCop, AOL, and other networks as noted on this FAQ page (top), and read your role accounts every day.
5. When spam is reported, take prompt action to stop it.
6. Do not trust spammers; they are inherently dishonorable people.
7. Treat with extra suspicion clients who say their business involves "mailing".
8. Don't give mailer clients large IP address ranges and allow them to use port-25 out on every IP. Limit them with your routers or firewalls. AOL or Hotmail/MSN only use a handful of IPs for tens-of-millions of users and tremendous mail volumes; why would your client need more?
|But I didn't receive any spam reports! Why not?
Some possible reasons you may not have seen spam reports before an SBL listing include:
Filtering stops most spam from reaching most mailboxes, so it never gets reported as spam.
LART fatigue: Many spam reporters have grown weary of trying to help ISPs keep clean houses and have stopped sending reports. Others report only to trusted ISPs.
You have a non-standard role account. The RFC2142 standard is abuse@domain. (See first item on this page.)
You haven't signed up for Feedback Loops (FBLs). (See first item on this page. Many first-time FBL subscribers get their server knocked down by the volume of such reports. Be prepared.)
Your abuse@ account is behind filters, particularly content filters, which filter out spam reports as well as spam.
Spammers have washed many reporting addresses out of their lists. Such third-party "suppression" lists are even available for sale. (Legitimate senders maintain legitimate suppression lists; they don't sell them.)
Spam services such as websites and DNS are reported less frequently than spam source. Other spam services such as spamming software or spam lists are reported even less frequently.
|How do I control affiliate spammers?
Some Internet businesses use "affiliate" advertising. They pay a commission to partners who drive traffic to their website, for example. While such programs can be effective, they can also be attractive nuisances for spammers. If the spamming remains uncontrolled, such programs may have their IPs listed in SBL as spam support services (SMTP, HTTP, DNS). Here are some ideas for keeping spammers away from your affiliate program:
- Verify each affiliate's real identity and check that they don't have a spammy history before they are allowed in the program.
- Delay commission payments for 30 to 60 days and/or require paid-in-advance forfeitable damage deposit.
- Have a strong Acceptable Use Policy which prohibits spam and terminate promptly for spam with no payment nor refund of deposit.
- Maintain your role accounts and feedback loops and process them daily.
- Register affiliate sending IPs, domains and the domains they will use for URLs, and update www.abuse.net to report those domains to you (and the domain's ISP).
- Monitor webserver logs for unusual traffic bursts from particular affiliates.
- Monitor "referer" headers in webserver logs for unusual traffic bursts from unexpected sources.
- Don't accept HTTP referals from domains in domain blocklists, nor from IPs in SBL or XBL.
- Be suspicious of freshly registered domains and don't allow anonymized whois or bogus registration info among your affiliates' domains.
- Point URLs for terminated affiliates' accounts to a "404" or similar non-promotional page which shows that you do terminate them and that you aren't trying to drum up traffic via disposable spam accounts. (That's mandatory for SBL removal.)
|We're a USA based ISP and a spammer is threatening to sue us if we block his spam. Can he?
In the USA, it's easy to sue anyone for anything, frivolous suits abound. But in this case, the US Congress put into the otherwise poor CAN-SPAM law a provision that states an ISP cannot be held liable for any "good faith to block the transmission or receipt of unsolicited commercial e-mail". The full text of this section is quite clear:
SEC. 4 (c)(1) ISP HELD HARMLESS FOR GOOD FAITH PRIVATE ENFORCEMENT- An ISP is not liable, under any Federal or State civil or criminal law, for any action it takes in good faith to block the transmission or receipt of unsolicited commercial e-mail.
Any spammer bringing such an action or threatening it probably knows the law very well. The filing of such an action could be considered a frivolous suit or a SLAPP suit. Based on this, one may wish to ask for additional punitive damages beyond the court costs when one requests the dismissal of any such suit.
|Some tips on IP allocation and identification.
"ARIN requires organizations to submit information for all IPv4 reassignments of /29 and larger and IPv6 reassignments of /56 or shorter prefix within seven days of the sub-delegation...There are two options for reporting reassignment information: SWIP and Rwhois."
Provide proper rDNS to distinguish dynamic from static ranges. Lack of rDNS normally means dynamic.
Mark your outbound mail hosts with SPF. Another handy SPF tools is here.