SPF, MTAs and SRS
Qmail does not have the same kind of plugin interface that the other MTAs do. Instead, SPF provides a patch that integrates SPF directly into Qmail. In addition, many Qmail users screen their mail with qpsmtpd: if you do, SPF is a plugin you can turn on easily.
James Couzens is the primary author of the C SPF library. libspf comes with a patch for Qmail and for other MTAs as well.
Once you've installed the plugin and turned it on, you should perform two tests. First and most important, legitimate mail needs to get through. If something broke, maybe you're not running something you need to—double-check. If it's still broken, back out the patch and report your experience to the spf-help mailing list.
Second, confirm that forged mail is rejected. If you can speak SMTP by hand, engineer a message with MAIL FROM:<email@example.com>. The domain altavista.com is not used for mail, so it always returns a FAIL message. They have asked that test messages contain the word test. This can be tricky to execute because if they recognize a trusted client, both your MTA and SPF will turn a fail into a pass. Therefore, don't telnet to localhost; use your machine's actual hostname, and if possible try to open the connection from an outside host. If you receive a 550 response and an error message that refers to spf.pobox.com/why.html, it's working.
If you use a secondary MX, tell your SPF client not to reject its mail. How to do this is described in detail in the installation instructions for your plugin.
You should notice that your mail now contains a Received-SPF header that carries a number of result codes:
NONE: the domain does not publish SPF records. Your MTA should proceed as usual.
PASS: the mail is not forged, but that doesn't mean it's legitimate. Remember, spammers can publish SPF too. You still should test its domain against a right-hand-side block list (RHSBL). But if the sender is on your trusted whitelist, you can skip further antispam checks with confidence.
FAIL: the mail is a forgery, and you can reject it with confidence. There is a miniscule chance the message is legitimate but was sent by a misconfigured sender. In that case, the error message they receive tells them they need to configure their MUA with SMTP AUTH. SPF's design philosophy is that it's better to fail obviously with a hearty error message than to risk silently burying mail in a spambox.
SOFTFAIL: the message could be a forgery, but the domain's ISP is working on switching its users to SMTP AUTH, so the message could be legitimate. You should accept the message, but subject it to more stringent antispam checks.
NEUTRAL: the domain just has started down the road to SPF, and their default response is ?all. They would like you to pretend the response was NONE while they consider moving the default toward SOFTFAIL and FAIL. Big ISPs with millions of users move slowly; it's not their fault.
ERROR: there was a temporary DNS lookup error. Normally, your MTA should return a 450 temporary failure when this happens.
UNKNOWN: a permanent error caused the SPF lookup to abort; perhaps there was a syntax error in the record, or maybe the record pointed to another domain that doesn't have an SPF record.
In the past ten years we have grown tremendously dependent on e-mail; we are made aware of just how dependent we are every time a worm hits. Analysts routinely announce that spam and viruses cost the economy billions of dollars. The success of SPF shows that people are desperate for change.
But, change has its own price. If there were such a thing as a painless solution to spam, we already would have adopted it. The war on spam has dragged on so long in part because the best experts on spam simply could not agree on exactly what trade-offs they wanted to make, but that phase of debate is drawing to a close. In every antispam future they have discussed, sender authentication is the first and fundamental step. Now, many possible sender authentication models are available, but the designated-sender scheme that SPF provides is probably the easiest to implement.
Cryptography definitely is in our future, but it's not here yet. Like first aid, SPF offers immediate benefit, and it's something we can do right away.
What is the price of SPF? Every designated sender scheme breaks two things. First, SPF breaks verbatim e-mail forwarding (Figure 1). Services that provide permanent e-mail addresses, such as pobox.com, are used to forward mail the way UNIX .forward and /etc/aliases files do. When the mail leaves their servers, the return-path address in the envelope is unchanged. But in an SPF world, resent messages now look a lot like forgeries. To fix this, forwarding services need to rewrite their return paths. So do other sites that depend on .forward and /etc/aliases to send mail off-site.
The solution is called SRS, sender rewriting scheme. It encapsulates the original sender address in the rewritten, SPF-compliant, return address. If a message should bounce, it comes back to you, and you unwrap the address and forward the bounce back to the sender. Forwarding services would have to do this even in a world without SPF, because ISPs already are performing pseudo-SPF checks. SPF simply gave everyone a standard way to do what they already were doing piecemeal. In the same way that responsible sites closed down their open relays over the past few years, in the coming months responsible sites will begin to operate SRS-compliant forwarding; pobox.com already is doing SRS, and other forwarding services are expected to follow.
The good news is the community that developed SPF already has produced SRS code for your MTA. Those patches are available from the same place you got your SPF patches. By the time you read this, they even might be bundled into your MTA. The goal is for the average installation to be able to upgrade to the latest version and have SRS magically work (see Resources).
So, this solves the e-mail forwarding problem. Getting SRS into the field is simply a matter of time. But SPF also breaks Web-generated e-mail. Greeting card sites and “e-mail me this news article” sites tend to use your e-mail address not only in the From: header but in the envelope sender too. In SPF terms, that kind of behaviour is indistinguishable from forgery.
To solve this problem, those sites can do one of two things. First, if the mail they send isn't that important, they can set the return-path address to firstname.lastname@example.org and eat the bounces. Newer, more progressive sites, such as Orkut, already do something like this. But if the mail is important, was sent on behalf of a user who was logged in to the Web site properly, and if the Web site had previously confirmed the user's e-mail address, then the Web site could perform SRS on itself—encapsulating the user's return address so that bounces would be properly forwarded.
What about the transition period, you ask. Won't there be a time of disruption while the forwarders groan their way toward SRS-compliance? What about the sites that are unwilling or slow to adapt?
Well, here's a little secret. We have a fairly good idea who the major culprits are; we know, for instance, that eBay sometimes sends mail with a legitimately forged envelope return-path. The people who developed SPF use eBay, too, and they don't want to lose e-mail any more than you do. So they came up with a hack. They set up a whitelist that identifies all these legitimate forgers; pobox.com is on the list, as are acm.org, eBay and the newspaper Web sites that do “e-mail me this article”.
Every SPF client we've talked about in this article knows about that whitelist. Every SPF client we know of gives that whitelist a chance to override a fail. If your mother sends mail from her AOL account to your acm.org address, your SPF client accepts that message, even though it's technically a forgery. (If you get forwarded mail through a system that's not on the list—from, say, a friend's home Linux box—you should whitelist that box in your MTA.) When acm.org implements SRS, the problem will go away.
SPF's critics tend to say “it breaks forwarding”. The SPF community rose to the occasion and did their best to ease the transition. They offered two solutions, one short-term and one long-term, that meet in the middle. Together they sugarcoat the bitter pill.
Change means pain. The transition to an SPF world won't be painless, but it's like the pain of an injection that makes the illness go away. E-mail is very sick. Some say it will not survive spam, but I don't agree. I think SPF will set it firmly on the road to recovery.
Resources for this article: /article/7465.
Meng Weng Wong is founder and CTO of pobox.com, the e-mail forwarding company, which celebrates its tenth anniversary this year. He is working on a science-fiction novel set on a planet where traditional fantasy magic turns out to be implemented, following Clarke's famous dictum, using nanotechnology.
Editorial Advisory Panel
Thank you to our 2014 Editorial Advisors!
- Jeff Parent
- Brad Baillio
- Nick Baronian
- Steve Case
- Chadalavada Kalyana
- Caleb Cullen
- Keir Davis
- Michael Eager
- Nick Faltys
- Dennis Frey
- Philip Jacob
- Jay Kruizenga
- Steve Marquez
- Dave McAllister
- Craig Oda
- Mike Roberts
- Chris Stark
- Patrick Swartz
- David Lynch
- Alicia Gibb
- Thomas Quinlan
- Carson McDonald
- Kristen Shoemaker
- Charnell Luchich
- James Walker
- Victor Gregorio
- Hari Boukis
- Brian Conner
- David Lane