SPF Overview

You can help eliminate the spam problem by making it easy to detect forgeries. Protect your e-mail address reputation with a simple DNS technique.

SPF is an emerging antiforgery standard that aims to prevent worms, viruses and spam from forging arbitrary e-mail addresses as the envelope sender in SMTP. SPF has two parts: domain administrators need to publish SPF records in the DNS, and e-mail administrators need to install SPF-enabled MTAs to read those records. SPF records indicate the servers from which a domain sends outbound mail. Mail coming from anywhere else is considered forged.

This article, the first of a two-part series, explains the concepts and trade-offs involved in SPF protection and shows DNS administrators how to set up SPF records. The second article is aimed at showing e-mail administrators how to activate SPF protection in their MTAs. This article was written in early January 2004 and reflects the state of the Internet current at that time.

Worms, Viruses, Joe-Jobs and Envelope Sender Forgery

I got spam from myself today. I founded pobox.com, and I'm an e-mail guy. So I pressed H for headers and read the Received lines. Just as I thought: like much of the spam I receive, this one came from a broadband machine. It's probably an old PIII running Windows 2000 unpatched, used for gaming and MP3s, quietly humming at the foot of someone's bed, draped in dirty underwear. Maybe it lives on a potato farm in Idaho; maybe it looks out over Central Park. Either way, it's probably infected with a variant of the Sobig virus, written under contract to a spammer. The machine's rightful owner has no idea he's infected, no idea his machine has been sending a few hundred spams and viruses every hour since that forgotten day long ago when he clicked on that weird attachment that didn't open.

Spam messages disguise their origins. Spammers use compromised machines to send the spam. They forge message headers. They fake Received headers to throw off the scent, make up bogus Subjects to trick Bayesian filters and forge From lines pretending to be PayPal or eBay.

Spammers also forge the return path. When messages are undeliverable, they bounce back to the sender whose address is in the return path. Not the From: address in the message headers, but the return path of the SMTP envelope, the RFC2821 MAIL FROM. Often, spamware uses lists of old addresses, or they simply guess common user names or launch a dictionary attack. The result is a lot of bad addresses and a lot of bounces.

Spammers don't want those bounces. They'd rather somebody else receive them. So, they pick an address at random or use the recipient's address. That's how they made it look like I got spam from myself. Sometimes they choose a hated enemy and maliciously forge his address so he gets flooded with thousands of bounces.

In 1997, a spammer forged a return address at joes.com, which then was flooded by so many bounce messages it went down for ten days—and gave the world the term joe-job. Hotmail and AOL get joe-jobbed every day: a lot of spam pretends to be from AOL but doesn't really come through their servers. Under conventional SMTP, AOL can't do anything about it. If you put the AOL logo on a T-shirt and tried to sell that shirt, AOL's lawyers would have you ceasing-and-desisting in a heartbeat. But spammers forge @aol.com every day. They can get away with it because they use SMTP.

The Simple Mail Transfer Protocol (SMTP) was designed more than 20 years ago—a kindlier, gentler time. The entire Internet was only a handful of research institutions. SMTP has served us well since then, but it's beginning to show its age.

SMTP is open and trusting. Its rules are relatively lax. You can assert any envelope sender and make up all the headers you want. You could argue today, though, that a protocol that lets joe-jobs happen is a little too open, a little too trusting. That's where sender authentication comes in. SPF tightens the rules.

Sender Authentication with SPF

When you send mail to a domain, your MTA does a DNS lookup (an MX query) to find out to which server to route the mail. Such a server is called a mail exchanger (MX). Small domains tend to have only one MX server. Big domains tend to have more. Mail to a domain goes to its MX servers.

Now for the big idea. In 99% of all cases, when a domain sends mail, that mail originates from a relatively small set of servers controlled by that domain. The domain could designate those servers using the DNS, then announce that any mail not received from those servers probably is forged. That's called a designated sender scheme (Figure 1).

Figure 1. With SPF, one mail server can check whether another server really is associated with the address the mail claims to be from.

Designated sender schemes are useful because they help fight forgery and are easy to set up. After all, domain owners already know which servers send mail from that domain. When I say send mail from that domain, I mean originate an SMTP transaction where the MAIL-FROM envelope sender shows that domain. I'm not talking about the From: header. This is an important distinction.

Mail from a domain tends to come from a small number of servers. That's true for domains large and small. Mail from aol.com comes from AOL's servers. Mail from my personal domain comes from my personal servers. It certainly doesn't come from a machine covered in dirty underwear.

Many ISPs already are implementing these kinds of rules in a haphazard and often slightly broken way. The problem is, one ISP doesn't know the insides of another ISP, and it's easy to guess wrong. Maybe aol.com's mail servers also originate mail for aol.net or vice versa. Wouldn't it be better if AOL themselves announced their designated servers in a simple, flexible, extensible, open format that everybody could use?

Well, they do. SPF is a standard, flexible, extensible, open format that everybody can use. At the time of this writing, AOL recently had started publishing their SPF record.

MTAs can interpret that record and use it to tell whether mail that claims to be from @aol.com is a fake.

Figure 2. SPF performs a simple DNS-based lookup for each incoming message.

All this rule tightening is purely voluntary: domains that don't publish SPF records can continue to send mail as before. Some unusual domains might be served better by not publishing SPF; it's up to them. But most domains should want to use SPF.

To publish SPF, a domain has to add only one line to its zone file. That line is a TXT record, and you can publish it today. Let's see what the TXT record looks like.



Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Re: SPF Overview

Anonymous's picture

I read this article after reading the May 2004 article because my subscription just started up. So, interestingly, everything seemed to flow. Thank goodness Linux Journal posts their articles!

I found this one here because I read the other article and thought "Hey, I need to do/learn this! Where's that previous article." A few clicks later, there it was. :)

I can understand the first person's frustrational comment about the definition of SPF. I explained a little of what I was doing to my signifigant other, and the initial response, after defining SPF, was "Oh, I thought it was like sunblock. You know, like SPF-15, SPF-45.."

In a way, it is like sunblock. It prevents your systems from burning up from processing all of that spam!

Re: SPF Overview

Anonymous's picture

Just for the record, the other half of this article, which is obviously more detailed due to the complexity (setting up the email side of thigns), is in the April issue:



Anonymous's picture

I don't believe this article even once tells us what SPF stands for.* Perhaps it means nothing, the latest fad in (non-) acronyms.

How did this piece make it past an editor?

*For the frustrated reader: it stands for Sender Policy Framework.

Re: SPF?

Anonymous's picture

You seem to be in the possession of Internet access... A quick trip over to spf.pobox.com would have answered your question rather quickly ;)

Re: SPF?

Anonymous's picture

I think you are missing out on some experience when it comes to technical writing. Technical lingo should be explained, but not to the depth of the PDR (Physician's Desk Refeence). Because the industry is so acronym-laden, many have found a common style to use the acronum once and place an explanation in parantheses afterwards, describing the acronym or term, but after that, to use the acronym (only) as the meaning has been explained. The initial reference can be seen in my reference to the PDR above.
And finally, but most importantly, material should be written|developed|edited so a reader doesn't have to read a sentence more than once for comprehension. Have you ever found yourself halfway into a sentence (technical, romance, etc. and say, "Huh?" then go back to the beginning of the sentence and start reading it again? That's an example of a bad book. Bad book! Bad bad book.

Re: SPF?

Anonymous's picture

I agree. Thanks for suppling the definition. I though the article was good.