Six months ago you couldn't open a computer trade mag or visit a tech web site without being confronted with at least one article about Sender ID, Sender Policy Framework (SPF) and the plans to make a standard for MTA authorization records in DNS to help fight spam.
But in late September the relevant Internet Engineering Taskforce (IETF) working group closed amid disillusionment that Microsoft had patented too much of the technology under discussion for any solution derived from it to be acceptable to a wide enough range of people. Since then things have gone a bit quiet and you could be forgiven for thinking that the fight against spam at the standards level has been abandoned.
In fact, SPF, the system devised by Meng Weng Wong CTO of Philadelphia, PA-based e-mail company Pobox, is still very much alive, as is Microsoft's Sender ID.
Both work in very similar ways, essentially verifying that an e-mail really has been sent from where it says it has. Wong says that between one and two million domains publish SPF records, and research shows that large ISPs can use SPF or SenderID to test between 20 percent and 40 percent of their incoming mailstreams (since a large proportion of their e-mail comes from a handful of large SPF senders including AOL and Hotmail.)
Another system, Yahoo!'s DomainKeys, uses public key encryption to verify that an e-mail has been sent by the person who has signed it (and that it has not been altered during transmission) and about 100 domains use it, according to Yahoo! figures.
But there is one huge question that needs to be asked: In the fight against spam, how do these systems help? Simply knowing where an e-mail comes from isn't enough to guarantee that it isn't spam. In theory a spammer can set up a server with SPF and send out a huge batch of spam. The recipients, assuming they had implemented SPF, would see that the e-mail comes from where it says it does - but it would still be spam. The same is true using SenderID or Domainkeys.
"What we need is for these systems to be tied in with a reputation system," says John Levine, chair of the Internet Research Task Force (IRTF) anti-spam research group. A global reputation system, Levine says, would work like a credit bureau, providing a source of information about people's past spamming activities and how likely they are to be trustworthy in terms of sending out spam. Someone with a long record of not being a spammer would likely be trustworthy; a new identity with no record of not sending spam may be a new e-mail user, or could be a spammer setting themselves up with a new identity.
Yahoo! is planning to link DomainKeys to its own reputation system in the coming months, according to Miles Libbey, anti-spam product manager at Yahoo! Mail. Pobox's Wong is working on a reputation aggregation service called Karma to work with SPF. All three systems are being used with small, independently run reputation systems already, but ultimately a single, global, non-proprietary reputation system run for the benefit of all Internet users may be preferable, Levine says.
Levine appears to favor DomainKeys' identity based approach over the SPF/Sender ID model, highlighting a potential problem with SPF and other systems which try to tie e-mail from a particular domain to certain registered SMTP servers "allowed" to send e-mail from that domain.
"Many people in New York who went to Cornell University have a Cornell e-mail address which they can use for life. But only about half of these people are still at Cornell – the rest send e-mail from what ever service they are connected to. So Cornell, for example, can't use SPF because mail with a Cornell address is sent via many different servers. SPF simply doesn't deal with that very well."
Black & White Approaches, and the Shape of Traffic to Come
SPF, the Sender ID framework and DomainKeys are certainly not the only games in town when it comes to fighting unwanted mail. Many anti-spam systems use the principal of blacklisting - blocking mail from servers that have been reported to send spam in the past for a short period of time. But these are becoming increasingly less effective as spammers now commonly use compromised zombie machines to send out e-mail, and they can be annoying for innocent people who find their mail is blocked from reaching certain addresses because someone else has sent spam e-mail from their mail server
Another solution is whitelisting - using systems like Mountain View, CA-based Habeas' Warranted Email or Alpharetta, GA-based CypherTrust's IronMail Bonded Sender program.
Using IronMail's system, a company promises not to send spam, and puts up some money which it forfeits if it does. Companies that sign up to the Bonded Sender program can whitelist e-mail from Bonded Senders, since it is unlikely to contain spam – the cost would be too great. The drawback is that this only really helps the sender ensure their e-mail gets though – it does nothing to stop spam clogging mail servers.
Perhaps the most promising new approach to spam control is traffic shaping, according to Ant Allen, a research director at United States based research house Gartner Group. This involves using a variety of analytic methods to make judgments about different e-mail sources. Servers that are estimated to be poor sources have limits imposed on the amount of SMTP connections they can establish. So while no server is blacklisted, suspect sites are restricted to transferring just a handful of e-mails per hour. This has the double-whammy effect of reducing spam and tying up the resources of spammers who may be trying to send bulk e-mails, making it too expensive to send spam to traffic shaping-protected servers.
As things stand, the spam story is one of both good and bad news. The good news, according to Levine, is that while spam levels have been rising exponentially for the past few years, they peaked earlier this year and now seem to have plateaued. The bad news is that because of the very anti-spam efforts that may have caused this plateau, e-mail is getting less reliable. Whereas five years ago an e-mail was either delivered of bounced back, now legitimate e-mail may be silently filtered by anti-spam software and never delivered. Vital messages sometimes don't get though, and the evidence is that many people are turning their backs on e-mail.
Ultimately, Levine believes that while anti-spam systems can help reduce the unnecessary costs and wasted time caused by unwanted mail, they will never solve the problem by themselves. "There is no simple solution to the spam problem as it's a legal rather than a technical problem. Technology can only help sort mail," he concludes.