According to a new study I cited by Pivotal Veracity, an e-mail deliverability consulting service, an average of 15% of the permission-based (opt-in) e-mail messages from the firm's clients were bounced or deleted by Internet service providers. Another 7% of the messages were incorrectly routed to a "spam" or "bulk" folder by ISPs. Different ISPs score anywhere from a low of 2.6% of such messages wrongly bounced or deleted (Yahoo) to a high of 53.4% incorrectly labeled as spam (Google Mail).
In another study, which won't be released until May, the consulting firm subscribed to the e-mail lists of more than 100 major corporations. One or more e-mail messages from 37% of the studied companies were misrouted to a spam or bulk folder, according to Pivotal Veracity's president, Deirdre Baird.
ISPs, of course, are simply trying to keep objectionable spam from reaching their end users. Unfortunately, the latest figures indicate that too many ISPs are throwing out good mail with the bad.
I received several responses to my column. The most interesting was from Suresh Ramasubramanian, the manager of security and antispam operations for Outblaze Limited, a respected e-mail service provider. Here's what he had to say.
Surveying The Field
"We are a messaging systems hosting provider based out of HKG [Hong Kong]," Suresh writes, "and we host email and spam filtering for 3 to 4 domains that were surveyed/referenced in your article." Three of the domains Outblaze hosts are Mail.com, Lycos and Register.com.
Pivotal Veracity had shown that 18.8%, 18.6%, and 8.5% of the permission-based e-mail messages sent to these three ISPs, respectively, were "Opt-In Mail Missing." Since Outblaze handles the spam filtering for all of these domains, the figures should have been identical instead of varying, Suresh said.
Fortunately, there's a simple answer to this. "Opt-In Mail Missing," as the column described it, represents opt-in e-mails that were bounced or deleted by the ISPs. E-mails that had been filtered into a spam folder by the ISPs were in a separate category called "Opt-In Mail Misrouted to Spam Folder."
That category, according to Pivotal Veracity's figures, showed that all three of the ISPs handled by Outblaze had a perfect record of spam filtering. You can't get a better score than 0.0% of opt-in e-mails misrouted, as these companies scored in the consulting firm's study.
Like several other ISPs in the study, these three providers seem to do a good job of recognizing that opt-in e-mails are not spam — now all they have to do is work on those bounces.
Let A Thousand Studies Bloom
Suresh also pointed out that more than one company studies the current collapse of e-mail reliability.
"There's more than one survey out there that measures deliverability," Suresh writes. "Besides Pivotal Veracity (whose survey gave you the data points for this article), there are Return Path, Lyris, etc."
At the time of those earlier columns, Return Path said opt-in e-mails that were deleted, bounced or misrouted as spam by 16 top ISPs totaled almost 19% of the permission-based mail.
If that weren't bad enough, Return Path's latest figures, which cover all of 2004, show that the top ISPs are now misrouting 22% of the legitimate mail the consulting group monitors. That's about 3 percentage points worse than a year earlier. Interestingly, 22% is the same percentage Pivotal Veracity currently cites (15% of opt-in e-mails go missing, while another 7% are received but shunted to a spam folder).
As for Lyris, an e-mail marketing and publishing service, its own report makes pretty much the same points as the others, although the figures are slightly lower. The company's latest study, covering the first quarter of 2005, says more than 12% of permission-based e-mails are sent astray by 29 major U.S. ISPs. That includes 3.4% that were misrouted as spam and 8.7% that were bounced.
The variation, such as it is, in e-mail reliability studies like these is almost certainly due to the fact that these consulting firms are measuring different mail streams. Each one has a client base that's used as a "test bed" to determine which e-mails get delivered to test accounts at which ISPs. Given that reality, it's remarkable that the latest three studies are as similiar as they are.
Michelle Eichner, vice president of client services for Pivotal Veracity, says these studies are a step toward gaining better reliability for corporations that now rely on e-mail to communicate with their customers.
"ISPs are forced to continue implementing new and different spam-filtering technology, which may not always be perfect," Eichner says. "We believe both 0% spam and 0% false positives will only be achieved through the joint efforts of the receiving and legitimate mailing community."
Here's a positive sign: One of the world's largest ISPs contacted Pivotal Veracity after last week's story appeared and asked how it can improve its delivery rate for permission-based e-mails, according to Eichner. Outreach like this might make e-mail a communications medium we can rely on once again.