Can We Restore Reliability to E-Mail?

I wrote in this space
last week that e-mail is continuing to become more and more unreliable as a
way for companies to reach consumers who sign up on notification lists.

According to a new study I cited by Pivotal Veracity, an e-mail
deliverability consulting service, an average of 15% of the permission-based
(opt-in) e-mail messages from the firm’s clients were bounced or deleted
by Internet service providers. Another 7% of the messages were incorrectly
routed to a “spam” or “bulk” folder by ISPs. Different ISPs
score anywhere from a low of 2.6% of such messages wrongly bounced or deleted
(Yahoo) to a high of 53.4% incorrectly labeled as spam (Google Mail).

In another study, which won’t be released until May, the consulting firm
subscribed to the e-mail lists of more than 100 major corporations. One or more
e-mail messages from 37% of the studied companies were misrouted to a spam or
bulk folder, according to Pivotal Veracity’s president, Deirdre Baird.

ISPs, of course, are simply trying to keep objectionable spam from reaching
their end users. Unfortunately, the latest figures indicate that too many ISPs
are throwing out good mail with the bad.

I received several responses to my column. The most interesting was from Suresh Ramasubramanian, the manager of security and antispam operations for Outblaze Limited, a respected e-mail service provider. Here’s what he had to say.

Surveying The Field

“We are a messaging systems hosting provider based out of HKG [Hong Kong],” Suresh
writes, “and we host email and spam filtering for 3 to 4 domains that were
surveyed/referenced in your article.” Three of the domains Outblaze hosts are
Mail.com, Lycos and Register.com.

Pivotal Veracity had shown that 18.8%, 18.6%, and 8.5% of the
permission-based e-mail messages sent to these three ISPs, respectively, were
“Opt-In Mail Missing.” Since Outblaze handles the spam filtering for all of these
domains, the figures should have been identical instead of varying, Suresh said.

Fortunately, there’s a simple answer to this. “Opt-In Mail Missing,” as the column
described it, represents opt-in e-mails that were bounced or deleted by the ISPs.
E-mails that had been filtered into a spam folder by the ISPs were in a separate
category called “Opt-In Mail Misrouted to Spam Folder.”

That category, according to Pivotal Veracity’s figures, showed that all three of
the ISPs handled by Outblaze had a perfect record of spam filtering. You can’t
get a better score than 0.0% of opt-in e-mails misrouted, as these companies
scored in the consulting firm’s study.

Like several other ISPs in the study, these three providers seem to do a good
job of recognizing that opt-in e-mails are not spam —
now all they have to do is work on those bounces.

Let A Thousand Studies Bloom

Suresh also pointed out that more than one company studies the current collapse
of e-mail reliability.

“There’s more than one survey out there that measures deliverability,” Suresh writes.
“Besides Pivotal Veracity (whose survey gave you the data points for this article),
there are Return Path, Lyris, etc.”

I reported in this space on
April 19, 2004, and

Sept. 14, 2004, about the reliability studies published by Return Path, a
provider that recently acquired
Bonded Sender, an e-mail reputation service.

At the time of those earlier columns, Return Path said opt-in e-mails that were
deleted, bounced or misrouted as spam by 16 top ISPs totaled almost 19% of the
permission-based mail.

If that weren’t bad enough, Return Path’s latest
figures, which cover all of 2004, show that the top ISPs are now misrouting
22% of the legitimate mail the consulting group monitors. That’s about 3
percentage points worse than a year earlier. Interestingly, 22% is the same
percentage Pivotal Veracity currently cites (15% of opt-in e-mails go missing,
while another 7% are received but shunted to a spam folder).

As for Lyris, an e-mail marketing and publishing service, its own report makes
pretty much the same points as the others, although the figures are slightly
lower. The company’s latest study, covering the
first quarter of 2005, says more than 12% of permission-based e-mails are sent
astray by 29 major U.S. ISPs. That includes 3.4% that were misrouted as spam and 8.7% that were bounced.

Conclusion

The variation, such as it is, in e-mail reliability studies like these is almost
certainly due to the fact that these consulting firms are measuring different
mail streams. Each one has a client base that’s used as a “test bed” to
determine which e-mails get delivered to test accounts at which ISPs. Given that
reality, it’s remarkable that the latest three studies are as similiar as they
are.

Michelle Eichner, vice president of client services for Pivotal Veracity, says
these studies are a step toward gaining better reliability for corporations that
now rely on e-mail to communicate with their customers.

“ISPs are forced to continue implementing new and different spam-filtering
technology, which may not always be perfect,” Eichner says. “We believe both 0%
spam and 0% false positives will only be achieved through the joint efforts of
the receiving and legitimate mailing community.”

Here’s a positive sign: One of the world’s largest ISPs contacted Pivotal
Veracity after last week’s story appeared and asked how it can improve its delivery
rate for permission-based e-mails, according to Eichner. Outreach like this
might make e-mail a communications medium we can rely on once again.

RELATED NEWS AND ANALYSIS

• Ethics and Artificial Intelligence: Driving Greater Equality

  FEATURE |  By James Maguire,
  December 16, 2020

• AI vs. Machine Learning vs. Deep Learning

  FEATURE |  By Cynthia Harvey,
  December 11, 2020

• Huawei’s AI Update: Things Are Moving Faster Than We Think

  FEATURE |  By Rob Enderle,
  December 04, 2020

• Keeping Machine Learning Algorithms Honest in the ‘Ethics-First’ Era

  ARTIFICIAL INTELLIGENCE |  By Guest Author,
  November 18, 2020

• Key Trends in Chatbots and RPA

  FEATURE |  By Guest Author,
  November 10, 2020

• Top 10 AIOps Companies

  FEATURE |  By Samuel Greengard,
  November 05, 2020

• What is Text Analysis?

  ARTIFICIAL INTELLIGENCE |  By Guest Author,
  November 02, 2020

• How Intel’s Work With Autonomous Cars Could Redefine General Purpose AI

  ARTIFICIAL INTELLIGENCE |  By Rob Enderle,
  October 29, 2020

• Dell Technologies World: Weaving Together Human And Machine Interaction For AI And Robotics

  ARTIFICIAL INTELLIGENCE |  By Rob Enderle,
  October 23, 2020

• The Super Moderator, or How IBM Project Debater Could Save Social Media

  FEATURE |  By Rob Enderle,
  October 16, 2020

• Top 10 Chatbot Platforms

  FEATURE |  By Cynthia Harvey,
  October 07, 2020

• Finding a Career Path in AI

  ARTIFICIAL INTELLIGENCE |  By Guest Author,
  October 05, 2020

• CIOs Discuss the Promise of AI and Data Science

  FEATURE |  By Guest Author,
  September 25, 2020

• Microsoft Is Building An AI Product That Could Predict The Future

  FEATURE |  By Rob Enderle,
  September 25, 2020

• Top 10 Machine Learning Companies 2021

  FEATURE |  By Cynthia Harvey,
  September 22, 2020

• NVIDIA and ARM: Massively Changing The AI Landscape

  ARTIFICIAL INTELLIGENCE |  By Rob Enderle,
  September 18, 2020

• Continuous Intelligence: Expert Discussion [Video and Podcast]

  ARTIFICIAL INTELLIGENCE |  By James Maguire,
  September 14, 2020

• Artificial Intelligence: Governance and Ethics [Video]

  ARTIFICIAL INTELLIGENCE |  By James Maguire,
  September 13, 2020

• IBM Watson At The US Open: Showcasing The Power Of A Mature Enterprise-Class AI

  FEATURE |  By Rob Enderle,
  September 11, 2020

• Artificial Intelligence: Perception vs. Reality

  FEATURE |  By James Maguire,
  September 09, 2020