False Positives: Spam's Casualty of War Costing Billions

IT workers are fighting every day to keep spam out of their corporate networks. They're enlisting black lists. They're installing filters. They're educating users.

And while the spam continues to flood in despite their best efforts, another problem is lurking in the shadows. Legitimate email -- important email -- isn't getting in when it should. Business propositions, partner contacts, resumes... they're all getting swept away by the same tools that are filtering out the spam.

And industry analysts say money is being lost, customers are being lost and key opportunities are being missed because our best-laid spam efforts are mistakenly throwing the baby out with the bath water. Blocked legitimate email, or false positives, is costing U.S. businesses roughly $3.5 billion this year alone, according to a new study from San Francisco-based Ferris Research Inc.

Analysts say false positives are increasingly becoming the flip side of spam.

''Of great importance to corporate is that 70 percent of people have not gotten email that was expected,'' says Vincent Schiavone, president of Philadelphia-based ePrivacy Group Inc. ''When it comes to blocked email, the consumer is inconvenienced. The enterprise could be losing an expensive deal... When you send a business-to-business email, you don't need it caught in a spam filter. That stops business. False positives damage business.''

And that damage is sometimes overlooked in the heated battle against spam, say analysts. Spam is more than a constant nuisance. It overruns email systems. It wastes workers time, and it brings porn and viruses into the company. When business executives are loudly complaining to IT these days, they're often complaining about spam.

So IT works, struggles, to keep spam out of their system. If a few legitimate emails are blocked in the fight, well, that's just a casualty of war.

But, analysts warn, it's an expensive casualty -- one that most companies may not be able to afford to make.

''You're damned if you do, damned if you don't,'' says Sara Radicati, president and CEO of The Radicati Group, Inc., a Palo Alto, Calif.-based market research and consulting firm specializing in messaging issues. ''We're all in the information business really. If you lose an important piece of information that your competitors get, you lose competitive advantage. You could lose deals. It could lead to major disconnects with clients. There could be a lot of losses.''

Radicati says there is not fixed rate of false positives when it comes to filtering technology. The rate varies with each individual product. She notes, however, that it's generally accepted that most filtering software has a false positive rate of between 1 percent and 10 percent.

Phil Goldman, CEO and founder of Los Altos, Calif.-based Mailblocks Inc., a personal email service company, says IT managers need to be aware that the cost of missing an email is much greater than the cost of inadvertently reading spam.

''IT managers are extremely concerned about it,'' says Goldman. ''If an email is lost, who is going to get blamed? It's the IT manager. They're caught between a rock and a hard place. If they turn down spam protection, they bear the brunt of a lot of spam coming through and the bandwidth use and the productivity loss. But if they block out the spam and lose emails, it could be even worse. Any message could be a mission critical message.''

Analysts say that's why many IT managers have chosen to go easy on spam. More offers of wild porn, hair regrowth tonics and body enhancers get through to users' inboxes, but at least they're not missing the big emails.

''Sadly, we talk to a lot of firms that say they'd rather put up with spam than lose potential business,'' says Radicati. ''They'd rather have employees hit delete 20 or 30 times than lose important information.''

And Radicati says it's not an easy problem to deal with. When it comes to eliminating spam but eliminating false positives, as well, there aren't a lot of solutions out there yet.

''Until the technology improves, there aren't a lot of options,'' she adds. ''Right now, the products out there that block spam have false positives. There are some solutions that let you go in and look at what's been rejected so you can recover something. But that takes up a lot of time. Then it becomes someone's job to sift through it... And what we hear from everybody about white lists is that they don't work very well. If you know everyone who is emailing you, then it's fine. But what about new business and emails coming from people you just don't know yet.''

Ferris Research's Chris Williams says IT managers should keep in mind that while false positives are costing American businesses about $3.5 billion this year, spam is costing them $10 billion.

''False positives is a problem but it's still not as expensive a problem as spam,'' he notes. The answer, he says, is to try to find a solution that addresses both issues. But beware that it will be hard to find.

''There are many different ways anti-spam software can be implemented,'' says Williams. ''We can delete all mail we think is spam at the server. That's probably the wrong approach for people sensitive to false positives. A better approach for them is to quarantine it into a junk folder so people who really care about their mail can go and check it. IT managers need to pay attention to the false positive rate of these products. It's not just about getting a product that blocks 100 percent of spam, but how much legitimate email is blocked as well. That just isn't a good trade off.''

Mailbocks' Goldman says better solutions will be coming down the road as more and more companies start to worry about false positives, as well as spam. The more they worry, the more they complain to their software vendors.

''It's part of a more general and mature look at spam and anti-spam,'' says Goldman. ''It will go beyond 'Did I get spam or not?'. It will include other factors, like false positives and management overhead.''