When Microsoft decided last year to systematically survey users of all the sites within MSN, the world’s second-largest Web network behind AOL Time Warner’s, the company had an instinctive reaction — let’s develop the survey software in-house.
For starters, Microsoft had already built a simple survey tool that it used for internal research, housed inside its own firewall. And it’s not likely to find a deeper pool of programmers than within its own Redmond, Wash., complex walls.
|AT A GLANCE: Microsoft|
The company: The software giant is a major Internet force and operates MSN, the second-largest network of Internet sites in the world, with 6.5 million subscribers and 250 million unique global visitors.
The problem: The need for a customizable survey tool to get feedback from visitors to dozens of different Web sites in multiple languages spread across 33 countries.
The solution: Web-based survey software from Perseus Development Corp., which Microsoft uses to conduct about 50 surveys each month, with each generating from 2,500 to 5,000 responses.
But after considering the problem of creating surveys for thousands of visitors to dozens of Web sites in multiple languages in 33 countries, and tying all the results together in one database with a simple reporting tool, Microsoft backed away.
“It would have required creating a separate product unit just for survey tool creation, then update the program on an ongoing basis,” said Deepak Agrawal, research lead for the company’s MSN group. “Someone has to own a project like that, and there was no logical place to put it inside Microsoft. We’re not in the research business. So we decided to go outside.”
After considering four or five candidates, MSN settled on Perseus Development Corp. Perseus got its start in 1994 with software designed to allow Mom and Pop stores to create their own questionnaires quickly, in Microsoft Word word-processing documents, and then add the results to a database for easy analysis.
Easily Imported Into SQL Database
In 1997 Perseus Web-enabled the program, allowing Web site owners to create user surveys with a product called SurveySolutions. The program is compatible with Microsoft Office software, meaning tools, menus, and shortcut keys are similar to those used with Excel, PowerPoint, and other Office programs. That made it easy for Perseus users familiar with Office to design surveys.
As a bonus, it made Perseus more attractive when the Redmond giant itself came calling. Not only was the company’s survey-creation software compatible, but responses could be easily imported into an SQL database server on Microsoft’s back end. (The fact that a recent version of SurveySolutions last year won a PC Magazine award didn’t hurt, either.)
MSN has about 50 people spread across 33 countries responsible for data collection of its various Web sites (including Gamingzone, MoneyCentral, CareerBuilder, Hotmail, and many more). Each of the 50 can download SurveySolutions Enterprise (designed for large organizations) in about 30 minutes. Each can then create a survey in a Word document, convert it into a Perseus document, and, after an hour’s labor, have a standard, 30-question survey ready to publish. The survey can then be posted to a unique Web address outside the Microsoft firewall, and MSN visitors will begin to see pop-up windows inviting them to take a survey.
MSN does about 50 surveys a month, each generating between 2,500 and 5,000 responses. The network wanted to find out more about its visitors for the same reasons most Web businesses do — to get feedback allowing the company to make the sites more appealing, thus generating more repeat visits and revenue. The surveys are mostly generic, asking visitors about their levels of satisfaction, what they like and dislike about the sites, and general demographic info. Virtually all of the surveys are anonymous; because MSN almost never runs contests or giveaways to attract poll-takers, it collects no personal data such as names or addresses.
Agrawal says MSN, which signed its contract with Perseus in June 2000 and began publishing surveys in October, has consistently found that 10 percent of people who see the pop-up window respond — about the mid-point of what it expected, and somewhat above typical online survey response rates (especially good considering that MSN offers no incentives).
Functionality That’s Not ‘Out Of The Box’
Perseus provides what Agrawal said was “a lot of day-to-day technical support and administration,” including customization of a secure, password-accessed Web portal in which the MSN employees can communicate. MSN workers can view the data over the Internet just as they would if the data were housed in their desktops — a function Perseus says is unique among online survey providers.
Perseus staff also help MSN set up more complex surveys, such as those carried out in 22 countries in multiple languages, with the data compiled in separate tables depending on which site the respondent was visiting.
“They provide a lot of functionality that’s not out of the box,” Agrawal said, noting among other things Perseus’ ability to handle many languages. “The other survey companies we looked at had a more canned, pre-determined approach. They were easier to use and learn, maybe, but they were much more restrictive and didn’t provide much freedom to create custom surveys. We knew we’d outgrow them. That’s not the case” with Perseus.
Agrawal declined to say how much MSN has paid for the service, saying the total of its 3-year contract will be less than $250,000. Most other survey companies planned to charge differently, typically between $1,000 and $1,500 per survey. If MSN does 50 surveys each month, a $1,000 per-survey cost would have totaled $1.8 million over three years.
Agrawal says it took almost six months before the Perseus software was deployed, a fact he described as “not desirable. We had problems both at our end and at their end. Perseus is a small company, and they were only too happy not to be pushed. As is true of anyone, if you don’t ask for something, you don’t get it. Once we found someone at our end” — Agrawal himself — “to start pushing, they proved very responsive. Overall, they’ve provided very good response.”
One year in, it’s too early for MSN to decide whether to renew its 3-year contract. Agrawal says, “We’re on the path to invest a lot of time and effort into this. We hope it will prove cost effective. The system is continually evolving, and they’ve helped us work out the bugs. Overall, we’ve been quite happy.”