Use metrics to find the right IT outsourcing partner

You may think that only classified ads or personnel evaluations will contain terms such as “innovative,” “flexible,” and “quality relationship.” Today, however, companies use these terms–called “metrics”–to judge how well outsourcing vendors perform. When you think about benchmarking your outsourcer, you naturally want to gauge whether it helps your company improve its return. Yet, according to a number of outsourcing experts, companies are increasingly judging the success of these deals on subjective issues, in addition to the tried-and-true quantitative benchmarks.

Subjective measures come into play as outsourcing moves beyond help-desk applications and mainframe maintenance into helping a company meet its electronic commerce initiatives, or more efficiently run its accounting department. The goals don’t just improve productivity, but they improve processes and new business as well.

The new economy

The Internet plays a major role in the move to value vendors based on more subjective measures. In the Web world, companies are not judging their outsourcer on code per person but on knowledge of Web-based technologies, quick installation of these technologies, and ultimately the company’s customer satisfaction.

“A primary driver for outsourcing is access to new technology and access to skilled human resources that a company can’t afford to bring in or doesn’t have the time to bring in and train,” says Cynthia Doyle, senior analyst for information systems outsourcing and business process outsourcing with International Data Corp., in Framingham, Mass.

Chicago-based Andersen Consulting refers to this type of outsourcing as “sourcing.” One partner says the firm has three times the number of sourcing deals in its pipeline as it has traditional outsourcing deals. Virtually all of these opportunities are in electronic commerce.

Typical performance metrics don’t seem to work in the new environment, says one corporate IT employee, familiar with her organization’s outsourcing contracts. Those in the field agree that benchmarks such as local area network uptime, server uptime, the amount of time it takes to close a call to a help desk are all good because they provide quantitative measures. These same experts agree that the challenge is how you qualitatively measure the service your customers perceive they’re getting.

The productivity ratio of code per person isn’t important when you’re asking a vendor to create a Web site or develop your e-commerce strategy. “If you look at code per person, the numerator is the output and the denominator is cost,” says Michael Mah, managing partner of Pittsfield, Mass.-based QSM Associates and senior consultant on the Cutter Consortium, Sourcing Advisory Service. (QSMA maintains databases of outsourcing contracts and the metrics used to measure the success of these arrangements. The consortium, part of Arlington, Mass.-based Cutter Information Corp., provides IT advisory and publication services.) Speed, a crucial element in today’s competitive environment, is not a factor in the ratio, Mah says. Companies are outsourcing to obtain skills that they don’t have in-house or would take too long to build up, he notes. “They’re looking to buy off the shelf and get to market faster,” Mah says.

These warm and fuzzy terms also come into play because outsourcing vendors are increasingly being judged by business results, notes Bruce Caldwell, a senior outsourcing analyst at Gartner Group Inc., in Stamford, Conn. “Metrics aren’t just faster, better, and cheaper, they’re changing the way companies take care of business, so the outcome is improved over the process,” says Caldwell, who is based in Riverhead, N.Y.

“IT outsourcing relationships have evolved from strictly a client-vendor relationship to more of a partnership,” says IDC’s Doyle. Metrics are now linked to strategic business value and customer satisfaction, she notes. “You can recognize business value through increased productivity or new business opportunities, and increased shareholder value,” she says. “These metrics are very different from traditional outsourcing metrics.”

Business metrics come into play in a variety of outsourcing deals, including business process outsourcing, and are harder to measure. Which cost savings do you measure? Do you include only IT cost savings? In terms of generating revenue, do you give the outsourcer credit for being innovative or simply look at the extra dollars generated?

BP Amoco p.l.c.’s deal with Andersen Consulting, signed in January 2000, is one example of a contract where the vendor will be judged based on more qualitative measures. The $200-million outsourcing agreement calls for BP Amoco’s downstream business to outsource its finance and administrative services to AC. As part of the 10-year agreement, AC will assist BP Amoco’s downstream business in moving manual processes toward electronic commerce. The downstream business accounts for all activities after oil is found, such as distribution, sales, and marketing.

BP Amoco will judge Andersen Consulting on how innovative it is, according to Gartner’s Caldwell. Andersen is “contractually obligated” to bring new ideas to BP Amoco, he says. The ideas need to focus on organizational structure and business processes within the finance and accounting operations. A governance body will ensure the delivery of the “innovation quotient” through a quarterly review, Caldwell notes. The contract also contains a risk/reward component.

Jim Dewar, the Chicago-based commercial manager in BP Amoco’s downstream business, says AC will indeed benefit from using innovative business practices that ultimately drive down costs. In particular, the contract stipulates AC needs to demonstrate innovation in creating Web-based strategies. “We’re looking to move to an e-world, wherever possible, that will facilitate cost savings, but not at the expense of service level delivery,” Dewar says.

Although AC’s global-managing partner for business-process management, Hugh Morris, declined to discuss the specifics of the BP Amoco arrangement, he discussed the terms of a business process and IT outsourcing deal the firm has with another company. AC is evaluated on a 100-point scale. Innovativeness accounts for 20 of those points. The deal is evaluated quarterly, but an annual evaluation determines the overall scoring in all categories and, ultimately, AC’s economic benefit. Morris says the interim and annual results are expected to be similar. AC wins all 20 points in the innovation category by coming up with business-process improvements, says Morris, based in London. In other words, the client will look at the number of new ideas Andersen Consulting generates in a period, how many ideas the client accepts, and the improvements or savings resulting from the ideas.

Andersen Consulting receives a fixed percentage of any savings during the first two years after the firm generates an idea. AC receives a greater percentage of the savings in year one than year two, Morris says. In addition, the contract has an incentive for AC to come in at or below the financial budget for the deal overall in any given year.

In an IT outsourcing arrangement with a United Kingdom-based utility, AC can receive up to 25 points (out of a total 120) for being flexible. How does AC score 100% in this category? “If the client came to us with a need, did we say ‘this is not in the contract,’ or did we say ‘how can we help’?” says Morris.

In other deals, AC is evaluated on responsiveness. “If somebody calls and says ‘I need help urgently,’ how quickly did we respond? Did we say there were 14 pieces of paper to fill in or that we would be there at 2 p.m.?” Morris says. The quality of the working relationship is also important. It’s evaluated by how respectfully the AC staff treats the client’s employees, and how committed AC professionals are to the client’s concerns, as opposed to their own.

Mixing the two

Morris notes that outsourcing arrangements generally include a mix of quantitative and qualitative metrics. When it comes to the latter, however, Morris says the clients use subjective measures rather than a formula to determine the number of points awarded to Andersen. “AC perceives value in its own way,” he says. “We might suggest that certain activities demonstrate the values they want from us.”

Customer satisfaction is another qualitative measure outsourcers are judged upon–in other words, is the client’s customer satisfied? The percentage improvement in customer satisfaction is being used as a metric, says Barbara Melby, an outsourcing attorney in the Washington, D.C., office of New York City law firm Milbank Tweed Hadley & McCloy, LLP.

One corporate IT employee who wishes to remain anonymous rates customer satisfaction as a top qualitative concern for her company. “It’s difficult to sit with a customer and explain quantitative measures if it affects them personally,” she says. “For example, if our customers place a call and they don’t get the response they want. Even if my outsourcing provider is hitting the mark 99% of the time, those quantitative measures mean nothing to our customer because they were not serviced at the time.”

The company is looking for more proactive partnerships with outsourcers. “We want a partner to deliver services that meet our customer’s need. We want them to be flexible and innovative in bringing the ideas to the table,” the corporate IT employee says.

Despite all the hype qualitative and business-results measures are receiving, quantitative measures are still important. All sources interviewed for this story say that their arrangements include quantitative measures as well. “However, with Web-based technology, the qualitative measures are more important than the quantitative,” says the IT employee.

“Qualitative gauges have a role on performance,” says QSMA’s Mah. “But there are also going to be a lot of hard numbers. When lawyers establish goals, the hard numbers generate commitment.” //

Lisa Gandy Wargo is a freelance writer living in Atlanta. She can be reached at [email protected].