Management-friendly test data: Page 2

(Page 2 of 2)

The most eloquent--and dramatic--example of this was when the new CEO of a software company asked me to review its operations to help him discover the reason why costs were increasing and revenues decreasing. After a couple of days of interviews, the mystery was solved. The company's budget for customer support was more than the budget for development, testing, training, and documentation combined.

Why was support so expensive? As the harried customer support manager explained it, the company had a huge backlog of bugs, some of which were actually years old, and these generated thousands of phone calls which her team was obliged to field.

Time and money wasted in support and maintenance on poor quality software could be reinvested in more resources to deliver new, high-quality products on time.
Why so many bugs? Because there weren't enough developers to maintain the products. Why not enough developers? Because there wasn't enough money to hire more. Why not enough money? Because support was so expensive. Why not increase revenues? Because they didn't have enough developers to create new products.

Get it? So did the CEO.

How do you say it?

It all comes down to this: How do you measure this phenomenon and communicate it in such a way as to have management understand and--most importantly--care?

Unfortunately, I cannot point to a magic answer. If you have one, send me an e-mail and I'll tell the world.

Until then, do the only thing you can do: Correlate all of your metrics into time and money. At a minimum, track the issues that arise after shipment and correlate them to the ones you either knew you shipped or could have predicted because you didn't finish testing. Remember, the number of reported problems is the number of defects multiplied by the number of customers who find them. So, shipping a single known defect can cause hundreds of problem reports.

The next trick is to convert this information into time and money. Money can be determined if you know the following:

What the budget is for customer support and maintenance;

How many problem reports are fielded.

And how many fixes were made.

Time is harder to convert, of course, because you have to know how long it takes to field a call, make a fix, test it, and ship it. If you have a robust problem-tracking system you may have this information. If you don't, add up the manpower spent in maintenance and support and convert that into time.

Be sure to make the point that the time and money wasted in support and maintenance on poor quality software could be reinvested in more resources to deliver new, high-quality products on time.

The point

The real point, of course, is to understand your audience. Management cares about time and money, in that order. Present your metrics in such a way that you can correlate what you are measuring to what it costs--or saves--in time and money. All of the graphs, charts, and tables in the world won't matter without metrics that make sense. //

Linda Hayes is CEO of WorkSoft Inc.. She was one of the founders of AutoTester. She can be reached at

Page 2 of 2

Previous Page
1 2

0 Comments (click to add your comment)
Comment and Contribute


(Maximum characters: 1200). You have characters left.