SHARE

Quality Quest: The Irrational Ratio

Linda G. Hayes Do you remember learning about irrational numbers? I always pictured them as being somehow unreasonable, numbers you had to coax or cajole into behaving rationally. Actually, they are numbers that never end–no matter how far you carry them out–like pi. In that sense, the term is particularly apropos for describing the proper […]

Written By

Linda G. Hayes

Sep 1, 1998

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More

Linda G. Hayes

Do you remember learning about irrational numbers? I always pictured them as being somehow unreasonable, numbers you had to coax or cajole into behaving rationally. Actually, they are numbers that never end–no matter how far you carry them out–like pi. In that sense, the term is particularly apropos for describing the proper ratio between developers and testers, because the answer keeps changing the longer you look at it.

Not a conference or class goes by that the question is not asked: What’s the magic ratio? Some breathlessly report that they heard Microsoft has a one-to-one ratio, while others abjectly confess that their companies have 10 or more developers for every tester. So what’s the best answer? That’s easy. It depends.

The ideal ratio between developers and testers depends on the type of development and the phase of the project. In some cases, developers should outnumber testers, but in others–believe it or not–the testers should outnumber the developers. The secret is to know when and why to adjust the ratio.

Planning and Design

In the earliest stages of developing a new application–that is, one being written from scratch–developers should outnumber testers. Although test planning and design can and should commence during the planning and design phases of the development project, the fact is that there is simply more work to be done by developers. Until the design stabilizes, which in most cases happens during coding, testers can at best define and prepare the test environment and process.

The ideal ratio at this stage is probably one tester per team, or about one for every four or five developers. Most applications are built with a team this size; especially large applications might have multiple teams, but each team typically stays small. We have all learned the hard way that adding more people does not always add more productivity.

Testing Ratio Tricks and Techniques

If you follow these basic rules, you will be in a better position to effectively use temporary resources and/or employ automation in testing when it’s appropriate. These tricks and techniques may also help you achieve a more rational method for managing the ratio of developers to testers in application development.

Plan your test

Document your tests

Dumb-down your tests

Standardize your tests

There is an exception, however. In some cases, new applications are really just modifications of existing ones. For example, a retail data analysis firm developed six applications that were simply just special versions of the same system customized for different customers. The same application in other cases might be ported to execute on multiple platforms. The effort in these situations is more like maintenance and enhancement, so the rules are different.

Coding and Testing

As a new application completes the coding phase and moves into test, the ratio should start to shift in favor of testers. In fact, a ratio of one or two developers per tester is probably realistic.

This happens because the testers now have enough information to design, develop, and execute test cases. It takes time to review the application’s intricacies in order to design the proper tests, then more time to develop the tests, and then even more time to execute the tests and document the results. Then it takes yet more time to work closely with developers to resolve issues. The iterative retesting of corrections and changes adds substantially to the testing workload as the application nears release.

Developers, on the other hand, start to wind down in this phase, primarily addressing issues that arise, but–hopefully–not adding anything new. In fact, developers often start looking for a new challenge as the application moves inexorably toward that programmer purgatory known as maintenance.

Maintenance and Enhancement

This is the phase where the ratio usually gets out of whack. If I tell you that you need more testers than developers, you’ll think I’m crazy. But it’s true.

Look at it this way. Let’s say you have a 10,000-line application to which you’re adding some features as well as fixing some problems, and let’s estimate that you will add or change 25% of the code. So, from a development point of view, you have a quarter of the development effort originally required to create the application.

But from a testing point of view, you still have 100% of the code to test, because of the well-known phenomenon of unexpected impact from changes. An apparently tiny, isolated code change can have far-reaching effects. Examples abound, but the most famous is the single-line (some say single-character) aberration that brought down AT&T’s network in January 1990 and cost the company and its customers more than a billion dollars.

So the rule is that all features of an application, whether new or old, are tested every time there are any changes, no matter how theoretically minor. Testing old features is called “regression testing,” which refers to the problem of a prior capability being negatively affected by new features or modifications.

A similar situation arises when a new application is created by modifying an existing one, or by porting one to a new platform. From a development point of view, the level of change may be only a fraction of the original code, but from a testing point of view, it’s a completely new application.

What this all means is that it takes fewer developers to enhance or maintain an application, but the same number of testers as when it was brand new. In fact, it may take even more testers, since successive releases of applications almost always add new functionality without taking any away. //

Linda Hayes is CEO of WorkSoft Inc. She was one of the founders of AutoTester. She can be reached at linda@worksoft.com.