Stress for sale

One of the biggest trends in software lately is the shift toward application service providers (ASPs)--vendors that deliver their applications over the Internet. That is, instead of investing in the infrastructure of software needed to run a system--servers, databases, networks, and so forth--customers simply pay a fee to access the same software over the Internet. It's a great idea, really, because it allows customers to focus on their own core business instead of having to deal with the IT demands of an internal infrastructure.

That same concept is now being applied to testing: specifically, to the stress testing of Web sites. For example, Mercury Interactive Corp., a Sunnyvale, Calif., test tools vendor, was the first to announce the availability of a remote-hosted traffic service to test the performance and capacity of a company's Web site. While Mercury is currently the only company offering these services, others are poised to enter the market.

What does it all mean?

Stress circus

Stress testing has always been, well, stressful. The objective is to create enough user and transaction volume on a system to measure its capacity and performance and to determine if it will stand up to real-world demands. Many a company has launched a new application that was functionally robust, only to have it fail because of unacceptable response times, or because it simply could not support the number of users who needed access to it.

The brute-force approach to stress testing is to invite a large population of users in on a weekend or holiday and have them bang away at your site. This process is cumbersome and expensive for obvious reasons, not the least of which is the difficulty of coordinating user activity so that the actual breakpoints can be identified. The next-generation solution is software that creates virtual users by simulating the activity of a number of terminals or workstations. This process is much better because it reduces the required manpower and can be fine-tuned to measure gradual increments of traffic.

But even the software approach is hard. The tools themselves are arcane, the scripts are complicated to write, and it requires a lot of hardware to generate enough traffic for a meaningful test. Many a company has made a major investment in software, servers, and consultants to set up and execute a stress test that only runs for a few hours. This is like staging a Broadway show that closes after one performance, only to be re-staged when something changes and the whole production has to be re-tested.