Virtual stress testing

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

What do you do when you’re charged with implementing a test strategy to assure the quality and availability of a very large, very complex Web site? You turn to automation, of course, just like Richard Lacroix of The Manufacturers Life Insurance Company (Manulife Financial) did.

By adopting one of a new class of stress-testing tools that are specifically designed to simulate expected Web traffic, you can ensure that the entire back-end infrastructure can deliver accurate results in a timely manner.

When Lacroix joined the test team of Manulife, a Toronto-based firm that provides financial protection products and investment management services, in Aug. 1998, he faced early demand projections of 2,000 concurrent users. And he quickly realized that the required user volume testing effort far exceeded what could be accomplished with traditional manual testing.

This is not a unique scenario. In the past, IT could count terminals and desktops to determine the maximum user volume. Today’s Internet access has blown the roof off the upper limit, with potentially unlimited demand on system resources. But by adopting LoadRunner from Mercury Interactive Corp., Performance Studio from Rational Software Corp., or Silk Performer from Segue Software Inc.–products in a new class of stress-testing tools that are specifically designed to simulate expected Web traffic, you can ensure that the entire back-end infrastructure can deliver accurate results in a timely manner.

How important is an accurate performance appraisal? Just let an e-trading site go down, and you’ve got material for the front page and the network news. What’s especially painful about Internet performance missteps is that they’re brought on by success: Many failures are related to skyrocketing demand. A $20 billion e-commerce market today is projected to be worth $300 billion within three years and in the trillions within a year or two after that, according to industry analysts.

Choosing and implementing the best tool

Manulife operates in 13 countries worldwide, with approximately 20,000 employees and agents managing almost a hundred billion dollars. The company launched a new project that would allow its members and participants to perform account balance inquiries, issue investment instructions, and make inter-account transfers on its Web site over a secure Internet connection. The necessary application relied on a development environment with multiple layers, including IBM’s Websphere and MQSeries, along with its Lotus Notes and Domino products, all funneling into an IBM mainframe and DB2 database.

In a stress-testing product Lacroix required the automation tool to support 100% interoperability with the Manulife development environment and to properly handle SSL v3. Lacroix also needed a tool that allowed the technical QA staff to initialize a test bed and then hand it off to a team of nontechnical staff to run against daily content or code changes.

Manulife chose the e-TESTER and e-LOAD products from RSW Software Inc., in Boston. e-TESTER creates test scripts, which drive typical user interactions, and e-LOAD generates traffic based on test profiles. Lacroix was delighted to discover that his company’s chosen vendor practiced what it preached. “After expressing interest in the RSW solution, I had a copy of the software on my desk within 24 hours and the RSW technical team at the ready to assist in answering my questions as I test drove the products,” he says. “I installed the product, and within 25 minutes I was running load tests with 400 virtual users.” And, as the Jan. 1999 deadline loomed, RSW was available 24×7 to address technical support issues. Welcome to Internet time.

Mainframes have become the spider at the center of the Web. In fact, mainframes themselves are being multiplexed to accommodate the astounding volumes that e-business is generating.

The project made its deadline, and going forward Lacroix and his colleagues at Manulife can now rest easier knowing that the test effort is consistent, predictable, and rapid enough to meet the pace of cyberspace. And, for future verification of the inevitable enhancements, Lacroix wisely recognized that creating a test library is a development effort. As a result, he apportioned the work between the technical staff–for development–and the nontechnical staff–for production use. Thus, it does not require technical expertise to execute tests on an ongoing basis.

Scaling for the future

It’s interesting to note that Manulife was able to predict its peak load demand based on its customer and participant inventory. But even at a relatively low 2,000 users, the RSW technology required about seven Windows NT machines with high-speed processors and lots of memory to generate sufficient traffic.

Although this was significantly less hardware than the other vendor offerings Manulife considered, it still raises several disturbing questions: What do you do if you expect to have 200,000, or two million, concurrent users? Are automated stress-testing tools scalable? RSW is confident it can keep pace, but can the hardware platforms? No one knows for sure.

There is an irony buried in here. Obviously, the demise of the mainframe in the 1980s was prematurely announced. Instead of being replaced by client/server systems and now the Internet, mainframes have become the spider at the center of the Web. In fact, mainframes themselves are being multiplexed to accommodate the astounding volumes that e-business is generating.

Viewed this way, the mainframe is the nerve center and the point at which all channels converge–whether through the Internet, interactive voice response (IVR), electronic data interchange (EDI), or desktops and terminals–and therefore the ultimate choke point. While the Internet no doubt represents the most dramatic pace of growth, these other channels also compete for the same resources. How can you simulate them all? //

Linda Hayes As always, our problems continually leapfrog our solutions. Has your organization uncovered a choke point? Write me at linda@worksoft.com.

Linda Hayes is CEO of WorkSoft Inc. She was one of the founders of AutoTester. She can be reached at linda@worksoft.com.