Stressed out from stress testing: Page 4

Posted October 26, 1999

Rich Levin

(Page 4 of 5)

Risk avoidance

Flow says that, without the availability of Web-savvy integration testing tools, Frontier's entire application portfolio would be at risk. "If we didn't have these automated tools, we simply couldn't do the testing," he says. "We'd be in a world of hurt right now."

Certainly automated testing tools can ease the pain of integration testing and help ensure a site's ability to withstand heavy user loads, the likes of which no legacy IT app has ever been asked to sustain.

But not one of the automated testing tools available today can replace the need to beta test, using qualified users culled from the application's target audience. It's the only engineering process known that can isolate bad user interfaces.

It's also a process that ISVs live and die by, and that IT has historically sworn off. But that too is changing as IT organizations rise to meet the challenges, and reap the rewards, of today's wired world.

"As a [testing tool] vendor, I hate to say that our tools can't perform a certain function, but the truth is, usability testing is the one thing no automated tool can do," says Diane Hagglund, senior manager for e-business product marketing at Mercury Interactive.

Hagglund says usability testing might never be automated, because it has to do with responding to human emotions--something that has yet to be computerized. "We're seeing more and more traditional IT shops doing what ISVs would call beta testing, under the guise of usability testing," she says.

Banking on the beta

That's exactly what's happening at Acentris Wireless Communications, a telco services reseller in Seattle. There the beta-test process has been integrated into the overall development lifecycle, with a core group of developers, internal users, and customers comprising Acentris' beta test team.

The company recently migrated from its legacy VB4 client/server system to a fully distributed platform. The new system is built in VB6 and leverages several beta technologies itself, including a COM+ framework and Windows 2000 Beta 3 RC1 servers.

"We prototyped the Web UI first, and sent it out to a small group of customers and internal users for beta testing," says Acentris VP Darren Lang. "That gave us a huge head start, because we were able to fine-tune the user experience and hand the UI off to the programmers early in the development process."

Acentris' development team was then free to focus on the migration's nuts and bolts, and use automated tools to stress and regression test the application architecture, knowing usability was already in hand.

"The reputation of the IT department no longer rides on how well they manage the printers, back up the servers, or get a new PC on your desk," says Michael Marquardt, president of Internet Operations Center Inc., an e-commerce application hosting company in Southfield, Mich. "It's now the software development arm of the business, and that means we need to think and act more like ISVs, and less like islands of technology." IJ

About the author:

Rich Levin covers IT for CBS Radio and the Coast to Coast Radio Network. He can be reached at, hit his Web site at, or visit his BBS at

Lessons learned about automated testing technologies
  • Think like an ISV. Recognize that development in the e-business age requires IT organizations to conceptualize systems as commercial applications. To maximize application system quality, adopt proven best practices long used by successful ISVs, such as heavy use of automated testing tools, technologies, and development methodologies.

  • Test early and often. Integrate your QA testing team into the development process from day one. The more the testers know about the application requirements, coding, and resulting app, the better they'll be able to devise test scenarios and regression scripts.

  • Prototype the user interface first. Send the resulting UI demo out to customers, partners, and other users for early feedback. By defining the UI early, the IT team eliminates the possibility of show-stopping usability bugs late in the development lifecycle.

  • Don't overlook load testing. Applications that perform well under low to moderate loads can behave unpredictably, or even implode, when stress scales up. Use an automated load-testing tool to stress application architectures two or three times beyond what you expect the actual user load to be.

  • Test the application's technology as well as its logic. Your application code might be robust, but a single bug in your vendor's app server or OS can bring your organization to its knees.

  • Build a multidisciplinary QA team. Software quality assurance today involves far more than GUI regression and load testing. Web applications are inherently complex, leveraging multiple new technologies and heterogeneous legacy systems. QA teams need to test integration points, database integrity, object messages and communications, and more.

  • Evaluate all the major test tool product lines. Automated testing tools have come a long way in a short time. Most vendors have completely updated their offerings to address the requirements of e-business development efforts.

  • Have contingency plans. Even applications subjected to the most rigorous QA processes can fail. Be sure your IT team is prepared for a major system failure, and have a disaster plan in place to bring systems back up as quickly as possible. Test the contingency plans often with drills.

Page 4 of 5

Previous Page
1 2 3 4 5
Next Page

Comment and Contribute


(Maximum characters: 1200). You have characters left.