Quality Quest: I Want My Test API

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More .

Linda G. Hayes

You’re trying to automate application testing. Unfortunately, nonstandard objects that your test tool can’t recognize are torturing you. And don’t forget the objects that the test tool can’t remember. If you have ever found yourself in this situation, I want to challenge your thinking.

The test tool–the obvious culprit–is actually not the guilty party. The problem is that applications are not designed to be testable. They operate within a private process space in memory and do not offer an interface that enables test tools to interact with application components during execution. Without access to application components, test tools must be inserted into the messaging layer to communicate, or into the common object model (COM) layer or even device drivers when applications don’t employ standard objects and messaging. The problem is the lower the test tool level, the less available information about the application. This makes test scripts arcane and unstable.

One way to resolve the problem of access to application components during test execution is to load an interface to the test tool via an implant in the application’s source code. This small piece of code, once inserted, can pass information back and forth between the application and the test tool.

While this is a rather straightforward solution, the mere mention of inserting anything into the code for test purposes is met with immediate–and often effective–resistance from software developers.

So why is software testability treated as optional and fiercely resisted? And what can you do about it?

Fear and Loathing

Software developers’ first objection to source code implants is the inserted code may alter the behavior of the application, thus assailing the integrity of the test. This position is hard to defend. The inserted code performs no task other than to give the test tool a window into the application’s processing space. Without clear logic or proof that the functionality of the application is compromised, the risk is remote compared to the benefits of automation.

The second objection from developers is inserted code must be removed for the production version. However, because a secondary software compile must occur, the test is invalidated. This defense also assumes that the inserted code can’t be shipped, which is not necessarily true. An execution option that enables or disables the interface can resolve any potential security problem in production. During test execution the interface is enabled, during production it is not. Another objection I have heard is compiling the implant code into the source code will require extra time. Most test tool automation provides either a single insertion point for the code or an automated way of proliferating it. The initial effort is not time consuming. And, since most commercial applications are compiled from source code management tools, the incremental effort of including a single piece of code in the software build is inconsequential time-wise.

The point is not to expend creative energy trying to imagine or discover all the reasons why a test interface won’t workit is to invest time in making the test work. The benefits of automation are too valuable to be lost.

Hope and Planning

In my view, it is impossible to keep up with the rate of software development using manual test approaches. Component-based development allows the assembly of powerful, complex applications in record time. Unfortunately, there is no offsetting relief for software testing. All functionality must be tested, whether purchased or coded. Without automation, reliability is always going to be at risk.

So the question is not whether a source code implant should be available for test automation, but how it should be integrated into the test process. If software developers are suspicious of inserting a foreign piece of code, then let them write their own.

The benefit of a test API is that your test tool and your application start working as a team. Once this happens, you can shift your test automation efforts from the struggle for object recognition to the design of highly effective tests. This is what automation is supposed to do–it should make you more productive, not distract you with yet another set of intractable test tool problems.

Imagine if your application came pre-wired for testing, just like your house is pre-wired for electricity or cable. You just plug in. Now imagine that this application wiring would establish two-way communication with your test tool, allowing the test tool to interact with the application at run-time, and also keep it informed about changes that may affect your tests. Xanadu!

This test scenario does not have to be a faint hope or farfetched vision. It is completely possible for a test API to be wired into any application, at any level. You only need the willingness to invest the time and effort to define and implement it. //

Linda Hayes is CEO of WorkSoft Inc. She was one of the founders of AutoTester. She can be reached at linda@worksoft.com.