Two takes on preproduction testing
- 28 July, 2004 15:25
It's a hard-core developer's worst nightmare: You pour your heart and soul into a new project, double-checking every .Net I-interface and cross-checking every T-SQL statement only to discover -- post-deployment -- that a hidden flaw is killing your program's scalability. Management is breathing down your neck and you've got that uneasy feeling that the outsourcing vultures are beginning to circle, but every line of code looks as perfect as when you first crafted it.
Time to swallow your pride and do what the smart developers do: Seek professional help. Thanks to advances in Application Performance Management (APM) technology, the testing process needn't be painful. Innovative predeployment testing tools like Empirix's recently updated e-Test suite and Segue Software's SilkPerformer make constructing and maintaining even the most sophisticated app test scripts a breeze.
Empirix e-Test suite 7.0
Usability is always a key consideration in comparing software testing solutions. After spending months or even years implementing a project, the last thing most developers want is yet another API to master -- hence the popularity of visually oriented testing tools.
With an emphasis on scripting by example -- where you walk through a Web app using a recording applet to log each interaction between the client
and server -- e-Test suite makes it possible to test the most critical layers of your solution without writing a single line of scripting code.
The e-Tester console is the heart of e-Test suite. Built around a custom Web browser window, e-Tester wraps each page with a sophisticated script-recording framework that parses the HTTP stream and constructs a visual, node-based representation of the application flow. Key elements including form field post data, embedded script commands -- even the order in which links are accessed -- all are saved and exposed for review.
New in version 7.0 is the ability to parse .Net elements, including Microsoft Web forms. Add to this a more intelligent script-maintenance process (e-Tester can now detect and correct for many common application design changes), and you have a potent preproduction scripting environment.
Generating the test script is only the first part of the equation. You also need to replay the script as a workload simulation to validate the application, and this is where e-Test suite really shines. Thanks to tight integration between e-Tester and its companion tools, you can go from zero to a working simulation package in a few minutes.
Each tier provides a wide array of configuration options. With e-Test, your recorded script is the common thread that integrates the solution. Modifying your simulation then becomes a simple matter of tuning how each tier handles the script processing to render the desired effect.
E-Test suite is easy to use and tackles most common Web integration tasks with aplomb. However, as is often the case with visual development tools, there's a trade-off between usability and functionality, especially when it comes to programmability.
Although e-Tester translates the HTTP stream into a highly structured, visual representation of an application's flow, you still cannot directly edit the underlying script logic, though you can embed custom VBA (Visual Basic for Applications) code as part of a scripted page transition. Basically, what you see is what you get.
Then there's the issue of depth: e-Test suite is targeted primarily at first-tier Web apps. As such, it lacks the ability to dig deeper into the underlying architecture. E-Test suite does well with relatively generic HTML and HTTP functionality, but it cannot address the myriad midtier components (Oracle Applications, Siebel, PeopleSoft, SAP) that often comprise an enterprise application stack.
Segue SilkPerformer 6.5
If e-Test suite is the epitome of the lightweight, visual, no-coding-required paradigm, then Segue SilkPerformer is its polar opposite. SilkPerformer is an industrial-strength, preproduction testing platform that leverages its core asset - a rich scripting language - to provide access to all layers of the enterprise application stack.
Whereas e-Test is limited mostly to Web apps, SilkPerformer's support runs the gamut of major networked environments: Oracle Applications, SAP, PeopleSoft, Siebel Systems, and even Citrix Systems's MetaFrame. The modular SilkPerformer Recorder application features plug-ins for these and other popular architectures, allowing you to visually create scripts that span disparate compute models.
Not keen on learning a new programming language? Try the SilkPerformer add-in for Visual Studio .Net. Select the SilkPerformer option from the New Project wizard page and you're presented with a code skeleton in the language of your choice (C#, VB.Net, or Managed C++).
Later, when you're ready to test your project, the SilkPerfomer add-in dynamically generates the required BDL code and creates the supporting SilkPerfomer test project. It's the kind of seamless, do-it-your-way integration that many vendors promise but few actually deliver.
Aside from the rich scripting -- a major advantage over lighter-weight tools like the e-Test suite -- SilkPerformer provides a set of companion applications similar to e-Test's. Two highlights are the Performance Explorer, for building custom metrics tracking packages to accompany SilkPerfomer scripts, and the True Log Explorer, for advanced script-tuning and results analysis.
Finally, Segue provides a number of supporting components to help manage the larger application lifecycle process and large-scale testing. These include SilkCentral Performance Manager, which provides ongoing application health monitoring; SilkCentral Test Manager for allocating resources and constructing test plans; and the SilkTest automation tool for automating complex testing packages.
Tools to Test
Gone are the days of hand-coding validation routines; these preproduction testing suites turn seemingly insurmountable testing hurdles into a highly visual, point-and-click affair.
On one end of the spectrum we have e-Test suite, a lightweight, easy-to-use set of testing tools that lets you get up and running quickly, no coding required. On the opposite end, we have SilkPerformer, a classic everything-including-the-kitchen-sink solution that allows you to test virtually any type of app architecture while it delivers a user experience only slightly less approachable than e-Test.
Setting its Web application limitations aside, e-Test suite is a well-designed solution that's both intuitive to use and easy to deploy. At $20,000 for 50 concurrent virtual users, it's also priced significantly below its enterprise competitors, including Segue. E-Test is a good choice for developers seeking an entry-level testing solution that doesn't require writing custom script code.
Segue's Silk platform, however, provides one of the richest, most feature-complete testing environments available today. The ability to test middleware components from major vendors makes SilkPerformer a more capable solution. That, plus a native scripting language (with extensions into Visual Studio) are what sets it apart.
You can accomplish virtually any testing task with some combination of SilkPerformer and its sibling tools. In fact, the only real hurdle is the up-front acquisition cost: $30,000 for as many as 50 concurrent virtual users. Still, if you need the best application testing solution money can buy, SilkPerformer fits the bill nicely.