Case Study: Test Automation Feasibility and Architecture

Challenge: This company from the insurance industry wanted to determine the feasibility of automating smoke and regression testing for a major application undergoing an upgrade from AIX database to SQL Server 2008 and from eWAM framework version 4.x to 5.x. A prior automation evaluation had concluded that HP QuickTest Professional (QTP) could not be used because of some of the application’s objects. This client was already using QTP for other applications’ test automation.

Approach: Our test automation feasibility study included reviewing the application’s objects, customization, manual test cases, test environments, and data refresh. We also reviewed QTP’s ability to support the application’s custom objects, dynamic objects and data, and exception handling. When reviewing the application, we determined that around 80% of the application’s objects were standard Windows objects, which can be recognized by most test automation tools. Approximately 20% were custom Windows objects defined by the developers (e.g. grid, tab, menu, etc.).

Based on the findings and that QTP licenses were already owned, we recommended using QTP and developing workarounds for the custom Windows objects. Other test automation tools have the same custom object recognition issue. To jump-start workarounds, we provided high-level steps that could be implemented in QTP to work with the customized objects.

Since successful test automation entails more than selecting an automation tool, we also designed a test automation architecture based on 15 factors that impact the short and long-term reusability and cost of test automation maintenance. A critical architecture component is the best fit test automation approach choosing from five common test automation approaches – record/playback, data-driven, modular, keyword/action-based, and database. We also recommended an automation test data strategy, test automation phases and activities, and resource skill sets needed.

Result: A cost effective approach for automated smoke and regression testing for the application was provided that used the existing investment in QuickTest Professional (QTP).  The erroneous conclusion of a previous test automation evaluation was dispelled. The application team gained better insight into QTP capabilities, learned about test automation approaches, and obtained the information that they needed to prepare a cost benefit analysis for test automation.