Quality assurance for systems integration
From Wikipractice
Contents |
Introduction
When organizations merge, integrating the applications that power them is often arduous. Proper quality assurance methods can help assure that your next integration project is headache free.
When the economy is roaring, companies acquire other companies as a growth strategy. When the economy is lean, corporations downsize and merge departments to save money. But no matter what the state of the economy, businesses will continue to restructure, merge, and consolidate. When they do, the applications that power those businesses must integrate. After all, how can two teams be expected to enjoy the benefits of cooperation if they aren't even talking?
Whenever companies undergo a large-scale integration, ensuring that the systems suffer no negative impact is critical. That's where quality assurance (QA) comes in. William Hetzel offered one of the earliest definitions of the QA process in his book Program Test Methods. He described software quality assurance as "establishing confidence that a program does what it is supposed to do."
Throughout the integration testing of data, systems, and user interfaces, Hetzel's definition holds true. We want to be confident of the data conversion program, the integrated applications, and the combined user interface of our merged applications.
To illustrate how the QA process works, let's look at a hypothetical integration project. Suppose two e-commerce portals, Alpha Widgets and Beta Widgets, are combining into a single site. How can these two companies be sure that their combined sites continue to operate at peak efficiency, with no compromised functions and no new bugs introduced as a product of the integration effort?
The first step is to involve QA analysts. As soon as site engineers have completed the integration in a staging environment, but before the changes are deployed on the live site, it's time for an analyst to validate the results. This is known as integration testing, which is the process of verifying proper functionality between two or more modules or components.
Database Validation
The database is a good place to start, because nearly every integration involves changing the underlying data structures. Integrating data is an effective way to avoid duplication in the business process. Some data might be in both databases, and other data might only be in one. Having two separate databases could result in situations such as assigning two account representatives to the same customer, sending a customer two bills for the same order, seemingly missing data, or underreporting inventory.
In very rare cases, the two sets of data are set up identically, and one set can be imported into the other without modification. This could be the case if both systems are using a commercial off-the-shelf application, or a standard like XML. However, even when industry standards are followed for record formatting, it's rare for two systems to store every column, row, and table identically in their databases.
To integrate the back-end data, a conversion program must often translate the data from one application into the proper format for the other. When validating proper data integration, the tester typically performs system testing on the conversion program to validate that it enforces the data rules, business rules, and system requirements for the application.
In addition to formatting, the conversion program must also consider the business rules and system requirements for each application. In our example, the two datasets will be merged into a single inventory. Data from Beta Widgets will be converted and imported into the database used by Alpha Widgets. Alpha Widgets has a business rule that requires a telephone number for each customer record, so the conversion program should check the customers from Beta Widgets for phone numbers before simply importing the records. Any record that doesn't follow the business rules should be noted, so that someone can resolve those issues later.
Data Validation Process
Ideally, application data from the production environment should be used to perform validation in the test environment. This provides an accurate reflection of the data range that the conversion program will encounter. If actual data cannot be used, then the tester must create a set of data that properly exercises the boundaries of the business rules.
In our hypothetical scenario, if the maximum length of the Last Name field is 40 characters, then data used during the test should probably include instances where the Last Name is blank, one character, 40 characters, 41 characters, and 150 characters. This would cover scenarios where the field is null, very small, in range, out of range, and very large. After the conversion program has translated some test data, the analyst can run queries to quickly parse the database and examine the results from a high level.
For example, let's say the Alpha Widgets database is set up with one record per physical product, while the Beta Widgets database has one record per product title with a row that indicates the quantity. After data is converted from Beta to Alpha, the analyst could run a query comparing the count of rows in the Alpha database to the sum of the quantity fields of the imported data from Beta. If there is a discrepancy, then the conversion program failed to import all of the data.
With small conversions, the data can be validated at the row, field, or character level. However, if the volume of data is larger, it can be impractical to manually validate all of the data by hand. In these cases, automated testing may be desirable. This can include data comparison tools, scripts created in record/playback tools, or statistical analysis software that can analyze trends in the data.
Every good data conversion process should include a rollback method, which should also be tested. What happens if something goes wrong during the implementation, or if problems are reported afterward? In such an emergency, the system must be able to restore itself to its original state before the conversion. Remember, the two separate systems will probably still work on their own. Rolling back to two systems that work is better than having an integrated system that doesn't.
When verifying this part of the process, you should test several conversions and rollbacks to ensure data integrity. Additionally, if partial rollback is allowed, test the process for proper version management. For example, you could convert some data, rollback a portion of that data, then convert a set of data that includes some new data and some previously rolled back data. Lather, rinse, repeat.
Finally, in some cases, not all of the data will be brought over from the previous database. The tester will need to ensure that the conversion program didn't migrate any unwanted data. Some data will be missing from the old database, so fields will be blank when the records are migrated. Sometimes data will be converted while the old system is still active, causing issues with data version management. All of these deviations from a full data conversion should be specified in the requirements, and then covered in the conversion program validation effort.
Application Validation
Besides changes to the database, many integrations also require you to combine existing applications. There are several ways to achieve this, and often they can be implemented incrementally. For example, the first phase of an integration effort might be to add communication between the existing systems for data integrity. The second phase might be to integrate a component from one system into the other. A final phase may be to completely decommission one system and move all users to the new system.
No matter what methods are used to integrate two applications, the test approach is similar. The analyst shouldn't test all of the functionality in each component. The purpose of integration testing is to verify proper functionality between components, not to retest every combination of lower-level functionality. The correct procedure is to test only those unique messages or actions that pass between the components, as well as any new code that's developed for the interface.
If the chosen approach is a new system that integrates the requirements from each of the two old systems, then the system test effort will effectively be the same as it would be for a system built from scratch. To ensure that the systems are properly merged, do the bulk of the analysis during the requirements verification phase (see the sidebar, "Requirements Verification," for more information). Give special attention to identifying mutually exclusive or inconsistent requirements, and to resolving any conflicts.
Once the two original systems are merged, use the requirements to test the new system. If the requirements specify that the system should perform in a similar manner to the previous application, then compare the previous application side-by-side with the new one. You can retrieve accounts, process test orders, and generate bills in both systems and compare them to ensure that the system change doesn't impact the customer.
Transitioning to a Single System
In our example, Alpha Widgets and Beta Widgets both work with similar products. For a quicker integration, the companies have decided not to merge their software, but instead to switch everything over to the Alpha Widgets application. However, the Beta Widgets application lets the user query for customersby product. This was critical to how Beta employees worked, so this functionality will be added to the Alpha Widgets application.
Additional changes may also be necessary to support the Beta Widgets user interface. In this case, the test effort will follow the normal procedure for any system enhancements. System requirements should specify the changes to the system and tests should be designed based on the requirements.
Again, side-by-side analysis of the old and new systems can help minimize risk by deploying the new system in a production environment. In this case, the newly integrated Alpha Widgets application (loaded with both the Alpha Widgets and Beta Widgets data) can be tested alongside the old Beta Widgets system. Customer, product, and order records should all be available, and should function similarly in both systems.
The tester in our hypothetical example would validate that the query for customersby product returns the proper values for Beta Widgets data by comparing it to the results of the same query in the old Beta Widgets system. The same tests can then be run on the Alpha Widgets data, to ensure that the query works with that data as well.
User Interface Validation
If the end result of an integration effort is a single application, then the system will generally have a single user interface incorporating data fields from both original applications. If the end result is two applications—where the integration just adds communication between the separate applications—then the user interfaces still may require changes. Additional form fields may be added to reflect the data passing between the two systems, as well as to display messages based on the data results.
Either way, the user interface should be as much a part of the QA process as the data and applications. When validating proper integration of the user interfaces, testers should again draw upon the requirements as the basis for their work. Each field should be checked for valid values and data types. Information returned from the system should be validated as well. The user interface is very often where the data conversion and application inconsistencies will manifest themselves.
For our example, the Beta Widgets Web site receives more traffic, so the companies decide that its design should become the primary user interface. Additionally, they plan to add a mobile interface to support their two sales teams when they're on the road.
To properly test this design, the Beta Widgets user interface must be tested to make sure it has the proper input fields to drive the Alpha application. Tests should be run based on both Alpha and Beta data. Additionally, the new mobile interface must be tested to ensure access to both sets of data.
Before the data is converted, the applications are altered, or the user interfaces are adjusted in the production environment, it's generally a good idea to walk through the changes with a user acceptance test. In this test, users work with the user interface to query data and perform functions in the test environment. The users, after receiving training on the functional changes to the application, should also be able to determine whether the altered system still meets their needs and assists them with their jobs.
Post-Implementation Validation
Once the changes have been thoroughly tested and all of the defined exit criteria have been met, the changes are ready to be implemented in the production environment. However, the quality assurance team should still validate the changes after they've been implemented. The key to post-implementation validation is to be nonpervasive. The testers should try to leave as small of a footprint in production as possible, so that their work doesn't affect live customer data, incur bogus charges, or skew reporting.
Appropriate tests include running high-level comparison queries or statistical analysis on the data. Testers can ensure that the appropriate files have been added or removed (for added or decommissioned components), and that proper versions of those files have been migrated. User interfaces can be validated by navigating the screens that don't affect live data.
Another useful tool is post-implementation user acceptance validation in the live production environment. In this test, real users are supposed to alter the data, so they can perform their work without worrying about accidentally affecting something they shouldn't. Meanwhile, the test team monitors users' actions, and their feedback can be used to evaluate the results of the implementation. The test team can continue to monitor production usage, even after this limited observation period, to ensure that the integration was successful.
Going Live
Ideally, once the QA process is complete and the new site has gone live, your customers shouldn't even realize that two companies have merged until they receive a courtesy letter with the new letterhead. On the other hand, the consequences of improperly integrating your systems can be dangerous, with problems ranging from erroneous billing to inaccurate inventories or customer data.
Any of these has the potential to cause a public relations nightmare. Worse, reacting to user feedback late in the game only leads to more system changes, which could end up costing far more than doing it right the first time.
First and foremost, "doing it right" means developing according to a well-defined process, rather than through trial and error. With a robust test plan that covers requirements verification, system validation, and a post-implementation walk-through, your team should be better prepared to manage the process. By observing correct quality assurance procedure throughout your integration project's life cycle, you should be better prepared to find high-risk problems before they affect your customers.
