In the End, End-to-End is Everything

Enterprises of any significant size depend on extensive software portfolios to operate. I know of several who manage more than two hundred applications and at least one whose inventory exceeds 500. Granted, some of these are small departmental systems, but many support mission critical operations. And, of course, most of these are undergoing continuous changes in the form of corrections, enhancements and updates. This results in a constantly shifting production landscape.

Yet software testing disciplines are almost exclusively focused on individual applications within the context of a given project. Traditional V-model approaches track the software development process for a particular system, for example, and while they may touch upon interfaces it is only for those systems directly connected. Even user acceptance testing is usually organized and performed within silos, with each participant focused on his or her area of expertise.

But the reality is that many of these systems are interconnected. Even if there is no direct interface, they may share data that originates elsewhere and terminates somewhere else, perhaps several applications removed. Thus, a change to one area often has unforeseen effects in another, and generally accepted test practices are not designed to uncover them.

The result is that even a well-tested application can wreak unexpected havoc once in production, and this is what keeps management up at night.

The solution, obviously, is testing end-to-end. That is, following information all the way from its origination to its termination, as in Order to Cash, Procure to Pay, and Hire to Retire. And even among these mega processes there are interrelationships: raw materials are procured that are processed by plant employees into finished goods for sale. Extensive integration is both the benefit and the curse of enterprise applications.

So why isn’t end-to-end testing widely practiced?

First, because it’s complicated. The fact is that enterprises are organized around functional silos and few, if any, truly understand all of the interconnections and relationships among them. Not everyone may even be aware of the existence of some systems, let alone how they affect, or are affected by, others. It takes a lot of diligence to identify the right end-to-end scenarios, then ferret out the right resources for each silo, then coordinate them all into a meaningful workflow.

Second, because it’s hard. Effective end-to-end testing requires a robust and controlled test environment that can reproduce – or at least simulate – production. This means installing the vast majority of the software portfolio and providing for both internal and external interfaces, either live or simulated. It also requires provisioning test data that is both comprehensive and coherent so that the flow of data from one end to the other is consistent. None of this is easy.

And, finally, it takes a lot of time. Just the logistics of coordinating each functional resource to perform their respective tasks in the right order is time-consuming. In one case it took five calendar days to execute one cycle of Order to Cash, most of it lost to the hand-off between people from role to role. This is often the showstopper, because once an application has completed its project and related testing, it is usually on a fast track to production. Or, even worse, the change is actually a fix to a problem that has created an emergency. But whatever the reason, changes may be made daily and typical end-to-end testing takes far longer. The result is that most end-to-end issues aren’t discovered until production.

But it doesn’t have to be that way.

End-to-end testing can be achieved and performed regularly, even daily. The key is to adopt the right strategy, implement the right tools, and enlist the right team.

Adopting the right strategy means including end-to-end testing as part of an overarching approach for every application. This means doing the analysis to understand the sources and uses of data for any application to reveal both immediate and far-flung interfaces. It also means applying risk analysis to understand which systems have the potential for critical operational impact, and making sure the test repository is kept current and updated as changes are made.

Implementing the right tools means deploying an automation solution that enables extensive end-to-end testing to run lights-out every night if necessary. In the above example of Order to Cash taking five days to execute manually, the same process was automated using Worksoft Certify…and it executes in less than a minute. How? No wait time, think time or hand-off time. Just rapid, error-free execution that is fully documented. This type of speed means hundreds of extensive end-to-end tests can be executed for each and every change before it reaches production.

And automation isn’t just for execution. Just knowing what changes have been made and what impact they may have on critical business processes is itself a challenging task. Wading through arcane release notes or change notifications or IT tickets is tedious enough, but even assuming you can understand what they say it doesn’t mean you can tell what the impact might be. Solutions like the Worksoft Transport Analyzer can simplify and accelerate the entire change impact analysis process so you know for sure what to test.

Enlisting the right team means not just finding and recruiting the right experts for each functional silo, but also assembling a centralized test team that can coordinate the human, hardware and software resources necessary to make end-to-end testing a reality.

Granted, even with these factors in place it won’t happen overnight. The key is to remember that even a little end-to-end testing is better than none, and all the time you will save from avoiding even one production crisis can be reinvested to continually expand your scope. With patience, effort and commitment you can eventually arrive at the point where your end-to-end execution validates hundreds of your most critical business processes...while you sleep, soundly.