Rewriting The Rules

We’ve all heard the rule of software delivery – you can choose any two of fast, good or cheap, but not all three. If you want it fast, it either won’t be good or it won’t be cheap. Or, conversely, if you want it to be good then it either won’t be fast or it won’t be cheap, and so forth. You get the idea.

Until now, that is. The engineering team at Worksoft has managed to deliver excellent quality both quickly and economically. They did it by “eating their own dog food”, or using Worksoft Certify to test itself, then wiring comprehensive test coverage around an agile model.

Here’s how it works. First, the fast part: the team has adopted an agile approach to development that calls for continuous integration. Each developer is responsible for introducing changes, testing them, and integrating them into the overall build that is then also tested. This means a fully tested build is available on a daily basis.

Next, the good part: the QA team invested the effort to develop an automated baseline regression test of core Certify functionality. This suite, containing hundreds of tests, is used to test the daily build. Because of automation it takes less than two hours to run, so each developer can execute a complete regression suite before he or she introduces the change to the build, with enough time left over to run the same tests against the full build.

Then the team took it one step further. Not only does each developer submit changes, they also submit new tests to exercise the functionality they added or changed. Virtual machines are set up with Certify pre-installed and pointed to a database (the upstream database) of functional tests that are continuously augmented by the developers. Integration builds are triggered by software check-in or through a timed nightly build. On successful build completion, the Certify virtual machines pull down the new version of Certify and run the functional tests (this process is very fast since tests are automated and does not add significant overhead to the build process). Results of the test run are then emailed to the build team for review.

So while Certify gains capabilities on a constant, continuous basis, the test suite also keeps pace. Hundreds of tests are on their way to thousands, and because of automation the entire suite can still be executed in a couple of hours while the developer moves on to the next task.

Finally, the cheap part: by producing a quality product literally every day, Shoeb’s team can focus their efforts on creating new value instead of fixing what broke in the last release or testing their changes for the next one. The developers are much more productive and so are Worksoft’s customers, since neither is distracted by chasing the problems that should have been caught in test. The net result is a better product with less time and resources.

Aside from redefining the rules about what is possible, by becoming dedicated users of the very product they are developing the engineering team has a much better grasp of customer needs and perspective. In fact they often identify new requirements before the customer does. In essence, the engineering team runs Certify in a production environment. They install and use Certify before any other customer, so any issues that managed to escape the comprehensive regression test are usually caught the next day.

And the best part? Successful, satisfied customers.