“It is not the strongest or the most intelligent who will survive but those who can best manage change.” Leon C. Megginson
Whoever said that change was the only constant may have been talking about software product development and by extension software testing. The Lean Startup movement drove in the Minimum Viable Product philosophy. Starting with a bare-minimum version of the product followed quickly by several versions with incremental feature additions based on market feedback has become a way of life. As the MVP approach caught on, so too did the iteration-friendly Agile development approach. Agile development and Agile testing are now the norm. The accelerated time to market and the extreme quality needs for each version makes automated testing indispensable to this way of life. The question though is when the product itself is undergoing so much change, how can you ensure that the automation testing suite and the overall automated testing strategy stays relevant?
This is an excellent reason to get the software testing team involved very early in the product planning – as early as the business requirements stage if need be. This will help those in the software testing team tasked with creating the test automation strategy create a long-term vision for the test automation that is aligned with the strategic direction of the product. Keeping this vision in mind will help the automation testing stay on track even as the specifics of the product evolve. At this stage, it would also help to try and identify those parts of the product that are less or more likely to change over time and to focus the automation efforts on the more stable portions of the product.
As has been mentioned, the “time to market” pressure is always high, and this squeezes the window available for software testing. This makes it impossible to drastically alter the automation strategy from release to release. The automation approach most likely to work in this situation is a more modular approach with independent components that can be used, reused or removed depending on how the product itself evolves. The architecture of the automation suite, the choice of the test automation tools and technology should all be made with this approach in mind.
So, there is time pressure, and apart from the testing team, even the subject matter experts (SMEs) could be involved in automating the testing. Under the circumstances, it would make some sense to consider ways and means to help create the automation suites speedily as well as easily. Scriptless test automation is one way to achieve that. Scriptless automation does not require specific coding skills and since it uses “ready” components, automation can be created faster that would otherwise be possible. This approach can help accelerate the automation process.
Lastly, we recommend keeping a sharp focus on maintenance of the automation suite – a product just like the product-under-test. As the product evolves, the automation suite will grow, code will be added and the regression risk starts adding up. Test maintenance becomes critically important. This is a tricky situation as a management decision must be made as the effort involved in maintaining the automation suite starts adding up. At some point the effort involved in maintaining it could become more than the value it is delivering and the plug may have to be pulled. That said, though, this situation is mitigated to an extent by following the modular approach mentioned earlier.
These are some factors we consider important – but no means are these all the factors that matter in helping the automation strategy stay relevant. There are a host of other considerations like the automation tools and technologies chosen, the skills available at your disposal and the scale and scope of the product under test that matter too. For those who made it this far – how did you keep your test automation strategy relevant? Or is your experience that the velocity of change made your test automation irrelevant?