Over the past few years, we have witnessed the growing integration of software testing with software development. The rise of development methodologies such as agile, DevOps, and Test Driven Development is a testament to the fact that software testing is indeed a critical contributor to product success. Testing is also becoming increasingly important because of the rise of the application economy. In 2015 there were over 25 billion iOS app downloads and 50 billion android app downloads. Statista estimates that the number of app downloads in 2017 will cross 268.69 billion. However, what is also true is that nearly 50% of the mobile app consumers will delete an application immediately if it is slow performing or has bugs. Clearly, software development companies recognize the importance of testing and have testing and automation plans in place. But despite that, as we have written previously, only 14% of test automation initiatives are successful and deliver the promised benefits.
The question that then needs to be answered is why do these automation initiatives fail? As development methodologies are changing it is also essential that testing strategies evolve. Automation testing can no longer be about record and playback but instead has to follow the same developmental methodologies as any software product development. Building a test automation plan as an afterthought and not aligning the automation strategy with the business objectives leads to testing failures.
We also need to take into consideration the speed of development when talking about automation testing. The main aim of automation is to test at the speed that matches the speed of deployment. According to a report by SauceLabs, only “21% have achieved all 5 key areas of agile testing maturity”. The report goes on to show that only 23% fix bugs right away and only “24% iterate small testable requirements rather than waiting for features to be completed”. With the main objective of agile development being faster releases, testing has to be highly automated and bugs have to be fixed almost immediately in order to deliver high-quality applications. This also means that the QA team and the development team have to work as partners and not in isolated environments to facilitate the speed of agile development. Clearly for development initiatives to be successful the roles of developers and testers have to become increasingly integrated.
One of the most important practices today is thus to test fast and test often. Making the development process micro-iterative by breaking down the features into small testable blocks increase the rate at which bugs can be identified. Using the test results and resolving failures immediately become imperative for automation success.
Another reason why a number of test automation initiatives fail is because organizations fail to leverage test data. Test data has to be easily available and should support different test combinations and maturity levels. To enable this, it becomes essential to make tests more repeatable and independent and allow different testing teams to create their own data to use and access when they need it. Leveraging frequent data dumps from production helps in creating relevant test data and come up with a reusable solution.
It is also essential to work on test maintainability and prepare for automation evolution from the word ‘go’. Any change to the application, whether it is an upgrade or a new feature, has to be tested. Identifying which test plans can be reused, building smaller and independent test cases, analyzing test data, identifying which test plans need to be discarded etc. thus become an essential part of any automation lifecycle strategy. Organizations have to realize that a test automation strategy cannot take a ‘fit it and forget it’ approach and that it has to be updated and upgraded according to the changing needs of an application. Having a robust maintenance model makes sure that the test suite does not fall behind and remains up to speed with changing workflows and functionalities of an application.
Identifying which test platform and testing tool suit your product requirement also is a key contributor to the success or failure of automation testing. Assessing the business objectives that need to be met with automation, identifying the candidates for automated test cases and the number of expected releases per year, assessing the application lifecycle, application technology etc. are some of the areas to look into when selecting the testing tools. Organizations also need to understand that developing an automation suite also needs a certain degree of coding. Hence, testers need to have some coding skills as they are the ones who know best which scenarios should be automated and which ones do better with manual testing. Given the fact that it is the testers who know the testing path, they are able to look at the piece of software more dispassionately and hence are more unbiased with the code. But all good testers are not great programmers. In the absence of such skills, scriptless test automation can be leveraged to meet testing requirements and build flexible automated test cases which can scale up or down as per the application requirement.
Finally, for the success of any test automation initiative, it is imperative to set the right expectation. The goal of any testing strategy should not be 100% automation. Instead, testers need to identify which areas will benefit most from automation and deliver time, cost and productivity gains. It is when this approach is applied that test automation initiatives are 100% successful and deliver on the ROI front.