Who will test the test automation code?

test-automation-codeWe are witnessing the rise and adoption of many new software development methodologies. Be it agile, Test Driven Development or Behavior Driven Development, one thread that is common to all these methodologies is the increased impetus on testing. Clearly, software testing has cemented its position as a critical contributor to software success. As software development and testing becoming increasingly intertwined, the lines dividing the roles of testers and developers is also blurring. Test automation today has emerged as the great enabler of the iterative and accelerated development methods, allowing every piece of code to be tested in the least possible time. Given that the test automation framework, too, needs coding the question that needs answering is, “who will test the test automation code?”

To begin this conversation it is imperative to note that the test automation framework is nothing less than a software product. Just like a software product needs to be tested to check for optimal performance, a test automation framework needs to be tested as well to ensure that it delivers on performance. A test automation suite must ensure that it increases the speed of test execution, widens the scope of test coverage and ultimately reduces the number of tests that have to be executed manually. The test automation framework developer thus should not only focus on writing tests and clubbing them to maintain functionality but also on ensuring that hierarchy of test cases are maintained correctly to improve the performance of the test suite. The test suite developer also has to ensure that the tests in the test suite are independent of one another, as in many cases if one test fails then all the subsequent tests either fail or are not executed. Interdependence of tests while developing a test automation suite thus results in unnecessary time wastage caused by test re-running or the inability to achieve the test completion criteria in terms of the percentage of test completion and ultimately lead to lower productivity and missed timelines.

Along with this, the test suite developer also has to make sure that the tests are maintainable easily. The lack of maintainability increases automation costs and intensifies the automation efforts to keep the test framework up and running. As the software product keeps evolving, the tester has to ensure that the test automation suite that he/she develops can evolve too and at the same pace for the test suite to remain relevant. The test suite developer thus must ensure that the automated testing suite comprises tests that can be repeated continually and fast. They can ensure the maintainability of the test suite by ensuring that they select the right testing tool, prepare the test bed carefully, and iterate the frameworks and features of the test cases correctly. They also have to check that the test cases that they develop are able to deliver on the schedule delivery timelines correctly. All this can only be achieved when the developer of the test cases runs mock tests on the test automation suite to check if the suite is performing as expected.

Since developing a test automation suite can be a time-consuming process, it is only prudent to develop the same keeping an eye on the prospective development of the product in the future. Frequent product upgrades, patch releases, and bug fixes are now a common part of the development cycle. The test automation suite being developed to cater to such a product too has to make sure that it can stay in step with these product changes and does not falter when the end product evolves. Running frequent regression tests on the test suite thus becomes an important part of the test suite development process to ensure that the tests perform as expected with the product changes. Along with this, testers have to identify which tests plans will remain relevant during product iterations and which will become redundant in the future to create test automation suites that are well tested, resistant to changes in the UI and capable of working with future versions of the product.

For all test automation initiatives to be successful the test developer has to assume the role of the framework developer and its tester as well. Given that the test framework developer has the analytical and investigative skills to understand where bugs tend to hide, how the test plan is expected to work and know how the code assets are built, they, by default, become the best resources to test the test automation code to ensure that the test suites perform optimally and in a stable manner.

2 thoughts on “Who will test the test automation code?

  • I fully agree with this article. My personal experience is that the risk of neglecting all these points is especially high in projects where you hire only one test automation expert. He/she becomes like a God and none knows whether she really is until someone else takes over her job.

    When I was about to leave the company, I realized that I cannot do a handover to my successors before a major refactoring of my test automation framework and also on my test scripts. I had such task on my TODO list long before already but the release pressure never allowed me to allocate appropriate time for it. When I finally got the time, it was a great experience for me. I also documented the whole architecture of the framework and added special notes about things none could knew except me due to my long experience in certain areas.

    And at the end I really felt I handed over a real product which was clean, properly documented and easy to maintain. It was a good feeling.

Leave a Reply

Your email address will not be published.