No – Agile Won’t Work Without Test Automation!

Agile TestingSince the manifesto arrived, the Agile development methodology has turned the software development world on its head. With Agile, organizations have been able to keep up with market demands and can implement product improvements, modifications, and updates faster. Speed, responsiveness, and quality are the hallmarks of agile development. Agile allows you to ensure that you can deliver completely functional software in the shortest period of time. By definition, the term ‘agile’ means to be quick and nimble. Thus, most Agile projects are characterized by shorter sprint cycles and faster code delivery. At the same time, when we talk about agile development we also have to ensure that the risks, such as bugs or broken code, are addressed at the speed of development, in Agile. This is where agile development testing and specifically Agile Test Automation comes into play.

“Just because you’ve counted all the trees doesn’t mean you’ve seen the forest.”— Anonymous

Given the number of sprints, faster development cycles and even faster deployment cycles, testing often and testing fast becomes imperative to make Agile development testing work. Almost all concede that the further down the development cycle a bug is discovered, the costlier it is to fix. Correcting issues regarding quality can not only be expensive to fix at the later development stages but can be hard to mitigate if the bug slips out to the customer. Remember how Toyota had to recall over four hundred thousand vehicles because of a bug in the brake control system, or the issues with the Obamacare website when it was launched?

Testing leads to failure, and failure leads to understanding – Burt Rutan

Clearly, in today’s business environment, the customer wants a working, and error, and bug-free software. The competitive marketplace ensures that there is no room for inaccuracy. Thus it becomes the responsibility of the development and testing teams to ensure higher code quality. In an Agile environment, developers and testers have to work together as a team in order to ensure that this happens. Testers need to understand how the piece of software is expected to perform and then write tests for the functionality accordingly. Being ruthless about software testing and assuming that “All code is guilty, until proven innocent” is the only way to success in an Agile development testing environment. Only when tests lead to code failure can development teams gain an understanding of ‘why’ the code won’t work and then work out ‘how’ it can work. It’s all about failing fast and failing often. This gives Agile the push to deliver better, faster. Given this increased emphasis on testing at pace, agile test automation becomes THE critical element for agile development to work.

“Testers don’t like to break things; they like to dispel the illusion that things work.”— Kaner, Bach, Pettichord

The sheer number of tests, across such a large number of iterative versions, that have to be done in Agile development makes an inescapable case for agile automation testing to gain predominance. Imagine, if we were to depend on manual testing alone, not only would we not be able to test all the pieces of code written, we would also not be able to test thoroughly in the limited time available. This would render the benefits of Agile redundant. Since unit tests need to be run after every build, an Agile Test Automation plan may comprise as many as 80% of these, tests given the volume. Acceptance testing, integration testing, and component testing are also good candidates for Agile Automation testing.

A robust, and comprehensive Test Automation strategy is also essential to check that all the moving parts mesh harmoniously and work in a cohesive manner with one another. Testers thus have to build test cases for automation and a test automation framework that take into account the business logic and, at all times, avoid technical debt that increases from missed bugs or code breaks in the system from earlier versions and delivers on the ROI for testing.

To our thinking, when you want to take the ‘test first, and test fast’ approach, as you take an iterative approach to product development in the push to roll out products faster, including testing automation as a key part of your development strategy is absolutely critical.

“The problem is not that testing is the bottleneck. The problem is that you don’t know what’s in the bottle. That’s a problem that testing addresses.”— Michael Bolton

Agile development demands technical excellence. Great quality code is then a natural consequence of this excellence. Yes, test automation demands a time investment to understand which test cases and scenarios to automate, which candidates render themselves to automation, which tests demand different configurations, and how the product is going to evolve to ensure maintainability of testing suite. Clearly, developing test cases also needs a time investment. However, once the automated testing suite is developed, it not only increases the speed of test execution but also dramatically increases the scope of test coverage and reduces the number of tests that need to be executed manually. Another way of easily and quickly building Agile automation testing is by adopting Scriptless Test Automation. This provides the testing and development teams the benefits of automation testing without the added hassle of writing code. An Agile test automation framework is the necessary arsenal that an Agile development team needs to gain confidence in the software that they are developing.

Agile development is not about taking shortcuts. Our view is that it is about enabling teams with effective and efficient means to work faster and work better in order to produce greater quality…and this takes some effort (read: writing test automation scripts). However, if you are not automating testing then it’s hard to see how you can effectively do Agile development. Agile is all about creating value and quality, within the time available. The words of John Ruskin summarizes this quite aptly, “Quality is never an accident; it is always the result of intelligent effort.” In agile development, that “intelligent effort” is Agile test automation, if you ask us!

2 thoughts on “No – Agile Won’t Work Without Test Automation!

  • The real question is whether Agile solve better our problems that we face in Software industry, as SW is consider problem solving discipline.
    So Agile (like any other methodology) can be consider as a tool that can/or can’t solve better the problems that we a facing during the development testing and releasing of System/Software products, and as I see it Agile is not a silver bullet, it won’t solve our problems, may be some of them.
    It depends on business domain and context what, when and how you will implement tools and development methodology.
    Three main parameters: BUDGET, SCHEDULE and QULITY determine if a Project is considered as a success or failure.
    Since Agile become popular and was implemented in more and more organizations I haven’t seen that the statistical data about projects success have been changed/improved, we still have about 65% of the projects that are consider as failures regarding the above mentioned parameters.
    My second comment is that throughout your entire article when you talk about Quality, the only means that you imply to achieve Quality is testing.
    It doesn’t matter what is your development methodology (including Agile) if you have tool or not and implement automation in testing, testing will not give you the required Quality.
    It seems like you believe that you can test in Quality into the product, but Quality can be achieved only by planning and designing Quality during the whole SDLC.
    I would like to mention just few things that you can’t accomplish by testing: Good requirements are not produced by more testing; Design is not improved by testing neither Maintainability.
    By testing you can’t improve the requirements, the design and maintainability of the SW.

  • Hi Doron – Thanks for your thoughts.
    1) I agree with you that Agile is not a silver bullet and that is certainly not the intended message of this article. You can take any methodology and if it is not implemented to suit the given situation and not embraced by people, then it will not solve what you wanted it to solve. The core success of success of Agile is in team work, initiative, ownership. Unless you truly have a self-organizing team with an ownership mindset, Agile can very well remain in the books.
    2) I am not sure why you thought quality = testing in the testing phase. We strongly believe that quality is a mindset that everyone on the team needs to possess. I also feel that there have to be reviews (self or peer or by team) in every stage of the deliverables – whether it is requirements review, design review (from all “ibility” perspective). I also believe that all reviews are completely aligned to finding “errors/ faults or defects” and improving quality of the deliverable at every stage and not just towards the end.

Leave a Reply

Your email address will not be published.