Recently, we were asked to contribute to a blog article which was supposed to be about a ‘best practice’ for implementing test automation in a DevOps environment. While we don’t disagree with the premise of the article – namely, that test automation in a DevOps environment is a good idea – we thought that writing a ‘test automation’ article was a bit of a wasted effort, as most of the information was already readily available.
So instead of writing an article about implementing test automation in a DevOps framework, we decided to write an article about why you would want to implement test automation in the first place, as there are still plenty of use cases where a manually driven test suite makes complete sense.
Manual testing is not ideal
The idea behind implementing test automation in a DevOps environment is to remove the human element from testing. Ideally, you would like to automate as much of the testing process as possible, so that even if you are not a devops expert yourself, you can at least trust that the testing has been done to a high quality.
Manual testing is, however, a slow, cumbersome process, prone to errors and a frustrating experience for anybody who has to go through it multiple times. For a manual test tester, each test can take hours, even days, to go through.
Imagine having to do the same task over and over again, for 10 hours a day, for a month straight, just to get through the whole testing process and find a few hundred bugs. That’s a month of your life gone, just to find a few hundred bugs, at a rate of one test case every two days.
Even worse is the fact that manual testing is quite often done in an ad hoc manner, with no real plan or strategy in place. Since it is usually a very time-consuming process and there is no incentive to make it quicker, some of the tests may never be repeated and many of the shortcuts which can be automated easily end up being ignored.
Test automation allows for better planning and reduces waste
Now, when planning out your test strategy, you can’t actually plan for every scenario. However, with automated testing, you can write a script which will trigger an expected set of events, and since these events are entirely deterministic, you can actually calculate pretty accurately how long it will take to run through the test suite.
As a result, you can estimate pretty accurately how much resource (man-hours) you will need for the project, assuming that all the tests are passing. Of course, this will vary depending on whether or not you run into any snags along the way, but it’s generally a good idea to plan for the worst, so you don’t end up surprising yourself by finding a bunch of bugs, after all.
Additionally, since automated testing is usually a fairly quick and straightforward process, you can do it in tandem with the development of the software itself. This means that both the developer and the test engineer can keep working on the same codebase, while also ensuring that the tests are being executed as expected and in a timely manner.
For example, let’s say that you are implementing a web app with Node.js, Backbone, and Marionette and you need to test out the functionality of the app. You can easily set up a CI/CD pipeline which will automatically spin up a VM, install all the required dependencies, and execute the tests every time a change is pushed to the master branch.
As a result, you not only get a consistent set of tests which can be run over and over again, but you also get to squeeze more value out of your automation efforts, by reducing the amount of manual intervention which is usually necessary.
The benefits of test automation are numerous
As we said above, test automation allows for better planning and strategy, since you can easily keep track of all the tests, their status and the issues which they encounter.
This means that you can easily determine what needs to be done to fix the issues and move on to the next test, instead of having to start over from scratch, every time something goes wrong. It also means that you can easily provide the person executing the tests with the proper training and support, when necessary.
Additionally, test automation can be a great way to find and monitor new bugs and issues which crop up in the software over time. One of the main reasons why people avoid automated testing, is because they think that it’s not possible to write a test which will cover every eventuality.
However, this is far from true, as every bug and issue which is ever encountered, is recorded in a log somewhere, and from there, it’s just a matter of searching for the log and implementing the fix. The fact that this is usually a straightforward process and requires minimal manual intervention, makes it all the more worth it.
Moreover, with automated testing, you can easily create a set of standard tests, which will be run over and over again, as part of the build process. As a result, everybody on the team can quickly see how the program is functioning, without having to execute it themselves, or depend on somebody else to interpret the results for them.
Additionally, since the tests are usually written in a ‘black box’ manner, meaning that they don’t depend on any particular implementation details, other than the set of expected inputs, they should be portable to any platform, and thus less prone to errors and more easily verified.
Summary
All in all, implementing test automation in a DevOps environment can be a good idea, as long as you have the right team, and the right approach. First and foremost, make sure that you have somebody who is experienced in both writing functional and performance tests.
Moreover, make sure that the team has sufficient ownership of the process, so that they are motivated to see it through to the end and will work with you, as a single unit, to identify roadblocks, and find solutions.
Lastly, make sure that you have somebody who is responsible for planning, coordinating and executing the tests, as this will determine the outcome of your efforts, more so than anything else.
