Test automation provides great benefits to the software testing process in terms of improving the quality of results, speeding up the process, increasing test coverage and minimizing the variability of results. In this article, we enlist 10 lessons that every software tester must keep in mind while automating tests.
However, before we provide these lessons, two ground rules must be kept in mind
You should design your tests first before you decide which ones are to be automated
This technique helps a tester to prevent from falling into the trap of automating tests that are easy to automate but weak at finding defects.
You should design automated tests differently from manual tests
Much of the power of automated testing comes from using a computer to do things a person cannot do. Look for opportunities, such as being able to repeat the same tests over thousands of different data files. This prevents you from falling into the trap of only automating tests from the existing (manual) test plans and missing the big opportunities for test automation. When designing manual tests, you aren’t likely to consider tests that apply repetitive operations over thousands of files; it would simply be too much work.
Lesson 1- Speed up the development process
Test automation efforts aimed at reducing testing costs rarely get the attention and cooperation they need to ensure success. Quick fixes keep code stable. Keeping code stable saves time (multiple people don’t waste time on the same bug) and facilitates refactoring and other efforts to improve code structure and straighten out convoluted codes.
Here are two examples of techniques for supporting the pace of development.
Build Verification/Smoke Tests
The connotation ‘smoke test’ denotes that once you plug in a new board and turn on the power and see smoke coming from the board, you should turn off the power. Smoke tests (a.k.a. build verification tests) extensively cover all the product features within a limited period- typically over lunch or overnight. If the key features don’t function effectively or if the key bugs haven’t still been removed, your team won’t waste any more time testing or installing or testing the build.
Automated unit tests
These tests also streamline a development process, prevent backtracking, and maintain development momentum.
Lesson 2- 100% automation is not a good thing
Some software managers suffer from the misconception that automated testing always equals better testing, and they mandate that all testing must be automated. Another myth to dispel is that of testing happening at the push of a button, automatically reporting any bugs that are found. Tool vendors who offer integration between their testing and tracking tools encourage this irresponsible fantasy. Only a proper mix of manual and automated testing is the key to better results.
Lesson 3- Don’t equate manual testing to automated testing
While lesson 2 teaches us that a mix is essential, it is also necessary to be clear on the difference between manual testing and automation testing. Automation does not make the computer do the testing that you do. It performs the testing you explicitly specify, unable to benefit from your tacit knowledge and awareness. The automated test does the same thing each time it runs, at the same speed, in the same order, with exactly the same mouse moves and key clicks. The manual tester, however, can’t help but vary a test every time he runs it. These variations can uncover unseen bugs.
The prepared mind is a fantastic test tool, beyond any conceivable automation.
Lesson 4- Don’t estimate the value of a test in terms of how often you run it
The value of a test comes from the information it provides. Skilled testing is often a matter of exercising such judgments well. Like all testing activities, behind the decision to automate some tests is a cost and benefit analysis. If you get the analysis wrong, you’ll allocate your resources inappropriately.
Lesson 5- Automated regression tests find a minority of the bugs
Various surveys point out that the number of bugs found by automated tests is surprisingly low. In fact, regression test automation typically finds more bugs during test development than when they execute tests later. Nevertheless, in case you take your regression tests and find occasions to reuse them in different environments (e.g. a different hardware platform or with different software drivers), your tests are more likely to find problems.
Lesson 6- Automate for immediate impact
- System setup and preparation
- Diagnostic assistance
- Session recording
- Test generation
Lesson 7- Encourage the development of unit test suites
Unit testing focuses on the smallest units that comprise a software system: the functions, classes, and methods that the programmers create. True unit testing tests units in isolation. Stubs are created to handle outgoing calls and drivers are created to provide incoming calls. Building these stubs can be a major effort.
A more popular form of automated unit testing avoids stub development by testing units in context (also called as unit integration testing). Unit testing is also seen as a core practice of extreme programming and other agile methods.
Conclusion
The above lessons bring out two important things that every tester should remember
- Automating without good test design may result in a lot of activity, but little value.
- Designing tests without a good understanding of automation possibilities may overlook some of the most valuable opportunities for automation.
Connect with KiwiQA to leverage focused capability for QA and Testing services.