Irrespective of how well conceived and tested a program is before being released, a software eventually has to be improved so as to fix bugs or respond to changes in user specifications. In simple terms, regression testing is retesting of previously working software after a change to ensure that unchanged software is still functioning as before the change.
This article discusses how to guide the planning and assessment of regression testing activities in large, complex reuse-based systems, when and how to execute regression tests and challenges that can occur to regression tests.
What is Regression Testing?
According to IEEE, regression testing is “Selective retesting of a system or component to verify that modifications have not caused unintended effects and that the system or components still complies with its specified requirements.”
As for testing, in general, the objective of regression testing is differs from business to business. The objective may either be to find the errors or to get an estimation of its quality. Traditionally regression testing is iteratively applied to the consecutive revisions of the software, where revisions refer to the variation in time of the evolving system.
Importance of Regression Testing
Regression testing is an important software maintenance activity that involves repeatedly running a test suite whenever the program under test and/or the program’s execution environment changes. By performing a regression test suite on the introduction of a new feature of a defect mix, a tester can ensure that the modification of the program is such that it does not negatively impact the overall correctness.
Efficient regression testing is vital, especially for organizations incurring huge costs in software development. It incorporates, inter alia, determination of test cases required to be re-executed (called as regression test selection) so as to verify the behaviour of the test which has been modified. The prime objective of regression testing analysis is to make sure that the changes that have been made to the software(s) have not resulted in an adverse effect on the essential aspects of the software. It must be conducted to confirm that recent program changes have not adversely affected existing features and new tests must be conducted to test new features.
However, regression testing can also be prohibitively expensive, particularly with respect to time, and thus, accounts for as much as half the cost of software maintenance. Some good practices in regression testing are-
- Run automated daily tests on module level.
- Focus automation below user interface.
- Visualize progress monitoring.
These practices are not specific to regression testing. The latter item is not specific testing at all but is a management practice that becomes critical to regression testing as it constitutes a key part of the development project progress.
When and How to perform Regression tests?
Regression testing is carried out at different levels (e.g. component level, system level, and module level [Q18-20]) and different phases of the process of software development. From focus group discussions it was found that some organizations perform regression testing comparatively earlier whereas the others perform regression testing later, and some claimed that regression testing is carried out continuously throughout the entire development process. The objective may slightly vary for the 3 scenarios; the early regression testing enables earlier detection of the defects, while the later regression testing is generally carried out for certification purposes. As far as frequency is concerned, some organizations perform regression testing every day while the others perform regression testing at every point of software integration, and at every milestone, or regression testing could be basically the selection of the developer’s tests from one specific regression testing suite, or newer test cases that are designed.
According to questionnaire responses, most commonly the test cases are reused. Strategies for regression test selection mentioned in the focus group were: complete retest, combination of dynamic and static selection, full retest of the critical safety units, selection of test cases based upon the modifications and potential side effects, smoke test, ad-hoc selection, prioritize and keep running as many as possible, and concentrate in the functional tests. A project may include several different regression testing activities, including manual and automatic regression testing techniques.
Challenges
Regression testing is a frequent and time-consuming test activity in most industrial software projects. It is applied in many different contexts, not based on research but based on experience. It ensures that nothing has been affected or destroyed, and give an answer to whether the software has achieved the desired functionality, quality and stability etc.
Repeating all previous test cases in regression testing after each minor software revision or patch is often impossible due to the pressure of time and budget constraints, but omitting test cases used in regression testing is also risky. Studies indicate that 80% of the testing cost is regression testing and more than 50% of software maintenance cost is related to other testing processes. Some common challenges to regression testing include-
- Test case selection– Accessing the consequences of modifications on the older coder and making a very good selection is often difficult. Even prioritizing the test cases based on the product risks and defect identification capability, and being confident not to miss out critical safety defects is not that easy. Identifying the required amount of tests was also considered a problem, and accessing test coverage is also quite hard.
- Test case design– Lack of time and resources for regression testing is also another issue in regression testing.
- Manual and Automated regression testing– It is not only costs to implement the automated tests which required being considered but also costs to maintain those test suites. In many cases carrying out manual analysis on the results is also considered quite costly.
- Design issues– It also play a crucial role in regression testing since a strong correlation exists between the effort needed for regression testing and software design. Design for testability, including modularization with well-defined and observable interfaces, helps to verify modules and their impact on the system. This could be addressed by including testability in design guidelines.
Conclusion
Regression testing increases in software projects as software becomes more and more complex with increasing emphasis on systematic reuse and shorter development cycles. Many of the challenges are not specific to regression testing but are general to all testing but have a significant impact on how effective the regression testing could become. Questions involving automated testing are of course particularly important for regression testing, as the same tests are repeated many times. Lastly, having a test-friendly design will go a long way in performing selective retesting processes like regression testing.
Reference source: https://pdfs.semanticscholar.org/96d6/ababe5f2a407252adbd9f811a9a982e7b6ca.pdf
Reference source: https://pdfs.semanticscholar.org/c724/679cdde3dd21ff168552875b5607da42e7fc.pdf