Software Regression Testing: Driving Stable Optimizations
Software regression testing is an important part of any software development life cycle. It can be the difference between a smooth software change and a long, complex nightmare.
So, what is it? Software regression testing is the act of checking recent code base changes for any new bugs or issues. When developing a new application, the best implementation approach is to write test cases while adding features, keeping the topic fresh in the developer’s mind. This allows the developer to continually add features while running the regression tests to ensure that existing functionality remains stable.
Regression testing also makes sense when testing for a specific function or block of code (called a unit test), or if the test involves multiple services and the integration between them (called integration tests). When a test involves integrations between multiple sub-systems, some specific situations need to be covered for the test to be effective and manageable.
Software Regression Testing In Practice
LYONSCG recently completed an internal project involving a custom database, a service layer above that database, and a UI, which interacts with the services.
Throughout the development process, we continually added integration tests as the service layer was developed. We were able to implement changes both for increased efficiency and for new features while using the regression tests to ensure that existing functionality remained unchanged.
This was very helpful in a number of ways. First, it caught accidental changes in the functionality and helped reassure the team that we weren’t breaking anything during development. The regression tests did require development and maintenance, but that cost was amortized over the lifetime of the project and was well worth the investment.
Because there was a database involved with data changing almost daily during development, our regression testing approach required some additional features. For example, some services retrieved a set of rows from the database which met the specified criteria. When data was added to the database, that list changed so the number of rows returned changed as well. This required displaying the count of rows returned as well as a note that the most recent results might not match the previous results because additional data was added to the database.
In this case, the software worked perfectly, but its results were different only due to database changes. As a future enhancement, we have a preliminary design for the ability to start each test with the database pre-set to particular content. This would allow us to retrieve the exact same results during each run of the regression tests.
Software Regressions Testing: When to Do It
More rigorous regression testing is always a good idea and should be done at multiple stages of the life cycle. Developers should be doing brief regression testing after they’ve introduced a change on a sandbox, making sure that the change hasn’t introduced more bugs or broken any existing functionality. Once the change is pushed to a development environment and tested by QA, it should be moved to Staging for another round of regression testing.
Lastly, when everything is deemed okay, a quick smoke test should be done on production. Creating an effective plan for regression testing should be a part of any team’s work process.
Software regression testing can effectively help save a project money due to catching bugs from minor issues to project breaking bugs before they get pushed to a live environment. This saves money by making sure time is not wasted by fixing an issue multiple times due to a lack of testing.
