5 types of regression software testing to try today

Guest post by

Sarah Roderus, CTFL, CI, ASE

TCT Computing Group, Inc.

 

The focus of regression tests should be to find application failures when the code has changed or the application is supported in a new environment. Look for reappearing old defects or new defects in the application.   Sometimes this happens with code changes or upgrades. 

 

Regression testing can also increase user confidence.   You need to make sure that the new version is better than – or at least as good as – the old version of the application. In this way, you provide your users with confidence when you make testing an integral part of the application release.

 

“Testing after release? We don’t need testing after a release!”

 

This all seems fine and good, the problem is that some applications don’t get tested after being released. You may test the application before it goes into production, but once it is put into production, it may never be tested again. A reason for this may be that testing resources move on, or they don’t exist any longer. This scenario might be common if you have a centralized testing group because the individual projects don’t have testers assigned to certain applications.

 

If upper-level management doesn’t see the importance in regression testing, it may not even be part of the testing policies.

 

There may only be testing policies to test the application before it goes into production. This is probably because regression suites can be time consuming to execute or because maintenance of the tests is very difficult.

 

Why should you perform regression testing? Developers test their fixes, so why do you need more testing? Even when a fix is good, other issues may be discovered during regression testing. Changes in the environment may also cause issues. Sometimes features don’t work, sometimes defects are brought forward or maybe there is a compatibility problem.

 

Some common methods used in regression testing include:

 

Run all of the existing tests


The most widely used regression strategy is to run all of the existing tests. If there is a test that has been written, and it was run before, it goes into the regression suite. But this strategy has a few downsides.  If all those tests are manual – after running the same tests 4-5 times, the testers are going to be bored and exhausted. If the regression suite includes all of the existing tests, it’s going to be enormous. Even when a regression suite contains all of the tests, and they all pass, this doesn’t mean there aren’t defects.

 

Testing an application 100 percent is probably impossible. Testing 100 percent of an application means testing every single field – every single combination of input, between all of the fields, menu options, output, etc. You typically don’t have time to do that. So, even those “run all existing tests regression suites”, only contain tests on a certain percentage of the application.

 

Run tests that are high risk


Another method used for regression suites is to run tests that are high risk. The tests you need to run should be prioritized by the business users. Tests that are the most important to the business users are those processes the business users do all the time—and the functionality is critical to them. Remember that as the application changes, the key business processes may also change.  It is important to specify a time limit in your test plan for the high-risk tests. Depending on what else needs to be included in the regression suite, 30-40 percent of your total regression time can be allotted for high risk tests.

 

High-defect features


The third method is high-defect features. This tests typical areas of the application that are high defect areas or areas that are highly complex. Perhaps it’s just the feature that is complex – it may include intricate calculations or integration with one or more other applications. This also includes functionalities that have had many defects in the past.

 

Exploratory testing


The final method is called exploratory testing – this is NOT random testing. True random testing can be done by anyone – with no knowledge of the application or testing – it is totally random. Exploratory testing is about doing test case design and execution at the same time. As you create and execute these tests, you discover problems and features within the application. These discoveries drive what is tested next.  Make sure there are enough notes for you and the developer to reproduce any issues. Exploratory testing takes advantage of the tester’s experience and understanding of application testing.

 

Automation testing


If you have a number of regression tests, automation can be wonderful to decrease the number of tests to run manually. Automation tools, such as HP’s Unified Functional Testing, will execute the tests quickly. Therefore, this reduces the amount of time needed to run the same test during the regression testing phase. A manual test may take five minutes to run, but that same test may take less than a minute to run through Unified Functional Testing. This can help you to get a more comprehensive regression test suite.

 

However, automation is not a quick fix. Automation scripts take time to develop. You have to allow for time between regression runs to develop and get the tests working. If your environment is changing, automation scripts will need to be updated accordingly.

 

5 Tips for your regression suite:

 

1.Re-evaluate every single time what is actually in that regression suite. If you have functionalities that are obsolete – they need to be removed from the regression suite.

 

2. If you have a test that you have run and it passed the last two versions, is there value in running that exact same test? If you change the inputs and/or outputs it’s a different test. So, is there value in actually running that same test over and over and over again?

 

3. Don’t run tests that already have defects that have been deferred. By a defect being deferred, that means the defect has not been fixed. You don’t want to produce the same errors because you will clutter the results of the regression test suite. Every error produced during a regression test execution should require attention. Therefore, remove the tests linked to a deferred defect.

 

4. Remember the users.  Think about what is typical for them to do and what the users consider risky may change over time.

 

5. High defect areas. Because the defects get fixed, the high defect areas can change every single regression test suite execution. Make sure you re-evaluate.

 

5 Strategies for your regression suite:

  1. Perform regression testing on a regular basis. Regression testing should be done when there has been a code change, defect fix, or new environment – it needs to be done on a regular basis, whatever that is.
  2. If issues are found, report it accurately. If issues are found be sure to report them accurately so you can determine if the application is regressing. As new code or features are added, sometimes old defects surface again.
  3. Automate if you can. If automation can be used to get more testing done in the regression suite – great – do it!
  4. Don’t forget non-functional characteristics. These characteristics include usability, portability, and performance testing.
  5. Tests should be changed and updated. Don’t run the exact same tests in the exact same order every single time.

 

Do you have any additional thoughts about regression testing? I would love to hear them. Feel free to reach out to me in the comments section below. Testing is a passion of mine, and I love connecting with others who understand.

Comments
DanielM(anon) | ‎08-21-2013 03:24 AM

Very good article Sarah

 

I think that regression testing is very important in the quality assurance environment and not including this to your development progress could make a big impact on the final product. 

 

The biggest problem I see with regression is that most of the companies do not have dedicated teams per project. The QA team is assigned to 3-4 project and this is why they don't invest time in regression testing.

 

Another wrong thinking is to let  only the developers to test their fixes. For my personal experience, developers test their fixes but most of them are using a single set of data or the correct set of data that will make his fix to work. But when a tester starts to verify the fix, he can find several issues with the fix that the developer verified and marked as working because the tester had some other types of data or steps in mind when he started to verify the fix.

Sometimes a tester has a different thinking that a developer and this type of thinking makes a difference. 

 

Another usefully type of testing is testing after release (aka post release testing).From my experience with post release testing, this helped us to identify and clear some ugly bugs that could create a bad impression to the customer. With post release testing the QA team managed to find issues caused by the fixes.

 

 

Robert Parsley(anon) | ‎09-12-2013 01:42 AM

Regression testing is very important when testing features that have not been modified themselves, but interact with other features or functionalities that have been implemented or modified in the application.

Also, regression testing should be especially performed on areas that are known to be frequently used and verified by the client or users.

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the Community Guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Search
Showing results for 
Search instead for 
Do you mean 
About the Author
This account is for guest bloggers. The blog post will identify the blogger.


Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation