How much do you test your applications?

When you decided to test, how do you decide what to test? Many people answer with “We must test everything!” Have you ever thought of doing that—testing EVERYTHING?

 

How long would it take for one tester to test JUST the front end of an application with 10 data entry screens, each with 10 fields that accept two data types? Remember, you also need to accomplish positive and negative testing. If one field will take up to ten characters, then you will need to test one, two, and then three all the way up to ten characters - for EACH field, on EACH screen. Also, there are the combinations between the fields. How long will it take?

 

Take some time and rationalize what is tested. Testing usually occurs under a timeline that is too short for testing to be thorough as you would like. Rationalization must allow you to thoroughly test as much of the application as needed. Risk assessment is one way to assist in test case rationalization.

 

Determine the risk

 

The answer to the original question, “How much do you test?” will probably come from the person who asked the question. The answer is “What is the risk?” Risk can be defined as the probability of an issue occurring and the impact of that issue. You shouldn’t test simply because testing is in the project plan. Testing is in the project plan to reduce risks.

 

 Sometimes risk is beyond the software under testing. The risk may cause the company to lose credibility or stock prices to go down. Project risks may be elevated due to regulations or other legal binding. Discover those risks and reduce them as much as possible, within the project’s constraints (time and budget).

 

Risk can help prioritize test cases. Ask yourself these questions to determine priority. Which tests cases use the most common pieces of the application or are performed on the most complex parts of the application? Processes that are most used and/or most important to the end users are typically higher priority. Tests that are likely to find the most defects will also be a higher priority. Time spent studying the basis for testing, requirements, user manuals, business processes, etc. can be essential in determining test case priority. Keep in mind that priorities can change each time the application is tested. Therefore, each regression test cycle may not be exactly the same. If the priorities change, so do the test cases.

 

testing wordcloud.pngDeliver accurate information

 

Delivering accurate information to the decision makers on the project is one of the most important responsibilities of a tester. Making sure that the information is not only accurate, but meaningful, is the hard part. Stating that the testers have found 100 defects is not the entire story. How those defects relate to the identified risks. Knowing the risks, test leads can assist in answering the question, “Should we keep testing?” They should also be constantly reviewing and evaluating the defined exit criteria to answer the question.

 

If this was a magical world and testers were given unlimited time, they would be able to test an entire application. (Of course that assumes that you don’t get bored to death testing the same application for your entire career!) Testing an entire application is just not feasible.

 

The answer to how much testing is enough will be different according to :

  • The industry
  • The availability of time
  • Budget
  • Testing resources
  • Experienced users

 

Most importantly, ensure the risks are defined in the test plan and are re-evaluated often!

 

 

I would love to know what you think. Do you wish there was an unlimited amount of time for testing? Do you think testing 100 percent is even a feasible option? Share your thoughts with me and other readers in the comments section below.

 

 

Written by

Sarah Roderus,

Vice President, Consulting Services

TCT Computing Group, Inc.

Comments
j gavin heck(anon) | ‎03-01-2013 11:28 AM

Just came from a meeting where I mentioned the importance of risk in TDD/agile.
Great article

J Gavin Heck, CSM

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the Community Guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Search
About the Author
This account is for guest bloggers. The blog post will identify the blogger.


Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation