Test Management - How Much Do You Test
03 January 2022
Many people answer with “We must test everything!” Have you ever thought that? How long would it take for one tester to test JUST the front end of an application with 10 data entry screens, each with 10 fields that accept two data types? Remember, you need to accomplish positive and negative testing. If one field will take up to ten characters, then you will need to test one, two, and then three all the way up to ten characters—for EACH field, on EACH screen. Also, there are the combinations between the fields. How long will it take?
So, you rationalize what is tested. Testing is typically under a timeline that is too short to be tested as thoroughly as you would like. Rationalization must allow you to thoroughly test as much of the application as needed. Risk assessment is one way to assist in test case rationalization.
The answer to the original question, “How much do you test?” will probably come from the person who asked the question. The answer is “What is the risk?” Risk can be defined as the probability of an issue occurring and the impact of that issue. You shouldn’t test just because testing is in the Project Plan. Testing is in the Project Plan to reduce risks. Sometimes risk is beyond the software under testing. The risk may cause the company to lose credibility or stock prices to go down. Project risks may be elevated due to regulations or other legal binding. Discover those risks and reduce them as much as possible, within the project’s constraints (time and budget)
Maximize the value
of your ALM
Govern quality and implement rigorous, auditable software lifecycle processes.
Delivering accurate information to the decision makes on the project is one of the most important responsibilities of a tester. Making sure that the information is not only accurate, but meaningful, is the hard part. Stating that the testers have found 100 defects is not the entire story. How those defects relate to the identified risks is much more helpful. Knowing the risks, test leads can assist in answering the question, “Should we keep testing?” Test leads should be constantly reviewing and evaluating the defined exit criteria to answer the question.
Given all the time you need, testers would be able to test an entire application. Of course that assumes that you don’t get bored to death testing the same application for your entire career! Testing an entire application is just not feasible. The answer to how much testing is enough will be different according to the industry, the availability of time, budget, testing resources, and experienced users. Most importantly, ensure the risks are defined in the test plan and are re-evaluated often!