Exhaustive testing (testing all possible data combinations) is impossible
As part of the Seven Testing Principles of the ISTQB Foundation Level Syllabus, on the exhaustive testing principle.
“Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Rather than attempting to test exhaustively, risk analysis, test techniques, and priorities should be used to focus test efforts.” ISTQB Foundation Level Syllabus
Risk analysis determines which types of software problems are most likely to cause serious problems if they are not caught by testing. Risk analysis also reveals how to prioritise test efforts by identifying which test techniques are most likely to uncover the riskiest problems. Test techniques determine how to test most efficiently.
Test techniques may include: generating inputs and preconditions that are expected to cause problems; monitoring outputs for unexpected changes; and observing how the software responds when it is placed under abnormal loads.
An efficient testing process can reduce the risk of bugs making it to production, increase the speed of regression testing, and increase the team’s efficiency. A good testing process includes:
- Deciding what to test based on risk.
- Using the right testing technique for each test.
- Creating a detailed plan for each test.
- Updating the plan as needed.
- Documenting your process and unifying your team.
The reason for this is that testing cannot prove the absence of something, only the presence of something else. The only way to prove the correctness of software is by inspecting every line of code and verifying that it does what it is supposed to do. There are no shortcuts to this.
