6. Feasibility

6. Feasibility
	Chapter 18. Dynamic Analysis

Full coverage is rarely possible. In fact, even achieving 100% statement coverage is usually impossible.

A criterion is said to be feasible if it is possible to satisfy it. In practice, criteria are not usually feasible.

Despite these problems, the idea of coverage is a very important one in practice. If there are significant parts of your program that have never been executed, you should not have much confidence in their correctness!

6.1. Practical Guidance

It should be clear why neither program-based nor specification-based criteria are alone good enough.

If you only look at the program, you’ll miss errors of omission.
If you only look at the specification, you’ll miss errors that arise from implementation concerns.

Experience suggests that the best way to develop a good test suite is to use specification-based criteria to guide the development of the suite, and program-based criteria to evaluate it:

You examine the specification, and define input subdomains.
Based on these, you write test cases.
You execute the test cases, and measure the code coverage.
If the coverage is inadequate, you add new test cases.

Runtime assertions, especially representation invariant checks, will dramatically amplify the power of your testing. You’ll find more bugs with fewer cases, and you’ll track them down more easily.


5. Subdomain Criteria		7. Lecture Notes