3. Criteria

3. Criteria
	Chapter 18. Dynamic Analysis

Suppose we have a program P that is supposed to meet a specification S. For simplicity P is a function from inputs to outputs, and S is a function that takes an input and an output and returns a boolean. Our aim in testing is to find a test case t such that:

S (t, P(t))

is false: that is, P produces a result for the input t that is not permitted by S. We will call t a failing test case.

A test suite T is a set of test cases. When is a suite ‘good enough’? We can apply an evaluation criterion:

A criterion that involves only the program is called a program-based criterion. People also use terms like ‘whitebox’, ‘clearbox’, ‘glassbox’, or ‘structural’ testing to describe testing that uses this criteria.
A criterion that involves only the specification is called a specification-based criterion. The terms ‘blackbox’ testing is used in association with it, to suggest that the tests are judged without being able to see inside the program. You might also hear the term ‘functional’ testing.

Most criteria don’t involve both the program and the specification.


2. Testing		4. Subdomains