Mutation Testing¶

Test quality can be assessed by systematically introducing faults (mutations) into the code and checking whether tests detect them - surviving mutations indicate tests that execute code without verifying its behavior.

Tension Addressed¶

Coverage vs. Quality

Recognizable As¶

Tools that modify operators, constants, or control flow; "mutation score" metrics; surviving mutants indicating weak assertions; equivalent mutant analysis

Canonical Sources¶

DeMillo, Lipton & Sayward, "Hints on Test Data Selection" (1978); Jia & Harman, "An Analysis and Survey of the Development of Mutation Testing" (2011)

Transfers To¶

Fault injection in reliability engineering; chaos engineering; adversarial testing in security

Mutation testing addresses coverage's blindspot. If changing > to >= doesn't cause any test to fail, tests aren't sensitive to that boundary condition. The technique is computationally expensive but provides genuine insight into test effectiveness.