Testing Strategies¶

Tier: 2 (Architectural Judgment)

Concepts: 14

Durability: Durable (Core testing concepts predate modern frameworks; the tension between confidence and cost is inherent to verification)

Fundamental Tensions¶

Tension	Description
Speed vs. Realism	Fast tests require isolation from real dependencies, but isolation reduces confidence that the system works as a whole. Real dependencies provide higher confidence but execute slower and introduce external failure modes. This tension is physical - I/O and network operations have irreducible latency.
Isolation vs. Integration	Isolated tests pinpoint failures precisely but miss interaction bugs. Integrated tests catch interaction bugs but produce ambiguous failures. No single test scope can optimize for both precise failure diagnosis and comprehensive interaction coverage.
Coverage vs. Quality	More tests increase the probability of catching bugs, but adding tests has diminishing returns and increasing maintenance cost. A codebase can have 100% line coverage yet miss critical edge cases, while excessive testing creates friction for change.
Sensitivity vs. Stability	Tests must be sensitive enough to catch real bugs but stable enough to not break during legitimate refactoring. Tests coupled to implementation details catch internal bugs but break constantly; tests coupled only to behavior miss internal corruption but survive refactoring.
Specification Precision vs. Exploration Coverage	Example-based tests precisely specify known scenarios but cannot explore the unknown. Property-based tests explore vast input spaces but require articulating invariants that may be harder to specify than examples.

This Domain Concept	Transfers To
Test Pyramid	Quality assurance, financial auditing, security (defense in depth)
Test Doubles	Simulation, training environments, prototyping
Test Isolation	Scientific method, database transactions, process isolation
TDD as Design	Contract-first design, specification-driven development
Behavior vs. Implementation Testing	API versioning, information hiding, encapsulation
Property-Based Testing	Fuzzing, Monte Carlo methods, formal verification
Testing as Design Feedback	Usability testing, code review, documentation quality
Seams	Plugin architectures, extension points, dependency injection
Coverage Limitations	Goodhart's Law, vanity metrics, audit compliance
Mutation Testing	Fault injection, chaos engineering, adversarial testing
Contract Testing	Legal contracts, interface specifications, SLAs
Flaky Tests	Reproducibility, race conditions, Heisenbugs
Arrange-Act-Assert	Scientific method, debugging process, proof structure
Test Brittleness	Tight coupling, change impact analysis, technical debt

Source	Coverage
Beck, "Test-Driven Development" (2002)	TDD philosophy, red-green-refactor, test-first design
Meszaros, "xUnit Test Patterns" (2007)	Test doubles taxonomy, test structure patterns, test smells
Feathers, "Working Effectively with Legacy Code" (2004)	Seams, testability, testing untested code
Freeman & Pryce, "Growing Object-Oriented Software, Guided by Tests" (2009)	TDD as design technique, mock objects, outside-in development
Cohn, "Succeeding with Agile" (2009)	Test pyramid concept, test portfolio strategy
Claessen & Hughes, "QuickCheck" (2000)	Property-based testing foundations
Fowler, "Refactoring" (1999)	Behavior preservation, tests enabling refactoring
Humble & Farley, "Continuous Delivery" (2010)	Test automation in deployment pipelines
DeMillo, Lipton & Sayward, "Hints on Test Data Selection" (1978)	Mutation testing foundations