Latency vs. Throughput Trade-off¶

Optimizing for fastest individual response often conflicts with maximizing total work completed; batching and queuing improve throughput at the cost of individual latency

Tension Addressed¶

Latency vs. Throughput

Recognizable As¶

Batch processing systems, request queuing, buffer management, latency percentiles vs. requests/second metrics

Canonical Sources¶

Little, "A Proof for the Queuing Formula: L = lambda W" (1961); Gregg, "Systems Performance" (2020)

Transfers To¶

Manufacturing, transportation logistics, network design, any resource scheduling domain

Sub-concepts: - Little's Law: L = lambda * W (average items in system = arrival rate * average wait time) - Latency Percentiles: p50, p95, p99 reveal distribution better than averages