Batching to Amortize Overhead¶

Grouping multiple operations into a single batch spreads fixed overhead costs across more work items; improves throughput at the cost of latency for individual items

Tension Addressed¶

Latency vs. Throughput

Recognizable As¶

Bulk inserts, write coalescing, request batching, buffered I/O, batch APIs

Canonical Sources¶

Gray & Reuter, "Transaction Processing" (1992); Hennessy & Patterson, "Computer Architecture"

Transfers To¶

I/O optimization, network protocols, transaction processing, any domain with fixed per-operation overhead

The N+1 query problem is a canonical example: instead of N queries with overhead each, one batched query with overhead once.