Skip to content

Alert on Symptoms, Not Causes

Alerts should fire on user-visible symptoms (high latency, errors) not internal causes (CPU usage, queue depth) - symptoms indicate actual impact while causes often self-resolve or lack user impact.

Tension Addressed

Automation vs. Human Judgment - symptom alerts preserve human attention for real impact

Recognizable As

Alerts on SLO breach vs. resource thresholds; "users are affected" vs. "something looks wrong"; multi-signal alerting requiring impact evidence

Canonical Sources

Google SRE Book, Chapter 11 "Being On-Call" (2016); "My Philosophy on Alerting" (Rob Ewaschuk)

Transfers To

Medicine (symptom-based triage), Quality management (customer-facing metrics), Business operations (outcome metrics)

High CPU is not inherently a problem; slow user requests are. Alerting on causes creates noise and alert fatigue; alerting on symptoms ensures pages represent real user impact.