SLOs over dashboards: observability that drives decisions
17 June 2026 · 7 min read · Nintech Reliability
Every mature engineering organisation has a wall of dashboards, and most incidents are still detected by a customer. That is not an observability tooling gap — the data was almost always there, on a panel nobody was looking at. Dashboards show everything and decide nothing. Service level objectives fix the deciding part: they turn 'how is the system doing?' into a question with a numeric answer, an owner, and consequences.
Measure what users experience, not what servers do
An SLI is a ratio: good events over total events, measured as close to the user as you can get. 'Proportion of checkout requests that complete successfully in under 500ms, measured at the load balancer' is an SLI. 'CPU utilisation on the checkout hosts' is not — a service can serve every request perfectly at 95 per cent CPU and fail half of them at 30 per cent. The discipline is to work backwards from user journeys: for each critical flow, define availability (did it work?) and latency (was it fast enough?) as the user would judge them, and instrument at the boundary where the user's experience is actually determined.
Keep the set small. Three to five SLIs per service is plenty; twenty is a dashboard wearing an SLO costume. And measure latency as a distribution, not an average — a 200ms mean can hide a P99 of four seconds, and your most valuable users, the ones with the most data and the heaviest usage, live disproportionately in that tail. 'X per cent of requests faster than Y ms' is the formulation that survives contact with reality.
Set targets you can defend, then spend the budget
An SLO is a target for an SLI over a window — 99.9 per cent of requests successful over 28 days — and the error budget is its complement: at 99.9 per cent, you are allowed roughly 43 minutes of full unavailability a month. That reframing is the whole trick. Reliability stops being an aspiration ('as reliable as possible', which is unfalsifiable and infinitely expensive) and becomes a resource you allocate. Budget healthy? Ship faster, run the risky migration, use deployments as chaos testing. Budget exhausted? Feature work yields to reliability work, by prior agreement rather than by escalation.
This is the shared language between product and operations that neither had before. Product managers understand budgets; they spend them for a living. When the conversation is 'this launch carries an estimated 15 per cent of monthly error budget and we have 60 per cent remaining', you get an actual decision instead of ops saying no and product overruling. Two honest caveats: pick targets from what users tolerate and what you currently achieve, not from vanity — every added nine costs roughly ten times more, and nobody's checkout flow needs five of them. And the budget-exhaustion policy must be agreed and written down before the first breach, because negotiating it mid-incident produces whatever the loudest person wants.
Alert on symptoms, not causes
Cause-based alerts — disk 80 per cent full, replica lag high, pod restarted — fire constantly and mean nothing by themselves, because most causes never become user impact. Redundancy absorbs them, retries mask them, and meanwhile the on-call engineer is being trained to ignore pages. Symptom-based alerting inverts this: page only when the SLI is degrading, because that is the definition of something being actually wrong. Every failure mode you care about eventually manifests as errors or latency at the user boundary; alert there, and you cover the causes you never predicted too — which are, reliably, the ones that get you.
The refinement that makes this practical is burn-rate alerting. Rather than paging when the budget is gone (too late) or on any blip (too noisy), you page on how fast the budget is burning: a fast-burn alert — say 2 per cent of monthly budget consumed in an hour — wakes someone up, while a slow steady burn files a ticket for working hours. Multi-window burn rates, per the Google SRE workbook, tune out the false positives. Cause-based signals do not disappear — they become debugging context you consult after the symptom pages, dashboards and annotations rather than pages. The measurable outcome, and the one worth reporting upwards, is pages per on-call shift falling while detection of real incidents improves.
From 'is it up?' to 'are we within budget?'
The deepest change is cultural, and it is a shift from binary to continuous thinking. 'Is it up?' invites theatre — a status page showing green while a third of users cannot log in. 'Are we within budget?' admits what distributed systems actually are: partially degraded most of the time, with the interesting question being whether the degradation exceeds what you have collectively agreed users will tolerate. That framing also kills the perfection trap. 100 per cent is the wrong target for almost everything, and chasing it silently taxes every feature, every deploy, every experiment. An error budget is explicit permission to be imperfect on purpose, spent where it buys the most.
Adoption is where this usually dies, so start deliberately small: one user-critical service, two or three SLIs, targets set at roughly current performance so the budget maths starts honest, and a standing agenda item where humans look at budget consumption and decide something. Expect the first quarter's targets to be wrong; revising them is the process working, not failing. The failure mode to guard against is SLO theatre — objectives written into a wiki, breached quarterly, with no consequence attached. An SLO nobody acts on is just a dashboard with a threshold drawn on it, and you already had plenty of those.
Working on something like this? Talk to an engineer.