A plain 200 OK means nothing for an LLM: the canary measures the real time to first token, and validates both the content and format of the response — not just the HTTP status.
Most LLM incidents still return 200 — with a token that never arrives, incorrect content, or malformed JSON. That's exactly the state classic uptime monitoring can't see.
HTTP 200, TTFT under threshold, content and format valid.
200 but TTFT exceeded, incorrect content, or invalid format — the most common case in practice.
Non-200, timeout, connection refused, interrupted stream.
An isolated 429: quota/backoff remediation, not a provider incident.
Temperature 0, trivial expected reply ("PONG") — near-zero cost even at 1 check per minute.
TTFT is measured at the first text chunk, not the first HTTP byte — otherwise the degradation goes unnoticed.
A "PONG" canary and a JSON schema canary also catch silent structured-output regressions.
Streaming time-to-first-token, threshold calibrated per model — never a global threshold.
The expected reply is validated word for word — catches a response silently drifting off.
A second check validates a JSON schema — catches structured-output regressions after a model update.
Three technical families cover the market — an OpenAI-compatible parser alone covers over two-thirds of providers, self-hosted included. ★ = reference format, implemented natively; the others build on one of the three.
Anthropic, OpenAI and AWS Bedrock cover the market's three stream formats (typed SSE events, SSE chunks, binary event stream) — every other provider listed builds on one of these three parsers.