Skip to content
OKStatus OKStatus Docs

LLM Canary

Overview

Section titled “Overview”
New LLM monitor dialog, step 1: provider, endpoint URL, API key and model

The LLM canary monitor sends a fixed prompt to an LLM provider’s chat-completions endpoint on a schedule and checks that it responds — and responds correctly — within acceptable time. It’s built for teams running production features on top of an LLM API who need to know the moment that dependency degrades, not just when it’s fully down.

LLM monitors are set up through their own dedicated flow (the LLM Monitor entry in the sidebar), separate from the general monitor creation wizard, since their configuration doesn’t overlap with a plain HTTP/TCP/DNS check.

Available on Max (up to 5 LLM monitors) and Enterprise (up to 20). An LLM monitor runs from a single region — the point is the provider’s health, not geographic reachability, and every check costs you real tokens.

Settings

Section titled “Settings”
SettingDescriptionDefault
Provideropenai-compatible, anthropic, or bedrockopenai-compatible
Endpoint URLThe provider’s chat-completions endpoint—
API keyCredential used to authenticate the request—
ModelThe model identifier to call—
PromptThe fixed prompt sent on every check—
RegionWhere the check runs from—
Max TTFTMaximum acceptable time-to-first-token, in ms, before the check is flagged as degraded1500 ms
Expected contentText the response must contain to pass the quality assertion—

Result states

Section titled “Result states”
Agent1 region fixed prompt · model streamed tokens LLM providerOpenAI-compatible · Anthropic · Bedrock TIMELINE OF ONE CHECK TTFT · time to first token TPOT · time per output token · ITL Only one region per LLM monitor — every region multiplies your real token spend. FOUR STATES up — in time, content matches degraded — slow TTFT or assertion failed ratelimit — 429, excluded from uptime % down — timeout, 5xx, no usable response

Unlike other monitor types, which only ever report up or down, LLM checks distinguish four states:

  • Up — Responded in time with content matching the assertion.
  • Degraded — Responded, but failed the quality assertion or exceeded the max TTFT (the provider is up, but not performing to spec).
  • Ratelimit — Got an isolated 429 from the provider — treated separately from a real outage so a burst of rate-limiting doesn’t page you as if the provider were down.
  • Down — No usable response at all (timeout, connection error, 5xx).

Use cases

Section titled “Use cases”
  • Catch silent quality regressions — A provider-side model update or an upstream config change can leave the endpoint technically “up” while responses degrade; the content assertion catches that.
  • Track latency drift — TTFT is often the first signal of provider-side capacity issues, well before outright failures.
  • Distinguish rate-limiting from outages — Avoid false alarms when you’re simply hitting your own quota.

Metrics collected

Section titled “Metrics collected”

Every check records the result state, response latency, time-to-first-token (TTFT), and whether the content assertion passed — feeding the same charts and PDF reports as every other monitor type.