AI model monitoring, built for LLM APIs.

A plain 200 OK means nothing for an LLM. The canary measures the real time to first token and validates the content and format of the response, not just the HTTP status.

Max+5 AI models on Max · 20 on Enterprise
Canary checks · AI models LIVE
api.anthropic.com/v1/messagesclaude-haiku-4-5 UP 412 ms
api.openai.com/v1/chat/completionsgpt-4o-mini UP 380 ms
bedrock-runtime.eu-west-1claude-3-haiku DEGRADED 2.1 s
api.mistral.ai/v1mistral-small 429 rate limit
Four states

A binary state isn't enough for an LLM.

Most LLM incidents still return 200: a token that never arrives, incorrect content or malformed JSON. That's the state classic uptime monitoring can't see.

UP 200 · 412 ms
HTTP 200, time to first token under threshold, content and format valid.
DEGRADED 200 · 2.1 s
200, but TTFT exceeded, incorrect content or invalid format. The most common case in practice.
DOWN 503
Non-200, timeout, connection refused or interrupted stream.
RATE LIMIT 429
An isolated 429: quota and backoff to fix on your side, not a provider incident.
How it works

A canary, not a guess.

Three checks, near-zero cost.

01 Deterministic canary Temperature 0 and a trivial expected reply ("PONG"): near-zero cost even at one check per minute.
02 Timing the real first token TTFT is measured at the first text chunk, not the first HTTP byte. Otherwise the slowdown goes unnoticed.
03 Content + format assertions A "PONG" canary and a JSON schema canary also catch silent structured-output regressions.
Why timing matters

Time the first token, not the first byte.

Where most checks stop the clock, and where the problem actually is.

request sentTLS establishedheaders 200 → waitingfirst token → stream

HTTP headers arrive well before the first token. A check that stops the clock at the headers hides exactly the slowdown it's meant to catch.

What's included

Three canaries per model.

Calibrated per model, never a global threshold.

Real TTFT
Streaming time-to-first-token, with a threshold calibrated per model.
Content canary
The expected reply is validated word for word: catches a response silently drifting.
Format canary
A second check validates a JSON schema: catches structured-output regressions after a model update.
Providers

15 providers, three stream formats.

Under the hood, three stream parsers cover the whole market, self-hosted models included.

SSE chunks OpenAI-compatible
✓OpenAI✓Azure OpenAI✓Mistral✓Groq✓Together AI✓Fireworks✓OpenRouter✓DeepSeek✓xAI (Grok)✓Google Gemini✓Cohere✓Ollama✓vLLM / TGI
The most common format. Gemini and Cohere go through their own OpenAI-compatible endpoints.
Typed SSE events Anthropic
✓Anthropic
Claude models through the Messages API.
Binary event stream AWS Bedrock
✓AWS Bedrock
Every model hosted on Bedrock, in your AWS region.

Your provider isn't listed? If it speaks one of these three formats, it already works.

FAQ

Questions about AI model monitoring.

All questions →

From Max up: 5 AI models on Max, 20 on Enterprise. Free and Pro cover the other monitor types.

Your first monitor in under a minute.

Free plan, no card. Hosted and operated in Europe.