Monitoring as code with Terraform and CI

Describe your monitors, alerts and status page as code, have every change reviewed in a merge request, and let CI apply it once merged. This tutorial builds a complete project end to end; the Terraform guide stays the reference for every attribute.

What you’ll build

A repository that monitors a small online shop:

• three monitors: the website, the payment API and the database;
• a Discord channel attached to all three, and a “Shop” group;
• a public status page and a scheduled maintenance window;
• a pipeline that shows the terraform plan in every merge request and applies it after merge on main.

Before you start

• The Max plan, for a Full API key (Settings → API Keys). Create a Read-only key too: it is enough for plan.
• Terraform 1.5 or later.
• A GitLab project. A GitHub Actions version is at the end of step 6.

1. The repository

okstatus/
├── versions.tf      # provider and backend
├── monitors.tf      # the three monitors
├── alerting.tf      # Discord channel and group
├── status.tf        # status page and maintenance
├── variables.tf
└── .gitlab-ci.yml

versions.tf

terraform {
  required_providers {
    okstatus = { source = "clearcloud-fr/okstatus", version = "~> 1.8" }
  }
  backend "http" {} # state kept by GitLab, configured by the CI (step 5)
}

provider "okstatus" {} # the key comes from OKSTATUS_API_KEY

variables.tf

variable "db_password" {
  type      = string
  sensitive = true
}

variable "discord_webhook" {
  type      = string
  sensitive = true
}

2. The monitors

monitors.tf

locals {
  regions = ["fr-dc3", "de-fra", "nl-ams"]
}

resource "okstatus_monitor" "site" {
  name     = "Shop · website"
  url      = "https://shop.example.com"
  interval = 60
  regions  = local.regions
  keyword  = "Add to cart"
}

resource "okstatus_monitor" "payments" {
  name                 = "Shop · payment API"
  url                  = "https://api.shop.example.com/health"
  interval             = 30
  regions              = local.regions
  latency_threshold_ms = 800
  slo_target           = 99.9
}

resource "okstatus_monitor" "db" {
  name                = "Shop · PostgreSQL"
  type                = "postgres"
  url                 = "postgres://db.shop.example.com:5432/shop"
  db_query            = "SELECT 1"
  basic_auth_user     = "okstatus_ro"
  basic_auth_password = var.db_password
  regions             = ["fr-dc3"]
}

With three regions, an alert goes out when at least two of them see the service down. One failing region on its own wakes nobody up.

3. Alert channel and group

alerting.tf

resource "okstatus_alert_channel" "discord" {
  type   = "discord"
  config = { webhookUrl = var.discord_webhook }
}

resource "okstatus_monitor_group" "shop" {
  name              = "Shop"
  monitor_ids       = [okstatus_monitor.site.id, okstatus_monitor.payments.id, okstatus_monitor.db.id]
  alert_channel_ids = [okstatus_alert_channel.discord.id]
}

Monitors that join the group get its channels. So don’t write alert_channel_ids on the monitors themselves: the next apply would detach the group’s channels from them.

4. Status page and maintenance

status.tf

resource "okstatus_status_page" "public" {
  slug          = "shop"
  title         = "Shop status"
  custom_domain = "status.shop.example.com"
}

resource "okstatus_maintenance" "pg_upgrade" {
  title           = "PostgreSQL upgrade"
  scheduled_start = "2026-11-12T02:00:00Z"
  scheduled_end   = "2026-11-12T03:00:00Z"
  monitor_ids     = [okstatus_monitor.db.id]
}

Pick the page’s monitors in the dashboard

The provider creates the page, its domain and its look, but not yet the list of monitors it shows. Add them once in Status pages → Sections.

5. The Terraform state

GitLab keeps the state and locks it during an apply: nothing to host. The backend "http" {} block from step 1 stays empty; the CI gives it the address at terraform init.

6. The pipeline

In Settings → CI/CD → Variables, create:

Variable	Value	Options
OKSTATUS_API_KEY	the Read-only key	masked
OKSTATUS_API_KEY_FULL	the Full key	masked, protected
TF_VAR_db_password, TF_VAR_discord_webhook	the secrets	masked, protected

A protected variable is only read on main. Merge requests compute their plan with the read-only key, and only the main branch can change your monitors.

.gitlab-ci.yml

image:
  name: hashicorp/terraform:1.9
  entrypoint: [""]

variables:
  TF_ADDR: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/okstatus

before_script:
  - terraform init -input=false
    -backend-config="address=${TF_ADDR}"
    -backend-config="lock_address=${TF_ADDR}/lock"
    -backend-config="unlock_address=${TF_ADDR}/lock"
    -backend-config="username=gitlab-ci-token"
    -backend-config="password=${CI_JOB_TOKEN}"
    -backend-config="lock_method=POST"
    -backend-config="unlock_method=DELETE"

stages: [plan, apply]

plan:
  stage: plan
  script:
    - terraform plan -input=false -out=plan.tfplan
  artifacts:
    paths: [plan.tfplan]
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      variables: # the real secrets are protected: placeholders for the review plan
        TF_VAR_db_password: merge-request-placeholder
        TF_VAR_discord_webhook: merge-request-placeholder
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

apply:
  stage: apply
  variables:
    OKSTATUS_API_KEY: $OKSTATUS_API_KEY_FULL
  script:
    - terraform apply -input=false plan.tfplan
  resource_group: okstatus # one apply at a time
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

The secrets (TF_VAR_…) are protected, so a merge request only gets placeholders: its plan shows the password and the webhook as changed, which is expected and never applied. On main, the job reads the real values; if one is missing, terraform plan stops with “No value for required variable” instead of applying a placeholder.

With GitHub Actions

GitHub has no built-in Terraform state: keep it in an S3-compatible bucket, for example on Scaleway Object Storage. Replace the backend of versions.tf:

versions.tf

  backend "s3" {
    bucket                      = "acme-terraform-state"
    key                         = "okstatus/terraform.tfstate"
    region                      = "fr-par"
    endpoints                   = { s3 = "https://s3.fr-par.scw.cloud" }
    use_lockfile                = true # Terraform 1.10 or later
    skip_credentials_validation = true
    skip_region_validation      = true
    skip_requesting_account_id  = true
    skip_s3_checksum            = true
  }

Add the read-only key as the repository secret OKSTATUS_API_KEY. Create an environment okstatus (Settings → Environments), restricted to main, holding OKSTATUS_API_KEY_FULL, DB_PASSWORD and DISCORD_WEBHOOK. The bucket keys (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY) go in the repository secrets.

.github/workflows/okstatus.yml

name: okstatus
on:
  pull_request:
  push:
    branches: [main]

env:
  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

jobs:
  plan:
    if: github.event_name == 'pull_request'
    runs-on: ubuntu-latest
    env:
      OKSTATUS_API_KEY: ${{ secrets.OKSTATUS_API_KEY }}
      TF_VAR_db_password: pull-request-placeholder
      TF_VAR_discord_webhook: pull-request-placeholder
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
      - run: terraform init -input=false
      - run: terraform plan -input=false

  apply:
    if: github.event_name == 'push'
    runs-on: ubuntu-latest
    environment: okstatus
    concurrency: okstatus
    env:
      OKSTATUS_API_KEY: ${{ secrets.OKSTATUS_API_KEY_FULL }}
      TF_VAR_db_password: ${{ secrets.DB_PASSWORD }}
      TF_VAR_discord_webhook: ${{ secrets.DISCORD_WEBHOOK }}
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
      - run: terraform init -input=false
      - run: terraform apply -input=false -auto-approve

The pitfalls

• A region outside your plan fails at apply, not at plan (error 400). Check your plan’s regions before merging.
• What you don’t write isn’t managed. A tag added in the dashboard stays and Terraform doesn’t see it. To pin everything, write the attribute, even empty: tags = [].
• Credentials are write-only. A password changed in the dashboard doesn’t show in the plan: change it in the CI variable.
• Existing monitors: import them with terraform import okstatus_monitor.site <id> before describing them, or you create duplicates.

Going further

• Terraform guide: every monitor type, private agents, import.
• Error budget in Grafana: follow the slo_target set here.
• Heartbeat monitors: watch the cron jobs of the same project.