curl http://127.0.0.1:8000/health

{"status":"ok","service":"pagerpal","version":"0.1.0"}

curl -u '<account-email>:<account-password>' http://127.0.0.1:8000/api/v1/system/jobs

{"jobs":[{"name":"notification_retry","enabled":true}]}

curl -u '<admin-email>:<admin-password>' -X POST http://127.0.0.1:8000/api/v1/alert-sources

{"id":1,"masked_api_key":"abc12345...wxyz"}

curl -u '<account-email>:<account-password>' http://127.0.0.1:8000/api/v1/alert-sources

[{"id":1,"source_type":"grafana","is_active":true}]

curl -u '<admin-email>:<admin-password>' -X POST http://127.0.0.1:8000/api/v1/alert-sources/1/regenerate-key

{"id":1,"api_key":"<revealed-once-alert-source-key>"}

curl -X POST http://127.0.0.1:8000/api/v1/alerts -H 'X-API-Key: <alert-source-key>'

{"id":42,"status":"triggered","severity":"critical"}

curl -X POST http://127.0.0.1:8000/api/v1/webhooks/grafana -H 'X-API-Key: <alert-source-key>'

{"id":42,"external_id":"monitoring-rule-id"}

curl -X POST http://127.0.0.1:8000/api/v1/webhooks/cloudwatch -H 'X-API-Key: <alert-source-key>'

{"status":"subscription_confirmed"}

curl -u '<account-email>:<account-password>' http://127.0.0.1:8000/api/v1/incidents

{"items":[],"total":125,"page":1,"total_pages":3}

curl -u '<account-email>:<account-password>' http://127.0.0.1:8000/api/v1/incidents/42

{"id":42,"status":"triggered","notification_count":0}

curl -u '<account-email>:<account-password>' -X POST http://127.0.0.1:8000/api/v1/incidents/42/ack

{"id":42,"status":"acknowledged","acknowledged_by":1}

curl -u '<account-email>:<account-password>' -X POST http://127.0.0.1:8000/api/v1/incidents/42/resolve

{"id":42,"status":"resolved","resolved_by":1}

curl -u '<account-email>:<account-password>' -X POST http://127.0.0.1:8000/api/v1/incidents/42/reopen

{"id":42,"status":"triggered"}

curl -u '<account-email>:<account-password>' -X POST http://127.0.0.1:8000/api/v1/incidents/42/escalate

{"id":42,"current_escalation_level":2}

curl -u '<account-email>:<account-password>' http://127.0.0.1:8000/api/v1/incidents/42/events

[{"event_type":"acknowledged","message":"Incident acknowledged"}]

curl -u '<account-email>:<account-password>' http://127.0.0.1:8000/api/v1/incidents/42/notification-logs

[{"status":"sent","channel":"sms"}]

curl -u '<account-email>:<account-password>' -X POST http://127.0.0.1:8000/api/v1/incidents/42/notification-logs/retry

{"retried_count":1,"queued_count":1}

PagerPal API Reference#

PagerPal exposes three API groups:

1. Operator/management API, used by the UI and administrative tooling.
2. Incident API, used for incident reads/actions and wrapped by the server-rendered UI.
3. Alert ingestion API, used by monitoring systems to create/update/resolve incidents.

The OpenAPI browser is available at /docs when the application is running, but it is not a replacement for the security notes below.

Authentication summary#

Note: API examples use placeholders only. Never commit or paste real API keys, passwords, tokens, or connection strings.

Operator API examples#

Management API routes accept HTTP Basic auth against real User Accounts. Use the account email as the username:

curl -u '<account-email>:<account-password>' \
  http://127.0.0.1:8000/api/v1/teams

The server-rendered UI uses /login to issue a signed pagerpal_session cookie. API Basic auth is retained for scripts and smoke checks, but it no longer uses shared process credentials.

Health check#

GET /health

Response:

{
  "status": "ok",
  "service": "pagerpal",
  "version": "0.1.0"
}

System worker status#

GET /api/v1/system/jobs
Authorization: Basic <base64-account-credentials>

Use this to confirm retry/escalation worker status from automation or deployment checks.

Account and password routes#

First-run admin bootstrap is exposed through the /login page when no login-enabled admin exists. For automation, use:

python scripts/create_admin.py --email '<admin-email>'

Change the current account password:

POST /api/v1/auth/password
Authorization: Basic <base64-account-credentials>
Content-Type: application/json

{
  "current_password": "old-password",
  "new_password": "new-long-password"
}

User create/update responses include role and last_login_at; password hashes are never returned. Creating a responder without a password remains valid and creates a responder-only identity.

Alert sources#

Create an alert source:

curl -u '<admin-email>:<admin-password>' \
  -X POST http://127.0.0.1:8000/api/v1/alert-sources \
  -H 'Content-Type: application/json' \
  -d '{
    "team_id": 1,
    "name": "Production Grafana",
    "source_type": "grafana",
    "config": null
  }'

Supported source_type values are generic, grafana, and cloudwatch. Generic sources send alerts to /api/v1/alerts; Grafana and CloudWatch sources use /api/v1/webhooks/grafana and /api/v1/webhooks/cloudwatch.

List alert sources:

curl -u '<account-email>:<account-password>' \
  http://127.0.0.1:8000/api/v1/alert-sources

Create and regenerate responses reveal the full api_key once so an operator can configure the monitoring system. Default list/get responses expose masked_api_key, not the full webhook secret:

[
  {
    "id": 1,
    "team_id": 1,
    "name": "Production Grafana",
    "source_type": "grafana",
    "config": null,
    "masked_api_key": "abc12345…wxyz",
    "is_active": true,
    "created_at": "2026-05-21T12:00:00"
  }
]

Regenerate a source key if a key is lost or suspected compromised:

curl -u '<admin-email>:<admin-password>' \
  -X POST http://127.0.0.1:8000/api/v1/alert-sources/1/regenerate-key

After regeneration, update the monitoring system with the new key.

The UI also includes a Send test alert action on /alert-sources. It creates a critical test incident using the selected source so operators can verify routing and notification logs without configuring an external monitoring system first.

Alert ingestion authentication#

Webhook endpoints accept an active alert source API key from any of these locations:

Prefer headers. Query-string keys often end up in web server logs, shell history, and screenshots.

Infobip delivery receipts#

Infobip delivery reports can update existing NotificationLog rows by messageId:

POST /api/v1/notifications/infobip/receipts
X-Infobip-Receipt-Token: <receipt-token>
Content-Type: application/json

{
  "results": [
    {
      "messageId": "infobip-message-id",
      "status": {
        "name": "DELIVERED",
        "description": "Delivered to handset"
      }
    }
  ]
}

DELIVERED marks the outbox row delivered; UNDELIVERABLE, EXPIRED, REJECTED, and FAILED mark it failed. Production deployments should set INFOBIP_RECEIPT_TOKEN and configure Infobip to send the same value as X-Infobip-Receipt-Token.

Generic alert ingestion#

POST /api/v1/alerts
X-API-Key: <alert-source-key>
Content-Type: application/json

Request body:

{
  "title": "CPU > 95% on prod-api-03",
  "description": "CPU has been above 95% for 10 minutes",
  "severity": "critical",
  "team_id": 1,
  "external_id": "prod-api-03-cpu-high",
  "source_id": 1
}

severity must be one of:

• critical
• warning
• info

external_id is important for deduplication. Repeated alerts with the same authenticated source and external ID update the open incident instead of creating a new duplicate incident.

Example:

curl -X POST http://127.0.0.1:8000/api/v1/alerts \
  -H 'Content-Type: application/json' \
  -H 'X-API-Key: <alert-source-key>' \
  -d '{
    "title": "CPU > 95% on prod-api-03",
    "description": "CPU has been above 95% for 10 minutes",
    "severity": "critical",
    "team_id": 1,
    "external_id": "prod-api-03-cpu-high",
    "source_id": 1
  }'

Successful response shape:

{
  "id": 42,
  "title": "CPU > 95% on prod-api-03",
  "description": "CPU has been above 95% for 10 minutes",
  "severity": "critical",
  "external_id": "prod-api-03-cpu-high",
  "source_id": 1,
  "team_id": 1,
  "status": "triggered",
  "current_escalation_level": 0,
  "assigned_user_id": 1,
  "acknowledged_by": null,
  "acknowledged_at": null,
  "resolved_by": null,
  "resolved_at": null,
  "first_notified_at": null,
  "last_notified_at": null,
  "notification_count": 0,
  "created_at": "2026-05-21T12:00:00"
}

Grafana webhook#

POST /api/v1/webhooks/grafana
X-API-Key: <alert-source-key>
Content-Type: application/json

Example alerting payload:

{
  "title": "RDS connection pool exhausted",
  "message": "Database connection pool is saturated",
  "state": "alerting",
  "ruleId": "rds-connection-pool",
  "labels": {
    "team_id": "1"
  }
}

PagerPal uses provider IDs/labels to derive a stable external ID. state: ok resolves the matching open incident when the alert source and external ID match.

CloudWatch/SNS webhook#

POST /api/v1/webhooks/cloudwatch
X-API-Key: <alert-source-key>
Content-Type: application/json

PagerPal accepts SNS SubscriptionConfirmation and Notification payloads. Confirmation messages must include an HTTPS AWS SNS SubscribeURL; PagerPal validates the host, does not follow redirects, fetches the URL, and returns:

{
  "status": "subscription_confirmed"
}

Notification messages are expected to include Message as a JSON-encoded CloudWatch alarm message.

Minimal example:

{
  "Type": "Notification",
  "Message": "{\"AlarmName\":\"prod-api-cpu-high\",\"NewStateValue\":\"ALARM\",\"NewStateReason\":\"CPU above threshold\",\"Trigger\":{}}",
  "MessageId": "example-message-id"
}

NewStateValue: ALARM creates or updates an incident. NewStateValue: OK resolves the matching open incident.

Incident API#

GET /api/v1/incidents supports:

List response shape:

{
  "items": [],
  "total": 125,
  "page": 1,
  "page_size": 50,
  "total_pages": 3
}

Core incident read/action endpoints:

Acknowledge, resolve, reopen, and manual escalation actions require a user_id body:

{
  "user_id": 1
}

The server-rendered UI wraps these actions with CSRF protection and a visible Acting as selector.

Schedule entries#

POST /api/v1/schedules/{schedule_id}/entries requires user_id to reference an active user and rejects normal entries whose time window overlaps another entry on the same schedule. Send is_override: true when the entry is intended to cover an existing window; override entries may overlap and are preferred by the current on-call lookup.

Route inventory#

Management resources#

Team member creation requires user_id to reference an active user. Existing inactive memberships stay visible for history/audit and can be removed.

Escalation level numbers must be unique within a policy. Duplicate level numbers return 409 because they make paging order ambiguous. Direct user assignees must reference active users; inactive user assignees return 400.

UI routes#

UI routes use a signed session cookie issued by /login; unauthenticated UI requests redirect to /login.

Configuration form posts under teams, users, schedules, escalation policies, and alert sources require admin. Incident action posts require responder or admin. Read routes accept any active login-enabled account.

Error handling notes#

Common status codes:

For browser forms, use UI routes rather than posting raw HTML form data directly to JSON API routes. The UI routes translate form submissions into application/API commands and redirect/render HTML responses.