Alerts

Alerts watch your project's metrics — latency, cost, token usage, error counts, and evaluation scores — and notify you when they cross a threshold. Each alert has a rule (what to watch) and one notification channel (where to send it).

How alerts work

1. You define a rule: a metric, an aggregation, a comparison operator, a threshold, and a look-back window.
2. The evaluator runs on a schedule, querying your project's traces or sessions over the window.
3. If the aggregated value breaches the threshold, an alert event is created and the notification is dispatched.
4. After firing, the rule enters a cooldown — it will not fire again for that duration even if the condition still holds.
5. Events progress through three states: Firing → Acknowledged → Resolved.

Opening Alerts

Open Alerts under the Observability section of the left sidebar. The page has two tabs:

• Alert Events — every event the alert engine has produced, with current state and notification delivery status.
• Alert Rules — every rule you've configured. Toggle enabled, edit, or delete from here.

Click Create Alert at the top right to open the rule editor.

Create an alert

Metrics reference

System metrics work on both Trace and Session scopes:

Score metrics appear as score:<score-name> and are populated from your Online Evaluations. Only scores produced by online evaluation (source = EVAL, environment = online_evaluation) are available — manual annotation scores and experiment scores do not feed alert rules.

Notification channels

Webhook payload

When an alert fires, the notification worker POSTs this JSON body to your webhook URL:

{
  "alertId": "cuid-of-the-rule",
  "alertName": "High latency on chat agent",
  "projectId": "your-project-id",
  "state": "FIRING",
  "message": "AVG latency exceeded 500 ms (actual: 742.31 ms) over the last 5 min",
  "condition": {
    "metric": "latency",
    "aggregation": "AVG",
    "operator": "GT",
    "threshold": 500,
    "scope": "TRACE",
    "windowMinutes": 5
  },
  "evaluation": {
    "metricValue": 742.31,
    "breached": true
  },
  "firedAt": "2026-05-11T14:00:00.000Z"
}

Delivery semantics:

• Method: POST
• Content-Type: application/json
• Your Custom Headers are merged into the request headers (your Content-Type will be overridden if you set one).
• Timeout: 10 seconds.
• Retries: up to 3 attempts with exponential backoff (handled by the queue).
• 4xx responses (except 429) are treated as permanent failures and skipped to the dead-letter queue immediately — fix your endpoint and resave the rule rather than waiting for a retry.

Email

Email recipients receive a formatted message containing the alert name, the breached condition, the actual metric value, and the time it fired. Email delivery uses the same retry policy as webhooks.

Slack

When an alert fires, a formatted message is posted to the Slack channel selected on the rule. Connect Slack once for your organization in Settings → Integrations (an admin authorizes the BrowserStack Slack app); after that, the Slack channel option lists your workspace's channels when you create or edit an alert.

Alert events

The Alert Events tab shows every event the engine has produced. Each event has a state:

Each event also records notificationResults — a per-channel delivery log: { channel, status, error? }. This is how you debug a notification that didn't arrive.

Permissions

Reading the Alerts page requires the alerts:read RBAC scope. Creating, editing, deleting rules, and acknowledging or resolving events use the corresponding write scopes on the same resource.

Limits

Use cases

• Cost guardrails — AVG total_cost > 0.05 on a 1-hour window catches a runaway pipeline cheaper than discovering it in next month's bill.
• Latency regression detection — P95 latency > 2000 ms on a 15-minute window over production only.
• Hallucination spike — AVG score:faithfulness < 0.6 on a 15-minute window, paired with an online evaluation that scores faithfulness.
• Error surge — SUM error_count > 10 on a 5-minute window — fastest signal that something is misconfigured downstream.