Guardrails
Author safety and policy rules in the evaluator library, validate them against a labeled dataset, and promote them live for the SDK to enforce around your model calls.
Guardrails
Guardrails are policy rules that inspect the content going into a model call (or coming back out) and act on it — redacting sensitive text, returning a safe canned response, throwing an error, or re-asking the model. You author and manage guardrails in the platform; the SDK enforces them in your application at runtime.
A guardrail is a kind of evaluator. It lives in the same evaluator library as your other evaluators, carries a name, a version history, and a lifecycle state, and every decision it makes is recorded as a score on the trace it ran on.
Guardrails are a gated feature and are off by default. Contact BrowserStack support to enable guardrails for your project. Until they are enabled, the authoring UI, the guardrails API, and SDK enforcement are all inactive.
How guardrails work
Each guardrail carries three decisions that together define its behavior:
| Concept | Values | Meaning |
|---|---|---|
| Hook point | Before model, After model, Before agent, After agent, Before tool call, After tool call | Where in the agent run the guardrail fires |
| On match | Return a default response, Redact / filter the match, Throw a 4xx error, Re-ask the model | What happens when the rule matches. Available actions depend on the hook point |
| Rule logic | Deterministic (keyword, regex, JSON schema, PII) or an LLM judge | How the rule decides whether the content matches |
Once a guardrail is live, you can enforce it from the SDK in your application. See Enforce guardrails from the SDK for how enforcement works.
A guardrail never breaks your application. If a rule cannot be resolved or evaluated — the configuration fetch fails, the judge times out, the payload is too large — the original call proceeds unenforced rather than erroring (this behavior is known as failing open). Every such skipped check is recorded as a score so you keep a complete audit trail. See Monitor guardrail decisions.
Authoring a guardrail
Guardrails are created from the evaluator library. Open Evaluators in the left sidebar, then open New LLM Evaluator or New Code Evaluator and pick one of the built-in guardrail templates (marked with a Guardrail badge). This pre-fills the hook point, action, and rule logic.
Selecting a guardrail template opens the guardrail editor, where you can adjust the fields before saving.
Form fields
Name
Give the guardrail a unique name. The SDK references guardrails by name, so the name is the lookup key and cannot be changed after creation — editing forks a new version under the same name.
Hook point
Choose where the guardrail fires: Before model, After model, Before agent, After agent, Before tool call, or After tool call. A "before" hook point guards the content going in; an "after" hook point guards the content coming back.
For example, if you don't want PII reaching the model, use a code-based PII detection guardrail at Before model so the prompt is checked before it is sent. See Common configurations for more patterns.
On match
Choose what happens when the rule matches. The available actions depend on the hook point above:
| Action | What it does |
|---|---|
| Return a default response | Stops the call and returns a fixed reply you define (the Default response text, e.g. "I can't help with that.") in place of the model's output. The caller gets your safe fallback instead of the flagged content. |
| Redact / filter the match | Doesn't block the call — masks only the matched portion (for example, replacing an email address or card number) and lets the cleaned-up content continue. Use it to allow the call while stripping the sensitive bits. |
| Throw a 4xx error | Stops the call and raises a 422 error in your application, so your own code decides what to do next (show an error, log it, retry). Nothing is returned to the model or user automatically. |
| Re-ask the model | Instead of returning the flagged output, asks the model to answer again — a chance to produce a compliant response. Available on after-model / after-agent hook points. |
When the action is Return a default response, a Default response field appears (placeholder I can't help with that.) for the substitute text.
Rule logic
The rule mechanism is set by the template you started from — it is not chosen freely on the form.
Deterministic rules show a Rule type selector:
| Rule type | Field | Notes |
|---|---|---|
| Keyword match | Keywords | Comma-separated; case-insensitive substring match |
| Regex pattern | Pattern | A regular expression; any text matching it is flagged (e.g. \d{3}-\d{2}-\d{4} matches a US SSN). Patterns use RE2 syntax. |
| PII detection | PII entities | Comma-separated; supported entities: email, credit_card, ip_address, mac_address, url |
| JSON schema | JSON schema | The output is flagged when it fails to validate against this schema |
LLM judge rules show a Judge prompt field and a provider / model picker. Describe what the judge should flag, and reference the content with {{input}} / {{output}}. At enforcement time the judge uses the selected model.
Saving a new guardrail creates it as a draft — it does not enforce anything until you validate and promote it.
Lifecycle: draft → validate → promote
A guardrail moves through three states, shown as a badge in the library and on the editor: Draft, Live, and Disabled. Only a Live guardrail is fetched and enforced by the SDK.
Draft --> validate --> promote --> Live --> demote --> Disabled
^ |
| v
+----------------- edit (forks a new draft) ------------------+Editing a live or disabled guardrail forks a new Draft — you re-validate and promote it to make it live again.
Validate
Before a guardrail can go live it must pass a validation run on its current version. Validation matters for two reasons:
- Accuracy — it shows whether the guardrail triggers on the right content before it starts acting on real traffic, so you don't ship a rule that over-blocks or misses.
- Time and cost — it runs the guardrail exactly as production will, so you can see the latency each check adds and (for LLM-judge guardrails) roughly what it will cost per call in your application — before you commit to it.
Click Validate to open the validation dialog.
Validation runs the guardrail against a labeled dataset and measures how well its decisions match the labels. Each dataset item's expected-output column must be a trigger label:
- Should-trigger labels:
block,redact,fail,unsafe, … - Should-pass labels:
pass,allow,safe, …
Items with any other value are skipped.
Click Run validation. A guardrail "triggers" when it decides to act (block or redact); the results compare its triggers against your labels:
| Metric | What it tells you |
|---|---|
| Precision | Of the items the guardrail triggered on, the share that should have triggered. Low precision means false alarms — it flags content that was actually fine. |
| Recall | Of the items that should have triggered, the share it actually caught. Low recall means misses — content that should have been caught slipped through. |
| F1 | A single 0–1 score that balances precision and recall, so you can compare versions at a glance. |
| Correctly triggered | Items that should trigger and did. |
| Wrongly triggered | Items that should have passed but triggered — false alarms. |
| Correctly passed | Items that should pass and did. |
| Missed (should trigger) | Items that should have triggered but didn't — what slipped through. |
| Latency p50 / p95 / p99 | How long one check takes: the typical case (p50) and the slow tail (p95 / p99). This is the delay the guardrail will add to each call in production. |
| Judge cost | For LLM-judge guardrails, the estimated cost of this validation run — a preview of per-call cost at scale. |
For LLM-judge guardrails, validation makes real model calls using the judge model you selected on the guardrail — or your project's default evaluation model if you didn't set one. Runs are bounded (up to 200 items for judge guardrails) so a validation stays fast and inexpensive. Deterministic guardrails are evaluated against every labeled item.
Promote
Once a validation run has passed on the guardrail's current version, a Promote button appears. Promoting sets the guardrail Live.
A guardrail can only be promoted if it has a validation run pinned to its current version. If you edit the guardrail after validating, you must re-validate before you can promote again. Only one version of a given guardrail name can be live in a project at a time — promoting a new version automatically demotes the previous one.
Edit, demote, and archive
- Editing a live guardrail demotes it to draft. The editor confirms this first (Editing a live guardrail → Save anyway); you must re-validate before it can go live again.
- Demote takes a live guardrail offline (state becomes Disabled). Attached agents stop applying it on their next rule refresh.
- Archiving removes all versions of the guardrail.
Built-in templates
The library ships ten guardrail templates you can start from:
| Template | Rule logic |
|---|---|
| PII Detect & Redact | PII detection |
| Regex Match | Regex pattern |
| Blocked Keywords | Keyword match |
| Sensitive Tool Call | Keyword match (before tool call) |
| Prompt Injection | LLM judge |
| Jailbreak Detection | LLM judge |
| Toxicity | LLM judge |
| Hallucination | LLM judge (after model) |
| Off-topic | LLM judge |
| Brand Safety | LLM judge |
Common configurations
Some patterns to start from — each maps to a template above:
| Goal | Rule logic | Hook point | On match |
|---|---|---|---|
| Keep PII out of prompts sent to the model | Code-based PII detection | Before model | Redact (or Throw a 4xx error) |
| Redact PII or secrets from a model's response | Code-based PII / Regex | After model | Redact |
| Block prompt-injection or jailbreak attempts | LLM judge (Prompt Injection / Jailbreak) | Before model | Throw a 4xx error or Return a default response |
| Stop toxic or off-brand output from reaching the user | LLM judge (Toxicity / Brand Safety) | After model | Re-ask the model or Return a default response |
| Catch hallucinated answers before they return | LLM judge (Hallucination) | After model | Re-ask the model |
| Guard a sensitive tool from being called | Code-based keyword match | Before tool call | Throw a 4xx error |
Prefer a code-based (deterministic) rule when the check is exact — PII, keywords, regex, or JSON shape — because it is fast and free. Reach for an LLM judge when the check is subjective, such as tone, intent, or factuality.
Permissions
Authoring, validating, and promoting a guardrail require permission to create and manage evaluators. Demoting a live guardrail is additionally restricted to a project Owner or Admin, or the guardrail's original author — because taking a live safety control offline is a sensitive change.