Automation rules watch your traces (or sessions) and, when one matches, run one or more actions on it — score it with an evaluator list (ONLINE_EVAL), add it to a dataset (DATASET), or both.
The CLASSIFIER action type shown in the dashboard is not available on the public API yet. Sending a rule with type: "CLASSIFIER" returns 400. Support is planned for a future version.
Use this to update rule-level fields like name, description, samplingRate, or targetingFilters. You can't change sourceType. To add or remove actions, use the action-level methods further down; to toggle per-metric routing, use updateAction() since metrics belong to the action, not the rule.
Updates the rule. Only the fields you send are changed —
anything you leave out stays the same. A few things to know:
- You can't change `sourceType`.
- To add or remove actions, use the `/actions` endpoints below.
To toggle per-metric human-review routing, use
`PATCH /actions/{actionId}` (metrics belong to an action,
not to the rule).
- Pass `description: null` to clear the description.
- `targetingFilters`, if you send it, replaces the whole filter
list — send the full set you want, not a delta.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
name
string
false
New rule name.
description
string | null
false
New description. Pass `null` to clear it.
samplingRate
number
false
New sampling rate (1–100).
status
AutomationRuleStatus
false
New status — `ACTIVE` or `PAUSED`.
targetingFilters
TargetingFilter[]
false
New targeting filters. Pass an empty array to clear.
targetingFilters.fieldName
string
true
The trace attribute to compare (for example `name`, `metadata.userId`, `input`, `output`).
updateStatus() pauses or resumes a rule. It's a convenience shortcut over update() — the effect is the same as calling update(ruleId, { status: 'PAUSED' }) or { status: 'ACTIVE' }, but it reads more clearly at the call site.
A convenience shortcut for pausing or resuming a rule. Same as
`PATCH /automation-rules/{ruleId}` with just `{ status }`, but
reads more clearly at the call site.
Add a second action to an existing rule — for example, attach a DATASET action to a rule that already runs ONLINE_EVAL. A rule can hold at most one action of each type, so adding a second ONLINE_EVAL (or DATASET) to a rule that already has one is rejected.
Returns the full updated rule, so you don't need a follow-up get().
Adds one action to an existing rule. Pass the action as a nested
`action` object — the same shape as an entry in `actions[]` when
you create a rule. A rule can hold at most one action of each
type, so adding a second action of the same type returns `409`.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
action
AutomationRuleActionInput
true
The action to add. A rule can hold up to two actions.
Param
Type
Required
Description
rule_id
str
true
The automation rule's id.
action
AutomationRuleActionInput
true
The action to add. A rule can hold up to two actions.
Param
Type
Required
Description
ruleId
String
true
The automation rule's id.
action
AutomationRuleActionInput
true
The action to add. A rule can hold up to two actions.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
Action
AutomationRuleActionInput
true
The action to add. A rule can hold up to two actions.
Update an action's config in place. For ONLINE_EVAL actions, you can optionally pass a metrics array to flip routeToHumanReview on individual evaluators (up to 50 per call).
Updates the action's `config` in place. For `ONLINE_EVAL`
actions you can also pass a `metrics` list to flip
`routeToHumanReview` per metric — this endpoint is the only way
to change those toggles.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
actionId
string
true
The action's id (returned in `actions[].id` on the rule).
action
AutomationRuleActionUpdateInput
true
The fields to update on the action. Anything you leave out stays the same.
Param
Type
Required
Description
rule_id
str
true
The automation rule's id.
action_id
str
true
The action's id (returned in `actions[].id` on the rule).
action
AutomationRuleActionUpdateInput
true
The fields to update on the action. Anything you leave out stays the same.
Param
Type
Required
Description
ruleId
String
true
The automation rule's id.
actionId
String
true
The action's id (returned in `actions[].id` on the rule).
action
AutomationRuleActionUpdateInput
true
The fields to update on the action. Anything you leave out stays the same.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
actionId
string
true
The action's id (returned in `actions[].id` on the rule).
Action
AutomationRuleActionUpdateInput
true
The fields to update on the action. Anything you leave out stays the same.
Use this to update rule-level fields like name, description, sampling_rate, or targeting_filters. You can't change source_type. To add or remove actions, use the action-level methods further down; to toggle per-metric routing, use update_action() since metrics belong to the action, not the rule.
Updates the rule. Only the fields you send are changed —
anything you leave out stays the same. A few things to know:
- You can't change `sourceType`.
- To add or remove actions, use the `/actions` endpoints below.
To toggle per-metric human-review routing, use
`PATCH /actions/{actionId}` (metrics belong to an action,
not to the rule).
- Pass `description: null` to clear the description.
- `targetingFilters`, if you send it, replaces the whole filter
list — send the full set you want, not a delta.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
name
string
false
New rule name.
description
string | null
false
New description. Pass `null` to clear it.
samplingRate
number
false
New sampling rate (1–100).
status
AutomationRuleStatus
false
New status — `ACTIVE` or `PAUSED`.
targetingFilters
TargetingFilter[]
false
New targeting filters. Pass an empty array to clear.
targetingFilters.fieldName
string
true
The trace attribute to compare (for example `name`, `metadata.userId`, `input`, `output`).
update_status() pauses or resumes a rule. It's a convenience shortcut over update() — the effect is the same as calling update(rule_id, status='PAUSED') or status='ACTIVE', but it reads more clearly at the call site.
A convenience shortcut for pausing or resuming a rule. Same as
`PATCH /automation-rules/{ruleId}` with just `{ status }`, but
reads more clearly at the call site.
Add a second action to an existing rule — for example, attach a DATASET action to a rule that already runs ONLINE_EVAL. A rule can hold at most one action of each type, so adding a second ONLINE_EVAL (or DATASET) to a rule that already has one is rejected.
Returns the full updated rule, so you don't need a follow-up get().
Adds one action to an existing rule. Pass the action as a nested
`action` object — the same shape as an entry in `actions[]` when
you create a rule. A rule can hold at most one action of each
type, so adding a second action of the same type returns `409`.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
action
AutomationRuleActionInput
true
The action to add. A rule can hold up to two actions.
Param
Type
Required
Description
rule_id
str
true
The automation rule's id.
action
AutomationRuleActionInput
true
The action to add. A rule can hold up to two actions.
Param
Type
Required
Description
ruleId
String
true
The automation rule's id.
action
AutomationRuleActionInput
true
The action to add. A rule can hold up to two actions.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
Action
AutomationRuleActionInput
true
The action to add. A rule can hold up to two actions.
Update an action's config in place. For ONLINE_EVAL actions, you can optionally pass a metrics array to flip route_to_human_review on individual evaluators (up to 50 per call).
Updates the action's `config` in place. For `ONLINE_EVAL`
actions you can also pass a `metrics` list to flip
`routeToHumanReview` per metric — this endpoint is the only way
to change those toggles.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
actionId
string
true
The action's id (returned in `actions[].id` on the rule).
action
AutomationRuleActionUpdateInput
true
The fields to update on the action. Anything you leave out stays the same.
Param
Type
Required
Description
rule_id
str
true
The automation rule's id.
action_id
str
true
The action's id (returned in `actions[].id` on the rule).
action
AutomationRuleActionUpdateInput
true
The fields to update on the action. Anything you leave out stays the same.
Param
Type
Required
Description
ruleId
String
true
The automation rule's id.
actionId
String
true
The action's id (returned in `actions[].id` on the rule).
action
AutomationRuleActionUpdateInput
true
The fields to update on the action. Anything you leave out stays the same.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
actionId
string
true
The action's id (returned in `actions[].id` on the rule).
Action
AutomationRuleActionUpdateInput
true
The fields to update on the action. Anything you leave out stays the same.
// Build a rule with two actions: an ONLINE_EVAL that runs an// evaluator list on a sampled slice of production traces, and a// DATASET action that captures the trace input/output as// expected-output pairs into a dataset for later experimentation.CreateAutomationRuleBody createBody = new CreateAutomationRuleBody();createBody.name = name;createBody.status = AutomationRuleStatus.ACTIVE;createBody.samplingRate = 50;createBody.sourceType = EventSchedulerType.TRACE;OnlineEvalActionInput evalAction = new OnlineEvalActionInput();evalAction.type = OnlineEvalActionInputType.ONLINE_EVAL;evalAction.config = new OnlineEvalActionConfigInput();evalAction.config.concurrency = 2;evalAction.config.delaySeconds = 30;OnlineEvalMetricInput metric = new OnlineEvalMetricInput();metric.evaluatorListId = "ee_latency_v1";metric.routeToHumanReview = false;evalAction.metrics = Collections.singletonList(metric);DatasetActionInput datasetAction = new DatasetActionInput();datasetAction.type = DatasetActionInputType.DATASET;datasetAction.config = new DatasetActionConfigInput();datasetAction.config.datasetId = "ds_prod_captures";datasetAction.config.mappingType = DatasetMappingType.EXPECTED_OUTPUT;datasetAction.config.extractionMode = DatasetExtractionMode.SINGLE_IO;createBody.actions = Arrays.asList(evalAction, datasetAction);AutomationRuleConfig rule = automationRules.create(createBody);
Creates the rule, its actions, and its targeting filters in a
single call. You need at least one action, and each action type
can appear at most once.
Param
Type
Required
Description
name
string
true
A name for the rule, 1–255 characters. Must be unique within the project.
description
string | null
false
Optional free-form description.
sourceType
EventSchedulerType
false
Whether the rule watches traces or sessions. Defaults to `TRACE`.
samplingRate
number
false
Percent of matching traces the rule fires on (1–100). Useful for sampling high-volume traffic.
Default: 100
status
AutomationRuleStatus
false
The starting status. Set to `PAUSED` if you want the rule created but not running yet.
Default: "ACTIVE"
targetingFilters
TargetingFilter[]
false
The conditions a trace must meet to match. Leave empty to fire on every trace.
Default: []
targetingFilters.fieldName
string
true
The trace attribute to compare (for example `name`, `metadata.userId`, `input`, `output`).
Use this to update rule-level fields like name, description, samplingRate, or targetingFilters. You can't change sourceType. To add or remove actions, use the action-level methods further down; to toggle per-metric routing, use updateAction() since metrics belong to the action, not the rule.
// Partial update — only non-null fields are sent on the wire.UpdateAutomationRuleBody updateBody = new UpdateAutomationRuleBody();updateBody.samplingRate = 75;updateBody.description = "updated description";AutomationRuleConfig updated = automationRules.update(ruleId, updateBody);
Updates the rule. Only the fields you send are changed —
anything you leave out stays the same. A few things to know:
- You can't change `sourceType`.
- To add or remove actions, use the `/actions` endpoints below.
To toggle per-metric human-review routing, use
`PATCH /actions/{actionId}` (metrics belong to an action,
not to the rule).
- Pass `description: null` to clear the description.
- `targetingFilters`, if you send it, replaces the whole filter
list — send the full set you want, not a delta.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
name
string
false
New rule name.
description
string | null
false
New description. Pass `null` to clear it.
samplingRate
number
false
New sampling rate (1–100).
status
AutomationRuleStatus
false
New status — `ACTIVE` or `PAUSED`.
targetingFilters
TargetingFilter[]
false
New targeting filters. Pass an empty array to clear.
targetingFilters.fieldName
string
true
The trace attribute to compare (for example `name`, `metadata.userId`, `input`, `output`).
updateStatus() pauses or resumes a rule. It's a convenience shortcut over update() — the effect is the same as calling update(ruleId, UpdateRequest.builder().status("PAUSED").build()), but it reads more clearly at the call site.
// Pause the rule while iterating on evaluator config.UpdateAutomationRuleStatusBody pauseBody = new UpdateAutomationRuleStatusBody();pauseBody.status = AutomationRuleStatus.PAUSED;AutomationRuleConfig paused = automationRules.updateStatus(ruleId, pauseBody);// ...and resume when ready.UpdateAutomationRuleStatusBody resumeBody = new UpdateAutomationRuleStatusBody();resumeBody.status = AutomationRuleStatus.ACTIVE;AutomationRuleConfig resumed = automationRules.updateStatus(ruleId, resumeBody);
A convenience shortcut for pausing or resuming a rule. Same as
`PATCH /automation-rules/{ruleId}` with just `{ status }`, but
reads more clearly at the call site.
Add a second action to an existing rule — for example, attach a DATASET action to a rule that already runs ONLINE_EVAL. A rule can hold at most one action of each type, so adding a second ONLINE_EVAL (or DATASET) to a rule that already has one is rejected.
// Attach another DATASET action to an existing rule — captures the// LLM output into a second dataset for regression comparison.DatasetActionInput extraDataset = new DatasetActionInput();extraDataset.type = DatasetActionInputType.DATASET;extraDataset.config = new DatasetActionConfigInput();extraDataset.config.datasetId = "ds_regression_capture";extraDataset.config.mappingType = DatasetMappingType.LLM_OUTPUT;extraDataset.config.extractionMode = DatasetExtractionMode.SINGLE_IO;AddAutomationRuleActionBody addActionBody = new AddAutomationRuleActionBody();addActionBody.action = extraDataset;AutomationRuleConfig addedAction = automationRules.addAction(ruleId, addActionBody);
Returns the full updated rule, so you don't need a follow-up get().
Adds one action to an existing rule. Pass the action as a nested
`action` object — the same shape as an entry in `actions[]` when
you create a rule. A rule can hold at most one action of each
type, so adding a second action of the same type returns `409`.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
action
AutomationRuleActionInput
true
The action to add. A rule can hold up to two actions.
Param
Type
Required
Description
rule_id
str
true
The automation rule's id.
action
AutomationRuleActionInput
true
The action to add. A rule can hold up to two actions.
Param
Type
Required
Description
ruleId
String
true
The automation rule's id.
action
AutomationRuleActionInput
true
The action to add. A rule can hold up to two actions.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
Action
AutomationRuleActionInput
true
The action to add. A rule can hold up to two actions.
Update an action's config in place. For ONLINE_EVAL actions, you can optionally pass a metrics array to flip routeToHumanReview on individual evaluators (up to 50 per call).
// Retune the ONLINE_EVAL action's concurrency + delay in place.OnlineEvalActionUpdateInput retuned = new OnlineEvalActionUpdateInput();retuned.type = OnlineEvalActionInputType.ONLINE_EVAL;retuned.config = new OnlineEvalActionConfigInput();retuned.config.concurrency = 8;retuned.config.delaySeconds = 120;UpdateAutomationRuleActionBody updateActionBody = new UpdateAutomationRuleActionBody();updateActionBody.action = retuned;AutomationRuleConfig updatedAction = automationRules.updateAction(ruleId, actionId, updateActionBody);
Updates the action's `config` in place. For `ONLINE_EVAL`
actions you can also pass a `metrics` list to flip
`routeToHumanReview` per metric — this endpoint is the only way
to change those toggles.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
actionId
string
true
The action's id (returned in `actions[].id` on the rule).
action
AutomationRuleActionUpdateInput
true
The fields to update on the action. Anything you leave out stays the same.
Param
Type
Required
Description
rule_id
str
true
The automation rule's id.
action_id
str
true
The action's id (returned in `actions[].id` on the rule).
action
AutomationRuleActionUpdateInput
true
The fields to update on the action. Anything you leave out stays the same.
Param
Type
Required
Description
ruleId
String
true
The automation rule's id.
actionId
String
true
The action's id (returned in `actions[].id` on the rule).
action
AutomationRuleActionUpdateInput
true
The fields to update on the action. Anything you leave out stays the same.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
actionId
string
true
The action's id (returned in `actions[].id` on the rule).
Action
AutomationRuleActionUpdateInput
true
The fields to update on the action. Anything you leave out stays the same.
A rule always needs at least one action, so deleting the last remaining action returns 400. Delete the whole rule instead if that's what you want.
// Remove an action from a rule (the rule must retain at least one// action — deleting the last remaining action returns 409).SuccessAck deletedActionResult = automationRules.deleteAction(ruleId, actionId);
A rule always needs at least one action. Deleting the last
remaining action returns `409` — delete the whole rule instead.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
actionId
string
true
The action's id (returned in `actions[].id` on the rule).
Param
Type
Required
Description
rule_id
str
true
The automation rule's id.
action_id
str
true
The action's id (returned in `actions[].id` on the rule).
Param
Type
Required
Description
ruleId
String
true
The automation rule's id.
actionId
String
true
The action's id (returned in `actions[].id` on the rule).
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
actionId
string
true
The action's id (returned in `actions[].id` on the rule).
Every step above, stitched together into one block you can copy and adapt. Fill in ruleId / actionId where a later step needs the id.
// 1. Set up the SDKimport com.browserstack.aisdk.AISDK;import com.browserstack.aisdk.clients.automation_rule.AddAutomationRuleActionBody;import com.browserstack.aisdk.clients.automation_rule.AutomationRule;import com.browserstack.aisdk.clients.automation_rule.AutomationRuleConfig;import com.browserstack.aisdk.clients.automation_rule.AutomationRuleStatus;import com.browserstack.aisdk.clients.automation_rule.CreateAutomationRuleBody;import com.browserstack.aisdk.clients.automation_rule.DatasetActionConfigInput;import com.browserstack.aisdk.clients.automation_rule.DatasetActionInput;import com.browserstack.aisdk.clients.automation_rule.DatasetActionInputType;import com.browserstack.aisdk.clients.automation_rule.DatasetExtractionMode;import com.browserstack.aisdk.clients.automation_rule.DatasetMappingType;import com.browserstack.aisdk.clients.automation_rule.DeleteAutomationRuleResponse;import com.browserstack.aisdk.clients.automation_rule.EventSchedulerType;import com.browserstack.aisdk.clients.automation_rule.ListAutomationRulesParams;import com.browserstack.aisdk.clients.automation_rule.ListAutomationRulesResponse;import com.browserstack.aisdk.clients.automation_rule.OnlineEvalActionConfigInput;import com.browserstack.aisdk.clients.automation_rule.OnlineEvalActionInput;import com.browserstack.aisdk.clients.automation_rule.OnlineEvalActionInputType;import com.browserstack.aisdk.clients.automation_rule.OnlineEvalActionUpdateInput;import com.browserstack.aisdk.clients.automation_rule.OnlineEvalMetricInput;import com.browserstack.aisdk.clients.automation_rule.SuccessAck;import com.browserstack.aisdk.clients.automation_rule.UpdateAutomationRuleActionBody;import com.browserstack.aisdk.clients.automation_rule.UpdateAutomationRuleBody;import com.browserstack.aisdk.clients.automation_rule.UpdateAutomationRuleStatusBody;import java.util.Arrays;import java.util.Collections;import java.util.List;AISDK sdk = AISDK.builder() .publicKey(System.getenv("AISDK_PUBLIC_KEY")) .secretKey(System.getenv("AISDK_SECRET_KEY")) .build();AutomationRule automationRules = sdk.automationRule();// 2. Per-run variablesString name = "Score production traces on latency evaluator";// 3. Create a rule// Build a rule with two actions: an ONLINE_EVAL that runs an// evaluator list on a sampled slice of production traces, and a// DATASET action that captures the trace input/output as// expected-output pairs into a dataset for later experimentation.CreateAutomationRuleBody createBody = new CreateAutomationRuleBody();createBody.name = name;createBody.status = AutomationRuleStatus.ACTIVE;createBody.samplingRate = 50;createBody.sourceType = EventSchedulerType.TRACE;OnlineEvalActionInput evalAction = new OnlineEvalActionInput();evalAction.type = OnlineEvalActionInputType.ONLINE_EVAL;evalAction.config = new OnlineEvalActionConfigInput();evalAction.config.concurrency = 2;evalAction.config.delaySeconds = 30;OnlineEvalMetricInput metric = new OnlineEvalMetricInput();metric.evaluatorListId = "ee_latency_v1";metric.routeToHumanReview = false;evalAction.metrics = Collections.singletonList(metric);DatasetActionInput datasetAction = new DatasetActionInput();datasetAction.type = DatasetActionInputType.DATASET;datasetAction.config = new DatasetActionConfigInput();datasetAction.config.datasetId = "ds_prod_captures";datasetAction.config.mappingType = DatasetMappingType.EXPECTED_OUTPUT;datasetAction.config.extractionMode = DatasetExtractionMode.SINGLE_IO;createBody.actions = Arrays.asList(evalAction, datasetAction);AutomationRuleConfig rule = automationRules.create(createBody);// 4. Grab the rule id for later stepsString ruleId = rule.id;// 5. List rulesListAutomationRulesParams params = new ListAutomationRulesParams();params.page = 1;params.limit = 50;ListAutomationRulesResponse page = automationRules.list(params);List<AutomationRuleConfig> rules = page.data;// 6. Fetch a rule by idAutomationRuleConfig fetched = automationRules.get(ruleId);// 7. Update the rule// Partial update — only non-null fields are sent on the wire.UpdateAutomationRuleBody updateBody = new UpdateAutomationRuleBody();updateBody.samplingRate = 75;updateBody.description = "updated description";AutomationRuleConfig updated = automationRules.update(ruleId, updateBody);// 8. Pause and resume// Pause the rule while iterating on evaluator config.UpdateAutomationRuleStatusBody pauseBody = new UpdateAutomationRuleStatusBody();pauseBody.status = AutomationRuleStatus.PAUSED;AutomationRuleConfig paused = automationRules.updateStatus(ruleId, pauseBody);// ...and resume when ready.UpdateAutomationRuleStatusBody resumeBody = new UpdateAutomationRuleStatusBody();resumeBody.status = AutomationRuleStatus.ACTIVE;AutomationRuleConfig resumed = automationRules.updateStatus(ruleId, resumeBody);// 9. Add an action// Attach another DATASET action to an existing rule — captures the// LLM output into a second dataset for regression comparison.DatasetActionInput extraDataset = new DatasetActionInput();extraDataset.type = DatasetActionInputType.DATASET;extraDataset.config = new DatasetActionConfigInput();extraDataset.config.datasetId = "ds_regression_capture";extraDataset.config.mappingType = DatasetMappingType.LLM_OUTPUT;extraDataset.config.extractionMode = DatasetExtractionMode.SINGLE_IO;AddAutomationRuleActionBody addActionBody = new AddAutomationRuleActionBody();addActionBody.action = extraDataset;AutomationRuleConfig addedAction = automationRules.addAction(ruleId, addActionBody);// 10. Grab the action id for later stepsString actionId = addedAction.actions.get(0).id;// 11. Update an action// Retune the ONLINE_EVAL action's concurrency + delay in place.OnlineEvalActionUpdateInput retuned = new OnlineEvalActionUpdateInput();retuned.type = OnlineEvalActionInputType.ONLINE_EVAL;retuned.config = new OnlineEvalActionConfigInput();retuned.config.concurrency = 8;retuned.config.delaySeconds = 120;UpdateAutomationRuleActionBody updateActionBody = new UpdateAutomationRuleActionBody();updateActionBody.action = retuned;AutomationRuleConfig updatedAction = automationRules.updateAction(ruleId, actionId, updateActionBody);// 12. Delete an action// Remove an action from a rule (the rule must retain at least one// action — deleting the last remaining action returns 409).SuccessAck deletedActionResult = automationRules.deleteAction(ruleId, actionId);// 13. Delete the ruleDeleteAutomationRuleResponse result = automationRules.delete(ruleId);
Use this to update rule-level fields like Name, Description, SamplingRate, or TargetingFilters. You can't change SourceType. To add or remove actions, use the action-level methods further down; to toggle per-metric routing, use UpdateAction() since metrics belong to the action, not the rule.
Updates the rule. Only the fields you send are changed —
anything you leave out stays the same. A few things to know:
- You can't change `sourceType`.
- To add or remove actions, use the `/actions` endpoints below.
To toggle per-metric human-review routing, use
`PATCH /actions/{actionId}` (metrics belong to an action,
not to the rule).
- Pass `description: null` to clear the description.
- `targetingFilters`, if you send it, replaces the whole filter
list — send the full set you want, not a delta.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
name
string
false
New rule name.
description
string | null
false
New description. Pass `null` to clear it.
samplingRate
number
false
New sampling rate (1–100).
status
AutomationRuleStatus
false
New status — `ACTIVE` or `PAUSED`.
targetingFilters
TargetingFilter[]
false
New targeting filters. Pass an empty array to clear.
targetingFilters.fieldName
string
true
The trace attribute to compare (for example `name`, `metadata.userId`, `input`, `output`).
UpdateStatus() pauses or resumes a rule. It's a convenience shortcut over Update() — the effect is the same as calling Update() with Status: "PAUSED" or Status: "ACTIVE", but it reads more clearly at the call site.
// Pause the rule while iterating on evaluator config.paused, err := client.UpdateStatus(ctx, ruleID, &UpdateAutomationRuleStatusBody{ Status: AutomationRuleStatusPaused,})// ...and resume the rule when evaluator config is finalized.resumed, err := client.UpdateStatus(ctx, ruleID, &UpdateAutomationRuleStatusBody{ Status: AutomationRuleStatusActive,})
A convenience shortcut for pausing or resuming a rule. Same as
`PATCH /automation-rules/{ruleId}` with just `{ status }`, but
reads more clearly at the call site.
Add a second action to an existing rule — for example, attach a DATASET action to a rule that already runs ONLINE_EVAL. A rule can hold at most one action of each type, so adding a second ONLINE_EVAL (or DATASET) to a rule that already has one is rejected.
Returns the full updated rule, so you don't need a follow-up Get().
Adds one action to an existing rule. Pass the action as a nested
`action` object — the same shape as an entry in `actions[]` when
you create a rule. A rule can hold at most one action of each
type, so adding a second action of the same type returns `409`.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
action
AutomationRuleActionInput
true
The action to add. A rule can hold up to two actions.
Param
Type
Required
Description
rule_id
str
true
The automation rule's id.
action
AutomationRuleActionInput
true
The action to add. A rule can hold up to two actions.
Param
Type
Required
Description
ruleId
String
true
The automation rule's id.
action
AutomationRuleActionInput
true
The action to add. A rule can hold up to two actions.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
Action
AutomationRuleActionInput
true
The action to add. A rule can hold up to two actions.
Update an action's Config in place. For ONLINE_EVAL actions, you can optionally pass a Metrics slice to flip RouteToHumanReview on individual evaluators (up to 50 per call).
Updates the action's `config` in place. For `ONLINE_EVAL`
actions you can also pass a `metrics` list to flip
`routeToHumanReview` per metric — this endpoint is the only way
to change those toggles.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
actionId
string
true
The action's id (returned in `actions[].id` on the rule).
action
AutomationRuleActionUpdateInput
true
The fields to update on the action. Anything you leave out stays the same.
Param
Type
Required
Description
rule_id
str
true
The automation rule's id.
action_id
str
true
The action's id (returned in `actions[].id` on the rule).
action
AutomationRuleActionUpdateInput
true
The fields to update on the action. Anything you leave out stays the same.
Param
Type
Required
Description
ruleId
String
true
The automation rule's id.
actionId
String
true
The action's id (returned in `actions[].id` on the rule).
action
AutomationRuleActionUpdateInput
true
The fields to update on the action. Anything you leave out stays the same.
Param
Type
Required
Description
ruleId
string
true
The automation rule's id.
actionId
string
true
The action's id (returned in `actions[].id` on the rule).
Action
AutomationRuleActionUpdateInput
true
The fields to update on the action. Anything you leave out stays the same.
Matched traces are processed on the same schedule as the rest of the ingestion pipeline. Here's where each action's output lands:
ONLINE_EVAL actions — scores show up on the trace or observation in the dashboard. See the Online Evaluations UI guide.
DATASET actions — matched items are added to the target dataset. See the Datasets UI guide.
Rule management — every rule you create through the SDK also shows up under Settings → Automation Rules. See the Automation Rules UI guide for filter syntax and the human-review routing flow.
// Build a rule with two actions: an ONLINE_EVAL that runs an// evaluator list on a sampled slice of production traces, and a// DATASET action that captures the trace input/output as// expected-output pairs into a dataset for later experimentation.CreateAutomationRuleBody createBody = new CreateAutomationRuleBody();createBody.name = name;createBody.status = AutomationRuleStatus.ACTIVE;createBody.samplingRate = 50;createBody.sourceType = EventSchedulerType.TRACE;OnlineEvalActionInput evalAction = new OnlineEvalActionInput();evalAction.type = OnlineEvalActionInputType.ONLINE_EVAL;evalAction.config = new OnlineEvalActionConfigInput();evalAction.config.concurrency = 2;evalAction.config.delaySeconds = 30;OnlineEvalMetricInput metric = new OnlineEvalMetricInput();metric.evaluatorListId = "ee_latency_v1";metric.routeToHumanReview = false;evalAction.metrics = Collections.singletonList(metric);DatasetActionInput datasetAction = new DatasetActionInput();datasetAction.type = DatasetActionInputType.DATASET;datasetAction.config = new DatasetActionConfigInput();datasetAction.config.datasetId = "ds_prod_captures";datasetAction.config.mappingType = DatasetMappingType.EXPECTED_OUTPUT;datasetAction.config.extractionMode = DatasetExtractionMode.SINGLE_IO;createBody.actions = Arrays.asList(evalAction, datasetAction);AutomationRuleConfig rule = automationRules.create(createBody);
// Partial update — only non-null fields are sent on the wire.UpdateAutomationRuleBody updateBody = new UpdateAutomationRuleBody();updateBody.samplingRate = 75;updateBody.description = "updated description";AutomationRuleConfig updated = automationRules.update(ruleId, updateBody);
// Pause the rule while iterating on evaluator config.UpdateAutomationRuleStatusBody pauseBody = new UpdateAutomationRuleStatusBody();pauseBody.status = AutomationRuleStatus.PAUSED;AutomationRuleConfig paused = automationRules.updateStatus(ruleId, pauseBody);// ...and resume when ready.UpdateAutomationRuleStatusBody resumeBody = new UpdateAutomationRuleStatusBody();resumeBody.status = AutomationRuleStatus.ACTIVE;AutomationRuleConfig resumed = automationRules.updateStatus(ruleId, resumeBody);
DeleteAutomationRuleResponse result = automationRules.delete(ruleId);
// Attach another DATASET action to an existing rule — captures the// LLM output into a second dataset for regression comparison.DatasetActionInput extraDataset = new DatasetActionInput();extraDataset.type = DatasetActionInputType.DATASET;extraDataset.config = new DatasetActionConfigInput();extraDataset.config.datasetId = "ds_regression_capture";extraDataset.config.mappingType = DatasetMappingType.LLM_OUTPUT;extraDataset.config.extractionMode = DatasetExtractionMode.SINGLE_IO;AddAutomationRuleActionBody addActionBody = new AddAutomationRuleActionBody();addActionBody.action = extraDataset;AutomationRuleConfig addedAction = automationRules.addAction(ruleId, addActionBody);
// Retune the ONLINE_EVAL action's concurrency + delay in place.OnlineEvalActionUpdateInput retuned = new OnlineEvalActionUpdateInput();retuned.type = OnlineEvalActionInputType.ONLINE_EVAL;retuned.config = new OnlineEvalActionConfigInput();retuned.config.concurrency = 8;retuned.config.delaySeconds = 120;UpdateAutomationRuleActionBody updateActionBody = new UpdateAutomationRuleActionBody();updateActionBody.action = retuned;AutomationRuleConfig updatedAction = automationRules.updateAction(ruleId, actionId, updateActionBody);
// Remove an action from a rule (the rule must retain at least one// action — deleting the last remaining action returns 409).SuccessAck deletedActionResult = automationRules.deleteAction(ruleId, actionId);
// 1. Set up the SDKimport com.browserstack.aisdk.AISDK;import com.browserstack.aisdk.clients.automation_rule.AddAutomationRuleActionBody;import com.browserstack.aisdk.clients.automation_rule.AutomationRule;import com.browserstack.aisdk.clients.automation_rule.AutomationRuleConfig;import com.browserstack.aisdk.clients.automation_rule.AutomationRuleStatus;import com.browserstack.aisdk.clients.automation_rule.CreateAutomationRuleBody;import com.browserstack.aisdk.clients.automation_rule.DatasetActionConfigInput;import com.browserstack.aisdk.clients.automation_rule.DatasetActionInput;import com.browserstack.aisdk.clients.automation_rule.DatasetActionInputType;import com.browserstack.aisdk.clients.automation_rule.DatasetExtractionMode;import com.browserstack.aisdk.clients.automation_rule.DatasetMappingType;import com.browserstack.aisdk.clients.automation_rule.DeleteAutomationRuleResponse;import com.browserstack.aisdk.clients.automation_rule.EventSchedulerType;import com.browserstack.aisdk.clients.automation_rule.ListAutomationRulesParams;import com.browserstack.aisdk.clients.automation_rule.ListAutomationRulesResponse;import com.browserstack.aisdk.clients.automation_rule.OnlineEvalActionConfigInput;import com.browserstack.aisdk.clients.automation_rule.OnlineEvalActionInput;import com.browserstack.aisdk.clients.automation_rule.OnlineEvalActionInputType;import com.browserstack.aisdk.clients.automation_rule.OnlineEvalActionUpdateInput;import com.browserstack.aisdk.clients.automation_rule.OnlineEvalMetricInput;import com.browserstack.aisdk.clients.automation_rule.SuccessAck;import com.browserstack.aisdk.clients.automation_rule.UpdateAutomationRuleActionBody;import com.browserstack.aisdk.clients.automation_rule.UpdateAutomationRuleBody;import com.browserstack.aisdk.clients.automation_rule.UpdateAutomationRuleStatusBody;import java.util.Arrays;import java.util.Collections;import java.util.List;AISDK sdk = AISDK.builder() .publicKey(System.getenv("AISDK_PUBLIC_KEY")) .secretKey(System.getenv("AISDK_SECRET_KEY")) .build();AutomationRule automationRules = sdk.automationRule();// 2. Per-run variablesString name = "Score production traces on latency evaluator";// 3. Create a rule// Build a rule with two actions: an ONLINE_EVAL that runs an// evaluator list on a sampled slice of production traces, and a// DATASET action that captures the trace input/output as// expected-output pairs into a dataset for later experimentation.CreateAutomationRuleBody createBody = new CreateAutomationRuleBody();createBody.name = name;createBody.status = AutomationRuleStatus.ACTIVE;createBody.samplingRate = 50;createBody.sourceType = EventSchedulerType.TRACE;OnlineEvalActionInput evalAction = new OnlineEvalActionInput();evalAction.type = OnlineEvalActionInputType.ONLINE_EVAL;evalAction.config = new OnlineEvalActionConfigInput();evalAction.config.concurrency = 2;evalAction.config.delaySeconds = 30;OnlineEvalMetricInput metric = new OnlineEvalMetricInput();metric.evaluatorListId = "ee_latency_v1";metric.routeToHumanReview = false;evalAction.metrics = Collections.singletonList(metric);DatasetActionInput datasetAction = new DatasetActionInput();datasetAction.type = DatasetActionInputType.DATASET;datasetAction.config = new DatasetActionConfigInput();datasetAction.config.datasetId = "ds_prod_captures";datasetAction.config.mappingType = DatasetMappingType.EXPECTED_OUTPUT;datasetAction.config.extractionMode = DatasetExtractionMode.SINGLE_IO;createBody.actions = Arrays.asList(evalAction, datasetAction);AutomationRuleConfig rule = automationRules.create(createBody);// 4. Grab the rule id for later stepsString ruleId = rule.id;// 5. List rulesListAutomationRulesParams params = new ListAutomationRulesParams();params.page = 1;params.limit = 50;ListAutomationRulesResponse page = automationRules.list(params);List<AutomationRuleConfig> rules = page.data;// 6. Fetch a rule by idAutomationRuleConfig fetched = automationRules.get(ruleId);// 7. Update the rule// Partial update — only non-null fields are sent on the wire.UpdateAutomationRuleBody updateBody = new UpdateAutomationRuleBody();updateBody.samplingRate = 75;updateBody.description = "updated description";AutomationRuleConfig updated = automationRules.update(ruleId, updateBody);// 8. Pause and resume// Pause the rule while iterating on evaluator config.UpdateAutomationRuleStatusBody pauseBody = new UpdateAutomationRuleStatusBody();pauseBody.status = AutomationRuleStatus.PAUSED;AutomationRuleConfig paused = automationRules.updateStatus(ruleId, pauseBody);// ...and resume when ready.UpdateAutomationRuleStatusBody resumeBody = new UpdateAutomationRuleStatusBody();resumeBody.status = AutomationRuleStatus.ACTIVE;AutomationRuleConfig resumed = automationRules.updateStatus(ruleId, resumeBody);// 9. Add an action// Attach another DATASET action to an existing rule — captures the// LLM output into a second dataset for regression comparison.DatasetActionInput extraDataset = new DatasetActionInput();extraDataset.type = DatasetActionInputType.DATASET;extraDataset.config = new DatasetActionConfigInput();extraDataset.config.datasetId = "ds_regression_capture";extraDataset.config.mappingType = DatasetMappingType.LLM_OUTPUT;extraDataset.config.extractionMode = DatasetExtractionMode.SINGLE_IO;AddAutomationRuleActionBody addActionBody = new AddAutomationRuleActionBody();addActionBody.action = extraDataset;AutomationRuleConfig addedAction = automationRules.addAction(ruleId, addActionBody);// 10. Grab the action id for later stepsString actionId = addedAction.actions.get(0).id;// 11. Update an action// Retune the ONLINE_EVAL action's concurrency + delay in place.OnlineEvalActionUpdateInput retuned = new OnlineEvalActionUpdateInput();retuned.type = OnlineEvalActionInputType.ONLINE_EVAL;retuned.config = new OnlineEvalActionConfigInput();retuned.config.concurrency = 8;retuned.config.delaySeconds = 120;UpdateAutomationRuleActionBody updateActionBody = new UpdateAutomationRuleActionBody();updateActionBody.action = retuned;AutomationRuleConfig updatedAction = automationRules.updateAction(ruleId, actionId, updateActionBody);// 12. Delete an action// Remove an action from a rule (the rule must retain at least one// action — deleting the last remaining action returns 409).SuccessAck deletedActionResult = automationRules.deleteAction(ruleId, actionId);// 13. Delete the ruleDeleteAutomationRuleResponse result = automationRules.delete(ruleId);