Create, manage, and query online-evaluation rules from the SDK across TypeScript, Python, Java, and Go.

Online Evaluations

Online evaluations score your live traces (or sessions) with evaluator lists as they come in. You define a rule — what to watch, how often to sample, which evaluators to run.

Setup

import { AISDK } from '@browserstack/ai-sdk';

const publicKey = process.env.AISDK_PUBLIC_KEY!;
const secretKey = process.env.AISDK_SECRET_KEY!;

const client = new AISDK({
  publicKey,
  secretKey,
});

import { AISDK } from '@browserstack/ai-sdk';const publicKey = process.env.AISDK_PUBLIC_KEY!;const secretKey = process.env.AISDK_SECRET_KEY!;const client = new AISDK({  publicKey,  secretKey,});

Create a rule

const rule = await client.onlineEval.create({
  name,
  samplingRate: 10,
  delay: 60,
  status: true,
  evaluatorListIds: [evaluatorListId],
});

const rule = await client.onlineEval.create({  name,  samplingRate: 10,  delay: 60,  status: true,  evaluatorListIds: [evaluatorListId],});

Creates a new online-evaluation rule in your project. You can have up to 5 rules per project — going over that returns a 400 error.

Param	Type	Required	Description
`name`	`string`	`true`	Human-readable name for the rule, 1-255 characters.
`description`	`string \| null`	`false`	Optional free-text description shown alongside the rule in the UI.
`sourceType`	`EventSchedulerType`	`false`	Defaults to `TRACE` if you don't set it. You can't change this after the rule is created. Session rules unlock `triggerTraceCount` and support a smaller set of filter fields (see `OnlineEvalFilterField`).
`datasetId`	`string`	`false`	The dataset to add matches to. Setting this turns on the dataset-addition action. If you use `LLM_OUTPUT` mapping, you also need to set `datasetTag`.
`datasetMappingType`	`DatasetMappingType`	`false`	How the trace becomes a dataset item.
`datasetExtractionMode`	`DatasetExtractionMode`	`false`	Whether to use the whole trace or a specific observation as the item.
`outputObservationName`	`string \| null`	`false`	If `extractionMode` targets a specific observation, this is its span name.
`datasetTag`	`string \| null`	`false`	Required when `datasetMappingType` is `LLM_OUTPUT`.
`samplingRate`	`number`	`false`	Percent of matching traces the rule fires on (1–100). Default: `100`
`delay`	`number`	`false`	Seconds to wait before firing, so late-arriving spans can complete. Default: `30`
`triggerTraceCount`	`number`	`false`	Session rules only. Set to `0` to disable (the rule fires on the idle `delay` instead). Passing a non-zero value on a trace rule returns an error. Default: `0`
`status`	`boolean`	`false`	Set to `false` to create the rule paused instead of active. Default: `true`
`filters`	`OnlineEvalFilter[]`	`false`	Conditions that decide which traces the rule applies to. Leave empty to match every trace or session of this `sourceType`. Default: `[]`
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluatorListIds`	`string[]`	`false`	The evaluator lists to run on matched traces. Each list bundles one or more evaluators — this is what the dashboard calls a "Metric". You can create and list them via the `/api/public/evaluator-lists` endpoint. Passing a non-empty array turns on the online-evaluation action. Default: `[]`

Param	Type	Required	Description
`name`	`str`	`true`	Human-readable name for the rule, 1-255 characters.
`description`	`Optional[str]`	`false`	Optional free-text description shown alongside the rule in the UI.
`source_type`	`Optional[EventSchedulerType]`	`false`	Defaults to `TRACE` if you don't set it. You can't change this after the rule is created. Session rules unlock `triggerTraceCount` and support a smaller set of filter fields (see `OnlineEvalFilterField`).
`dataset_id`	`Optional[str]`	`false`	The dataset to add matches to. Setting this turns on the dataset-addition action. If you use `LLM_OUTPUT` mapping, you also need to set `datasetTag`.
`dataset_mapping_type`	`Optional[DatasetMappingType]`	`false`	How the trace becomes a dataset item.
`dataset_extraction_mode`	`Optional[DatasetExtractionMode]`	`false`	Whether to use the whole trace or a specific observation as the item.
`output_observation_name`	`Optional[str]`	`false`	If `extractionMode` targets a specific observation, this is its span name.
`dataset_tag`	`Optional[str]`	`false`	Required when `datasetMappingType` is `LLM_OUTPUT`.
`sampling_rate`	`Optional[int]`	`false`	Percent of matching traces the rule fires on (1–100). Default: `100`
`delay`	`Optional[int]`	`false`	Seconds to wait before firing, so late-arriving spans can complete. Default: `30`
`trigger_trace_count`	`Optional[int]`	`false`	Session rules only. Set to `0` to disable (the rule fires on the idle `delay` instead). Passing a non-zero value on a trace rule returns an error. Default: `0`
`status`	`Optional[bool]`	`false`	Set to `false` to create the rule paused instead of active. Default: `true`
`filters`	`Optional[list[OnlineEvalFilterInput]]`	`false`	Conditions that decide which traces the rule applies to. Leave empty to match every trace or session of this `sourceType`. Default: `[]`
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluator_list_ids`	`Optional[list[str]]`	`false`	The evaluator lists to run on matched traces. Each list bundles one or more evaluators — this is what the dashboard calls a "Metric". You can create and list them via the `/api/public/evaluator-lists` endpoint. Passing a non-empty array turns on the online-evaluation action. Default: `[]`

Param	Type	Required	Description
`name`	`String`	`true`	Human-readable name for the rule, 1-255 characters.
`description`	`String`	`false`	Optional free-text description shown alongside the rule in the UI.
`sourceType`	`EventSchedulerType`	`false`	Defaults to `TRACE` if you don't set it. You can't change this after the rule is created. Session rules unlock `triggerTraceCount` and support a smaller set of filter fields (see `OnlineEvalFilterField`).
`datasetId`	`String`	`false`	The dataset to add matches to. Setting this turns on the dataset-addition action. If you use `LLM_OUTPUT` mapping, you also need to set `datasetTag`.
`datasetMappingType`	`DatasetMappingType`	`false`	How the trace becomes a dataset item.
`datasetExtractionMode`	`DatasetExtractionMode`	`false`	Whether to use the whole trace or a specific observation as the item.
`outputObservationName`	`String`	`false`	If `extractionMode` targets a specific observation, this is its span name.
`datasetTag`	`String`	`false`	Required when `datasetMappingType` is `LLM_OUTPUT`.
`samplingRate`	`Integer`	`false`	Percent of matching traces the rule fires on (1–100). Default: `100`
`delay`	`Integer`	`false`	Seconds to wait before firing, so late-arriving spans can complete. Default: `30`
`triggerTraceCount`	`Integer`	`false`	Session rules only. Set to `0` to disable (the rule fires on the idle `delay` instead). Passing a non-zero value on a trace rule returns an error. Default: `0`
`status`	`Boolean`	`false`	Set to `false` to create the rule paused instead of active. Default: `true`
`filters`	`List<OnlineEvalFilter>`	`false`	Conditions that decide which traces the rule applies to. Leave empty to match every trace or session of this `sourceType`. Default: `[]`
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluatorListIds`	`List<String>`	`false`	The evaluator lists to run on matched traces. Each list bundles one or more evaluators — this is what the dashboard calls a "Metric". You can create and list them via the `/api/public/evaluator-lists` endpoint. Passing a non-empty array turns on the online-evaluation action. Default: `[]`

Param	Type	Required	Description
`Name`	`string`	`true`	Human-readable name for the rule, 1-255 characters.
`Description`	`*string`	`false`	Optional free-text description shown alongside the rule in the UI.
`SourceType`	`*EventSchedulerType`	`false`	Defaults to `TRACE` if you don't set it. You can't change this after the rule is created. Session rules unlock `triggerTraceCount` and support a smaller set of filter fields (see `OnlineEvalFilterField`).
`DatasetId`	`*string`	`false`	The dataset to add matches to. Setting this turns on the dataset-addition action. If you use `LLM_OUTPUT` mapping, you also need to set `datasetTag`.
`DatasetMappingType`	`*DatasetMappingType`	`false`	How the trace becomes a dataset item.
`DatasetExtractionMode`	`*DatasetExtractionMode`	`false`	Whether to use the whole trace or a specific observation as the item.
`OutputObservationName`	`*string`	`false`	If `extractionMode` targets a specific observation, this is its span name.
`DatasetTag`	`*string`	`false`	Required when `datasetMappingType` is `LLM_OUTPUT`.
`SamplingRate`	`*int`	`false`	Percent of matching traces the rule fires on (1–100). Default: `100`
`Delay`	`*int`	`false`	Seconds to wait before firing, so late-arriving spans can complete. Default: `30`
`TriggerTraceCount`	`*int`	`false`	Session rules only. Set to `0` to disable (the rule fires on the idle `delay` instead). Passing a non-zero value on a trace rule returns an error. Default: `0`
`Status`	`*bool`	`false`	Set to `false` to create the rule paused instead of active. Default: `true`
`Filters`	`*[]OnlineEvalFilter`	`false`	Conditions that decide which traces the rule applies to. Leave empty to match every trace or session of this `sourceType`. Default: `[]`
`Filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`Filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`Filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`Filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`EvaluatorListIds`	`*[]string`	`false`	The evaluator lists to run on matched traces. Each list bundles one or more evaluators — this is what the dashboard calls a "Metric". You can create and list them via the `/api/public/evaluator-lists` endpoint. Passing a non-empty array turns on the online-evaluation action. Default: `[]`

List rules

const res = await client.onlineEval.list();

const res = await client.onlineEval.list();

Each row also includes traceCount — the number of distinct traces the rule has evaluated so far.

Lists the rules in your project, paginated. You can page through results and search by name — richer filtering isn't available on this endpoint.

Param	Type	Required	Description
`page`	`number`	`false`	1-indexed page number (default 1) Default: `1`
`limit`	`number`	`false`	Page size, max 100 (default 50) Default: `50`
`search`	`string`	`false`	Substring match on name + description (case-insensitive)

Param	Type	Required	Description
`page`	`Optional[int]`	`false`	1-indexed page number (default 1) Default: `1`
`limit`	`Optional[int]`	`false`	Page size, max 100 (default 50) Default: `50`
`search`	`Optional[str]`	`false`	Substring match on name + description (case-insensitive)

Param	Type	Required	Description
`page`	`Integer`	`false`	1-indexed page number (default 1) Default: `1`
`limit`	`Integer`	`false`	Page size, max 100 (default 50) Default: `50`
`search`	`String`	`false`	Substring match on name + description (case-insensitive)

Param	Type	Required	Description
`Page`	`*int`	`false`	1-indexed page number (default 1) Default: `1`
`Limit`	`*int`	`false`	Page size, max 100 (default 50) Default: `50`
`Search`	`*string`	`false`	Substring match on name + description (case-insensitive)

Get a rule by id

const fetchedRule = await client.onlineEval.get(ruleId);

const fetchedRule = await client.onlineEval.get(ruleId);

Get one online evaluation rule by id

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id

Update a rule

const updated = await client.onlineEval.update(ruleId, {
  samplingRate: 80,
  status: false,
});

const updated = await client.onlineEval.update(ruleId, {  samplingRate: 80,  status: false,});

Updates the rule. Only the fields you send are changed — anything you leave out stays the same. To clear the description, pass `description: null`. You can't change `sourceType` after the rule is created. To point the rule at different evaluators or a different dataset, send the new `evaluatorListIds` or `datasetId`. To pause the rule without deleting it, set `status: false`.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`name`	`string`	`false`	New rule name.
`description`	`string \| null`	`false`	New description. Pass `null` to clear it.
`onlineEvalEnabled`	`boolean`	`false`	Explicitly pause or resume the online-evaluation action. If you leave it out, the server derives it from `evaluatorListIds` (when that's part of this update).
`datasetEnabled`	`boolean`	`false`	Explicitly pause or resume the dataset-addition action. If you leave it out, the server derives it from `datasetId` (when that's part of this update).
`datasetId`	`string`	`false`	New dataset id.
`datasetMappingType`	`DatasetMappingType`	`false`	New dataset mapping type.
`datasetExtractionMode`	`DatasetExtractionMode`	`false`	New dataset extraction mode.
`outputObservationName`	`string \| null`	`false`	New target observation name for extraction.
`datasetTag`	`string \| null`	`false`	New tag applied to items added by this rule.
`samplingRate`	`number`	`false`	New sampling rate (1–100).
`delay`	`number`	`false`	New idle delay in seconds.
`triggerTraceCount`	`number`	`false`	Session rules only.
`status`	`boolean`	`false`	Set to `false` to pause the rule without deleting it.
`filters`	`OnlineEvalFilter[]`	`false`	New filter conditions. Pass an empty array to clear.
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluatorListIds`	`string[]`	`false`	The evaluator lists to attach — see the same field on the create body. This replaces what was there before; pass the full set you want, not a delta. Sending a non-empty array turns on online eval; an empty one turns it off.

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id
`name`	`Optional[str]`	`false`	New rule name.
`description`	`Optional[str]`	`false`	New description. Pass `null` to clear it.
`online_eval_enabled`	`Optional[bool]`	`false`	Explicitly pause or resume the online-evaluation action. If you leave it out, the server derives it from `evaluatorListIds` (when that's part of this update).
`dataset_enabled`	`Optional[bool]`	`false`	Explicitly pause or resume the dataset-addition action. If you leave it out, the server derives it from `datasetId` (when that's part of this update).
`dataset_id`	`Optional[str]`	`false`	New dataset id.
`dataset_mapping_type`	`Optional[DatasetMappingType]`	`false`	New dataset mapping type.
`dataset_extraction_mode`	`Optional[DatasetExtractionMode]`	`false`	New dataset extraction mode.
`output_observation_name`	`Optional[str]`	`false`	New target observation name for extraction.
`dataset_tag`	`Optional[str]`	`false`	New tag applied to items added by this rule.
`sampling_rate`	`Optional[int]`	`false`	New sampling rate (1–100).
`delay`	`Optional[int]`	`false`	New idle delay in seconds.
`trigger_trace_count`	`Optional[int]`	`false`	Session rules only.
`status`	`Optional[bool]`	`false`	Set to `false` to pause the rule without deleting it.
`filters`	`Optional[list[OnlineEvalFilterInput]]`	`false`	New filter conditions. Pass an empty array to clear.
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluator_list_ids`	`Optional[list[str]]`	`false`	The evaluator lists to attach — see the same field on the create body. This replaces what was there before; pass the full set you want, not a delta. Sending a non-empty array turns on online eval; an empty one turns it off.

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id
`name`	`String`	`false`	New rule name.
`description`	`String`	`false`	New description. Pass `null` to clear it.
`onlineEvalEnabled`	`Boolean`	`false`	Explicitly pause or resume the online-evaluation action. If you leave it out, the server derives it from `evaluatorListIds` (when that's part of this update).
`datasetEnabled`	`Boolean`	`false`	Explicitly pause or resume the dataset-addition action. If you leave it out, the server derives it from `datasetId` (when that's part of this update).
`datasetId`	`String`	`false`	New dataset id.
`datasetMappingType`	`DatasetMappingType`	`false`	New dataset mapping type.
`datasetExtractionMode`	`DatasetExtractionMode`	`false`	New dataset extraction mode.
`outputObservationName`	`String`	`false`	New target observation name for extraction.
`datasetTag`	`String`	`false`	New tag applied to items added by this rule.
`samplingRate`	`Integer`	`false`	New sampling rate (1–100).
`delay`	`Integer`	`false`	New idle delay in seconds.
`triggerTraceCount`	`Integer`	`false`	Session rules only.
`status`	`Boolean`	`false`	Set to `false` to pause the rule without deleting it.
`filters`	`List<OnlineEvalFilter>`	`false`	New filter conditions. Pass an empty array to clear.
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluatorListIds`	`List<String>`	`false`	The evaluator lists to attach — see the same field on the create body. This replaces what was there before; pass the full set you want, not a delta. Sending a non-empty array turns on online eval; an empty one turns it off.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`Name`	`*string`	`false`	New rule name.
`Description`	`*string`	`false`	New description. Pass `null` to clear it.
`OnlineEvalEnabled`	`*bool`	`false`	Explicitly pause or resume the online-evaluation action. If you leave it out, the server derives it from `evaluatorListIds` (when that's part of this update).
`DatasetEnabled`	`*bool`	`false`	Explicitly pause or resume the dataset-addition action. If you leave it out, the server derives it from `datasetId` (when that's part of this update).
`DatasetId`	`*string`	`false`	New dataset id.
`DatasetMappingType`	`*DatasetMappingType`	`false`	New dataset mapping type.
`DatasetExtractionMode`	`*DatasetExtractionMode`	`false`	New dataset extraction mode.
`OutputObservationName`	`*string`	`false`	New target observation name for extraction.
`DatasetTag`	`*string`	`false`	New tag applied to items added by this rule.
`SamplingRate`	`*int`	`false`	New sampling rate (1–100).
`Delay`	`*int`	`false`	New idle delay in seconds.
`TriggerTraceCount`	`*int`	`false`	Session rules only.
`Status`	`*bool`	`false`	Set to `false` to pause the rule without deleting it.
`Filters`	`*[]OnlineEvalFilter`	`false`	New filter conditions. Pass an empty array to clear.
`Filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`Filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`Filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`Filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`EvaluatorListIds`	`*[]string`	`false`	The evaluator lists to attach — see the same field on the create body. This replaces what was there before; pass the full set you want, not a delta. Sending a non-empty array turns on online eval; an empty one turns it off.

Delete a rule

const deleteResult = await client.onlineEval.delete(rule.id);

const deleteResult = await client.onlineEval.delete(rule.id);

Delete an online evaluation rule

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id

Analytics

Four read-only endpoints for pulling evaluation results into your own reports or dashboards.

Evaluator scores

Returns aggregated evaluator scores over a time window — one row per evaluator, per time bucket.

const scoresResult = await client.onlineEval.scores(ruleId, {
  fromTimestamp: new Date(Date.now() - 24 * 60 * 60 * 1000),
  toTimestamp: new Date(),
  granularity: 'hour',
});

const scoresResult = await client.onlineEval.scores(ruleId, {  fromTimestamp: new Date(Date.now() - 24 * 60 * 60 * 1000),  toTimestamp: new Date(),  granularity: 'hour',});

Returns one score series per evaluator attached to the rule, over the time window you specify. If you pass `granularity`, the results are grouped into buckets of that width. Series come back as `NUMERIC` (a score timeline) or `CATEGORICAL` (per-bucket category counts). Both timestamps are optional. If you leave them out, you get the **last 24 hours** (`toTimestamp` defaults to now, `fromTimestamp` to 24 hours before that).

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`fromTimestamp`	`Date \| string`	`false`	Inclusive lower bound, ISO 8601. Defaults to `toTimestamp − 24h`.
`toTimestamp`	`Date \| string`	`false`	Inclusive upper bound, ISO 8601. Defaults to the server clock at request time.
`granularity`	`Granularity`	`false`	Bucket width. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id
`from_timestamp`	`Optional[str]`	`false`	Inclusive lower bound, ISO 8601. Defaults to `toTimestamp − 24h`.
`to_timestamp`	`Optional[str]`	`false`	Inclusive upper bound, ISO 8601. Defaults to the server clock at request time.
`granularity`	`Optional[str]`	`false`	Bucket width. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id
`fromTimestamp`	`java.time.Instant`	`false`	Inclusive lower bound, ISO 8601. Defaults to `toTimestamp − 24h`.
`toTimestamp`	`java.time.Instant`	`false`	Inclusive upper bound, ISO 8601. Defaults to the server clock at request time.
`granularity`	`Granularity`	`false`	Bucket width. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`FromTimestamp`	`*time.Time`	`false`	Inclusive lower bound, ISO 8601. Defaults to `toTimestamp − 24h`.
`ToTimestamp`	`*time.Time`	`false`	Inclusive upper bound, ISO 8601. Defaults to the server clock at request time.
`Granularity`	`*Granularity`	`false`	Bucket width. When omitted, points are returned ungrouped.

Observation metrics

const metricsResult = await client.onlineEval.metrics(ruleId, { granularity: 'hour' });

const metricsResult = await client.onlineEval.metrics(ruleId, { granularity: 'hour' });

Returns cost and token usage over time for the observations this rule has evaluated, grouped by adapter, plus provider and model distribution breakdowns. Both timestamps are optional. If you leave them out, you get the **last 24 hours** (`toTimestamp` defaults to now, `fromTimestamp` to 24 hours before that).

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`fromTimestamp`	`Date \| string`	`false`	ISO 8601. Defaults to `toTimestamp − 24h`.
`toTimestamp`	`Date \| string`	`false`	ISO 8601. Defaults to server clock.
`granularity`	`Granularity`	`false`	Bucket width for the timeseries. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id
`from_timestamp`	`Optional[str]`	`false`	ISO 8601. Defaults to `toTimestamp − 24h`.
`to_timestamp`	`Optional[str]`	`false`	ISO 8601. Defaults to server clock.
`granularity`	`Optional[str]`	`false`	Bucket width for the timeseries. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id
`fromTimestamp`	`java.time.Instant`	`false`	ISO 8601. Defaults to `toTimestamp − 24h`.
`toTimestamp`	`java.time.Instant`	`false`	ISO 8601. Defaults to server clock.
`granularity`	`Granularity`	`false`	Bucket width for the timeseries. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`FromTimestamp`	`*time.Time`	`false`	ISO 8601. Defaults to `toTimestamp − 24h`.
`ToTimestamp`	`*time.Time`	`false`	ISO 8601. Defaults to server clock.
`Granularity`	`*Granularity`	`false`	Bucket width for the timeseries. When omitted, points are returned ungrouped.

Trace list (matched by a rule)

Lists the traces a specific rule has evaluated. You can filter by score, environment, tags, and more.

const tracesPage = await client.onlineEval.listTraces(ruleId, { page: 1, limit: 50 });

const tracesPage = await client.onlineEval.listTraces(ruleId, { page: 1, limit: 50 });

Lists the traces a rule has evaluated. Supports pagination and rich filtering — see the SDK docs for the filter DSL, allowed columns, and search options.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id.
`page`	`number`	`false`	1-indexed page number. Default: `1`
`limit`	`number`	`false`	Page size, max 100. Default: `50`
`searchQuery`	`string`	`false`	Substring match against the columns configured by `searchType`.
`searchType`	`('id' \| 'content')[]`	`false`	JSON-encoded `TracingSearchType[]` — e.g. `["id"]`, `["content"]`, or both. `id` matches small identifier columns; `content` substring-matches input/output text.
`filter`	`OnlineEvalTraceFilterSet`	`false`	JSON-encoded `OnlineEvalTraceFilterSet` — `{ must?, any? }`. See the schema in components for the shape.
`orderBy`	`{ column: TraceFilterColumn`	`false`	JSON-encoded `{ column: TraceFilterColumn, order: "ASC" \| "DESC" }`.

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id.
`page`	`Optional[int]`	`false`	1-indexed page number. Default: `1`
`limit`	`Optional[int]`	`false`	Page size, max 100. Default: `50`
`search_query`	`Optional[str]`	`false`	Substring match against the columns configured by `searchType`.
`search_type`	`Optional[str]`	`false`	JSON-encoded `TracingSearchType[]` — e.g. `["id"]`, `["content"]`, or both. `id` matches small identifier columns; `content` substring-matches input/output text.
`filter`	`Optional[str]`	`false`	JSON-encoded `OnlineEvalTraceFilterSet` — `{ must?, any? }`. See the schema in components for the shape.
`order_by`	`Optional[str]`	`false`	JSON-encoded `{ column: TraceFilterColumn, order: "ASC" \| "DESC" }`.

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id.
`page`	`Integer`	`false`	1-indexed page number. Default: `1`
`limit`	`Integer`	`false`	Page size, max 100. Default: `50`
`searchQuery`	`String`	`false`	Substring match against the columns configured by `searchType`.
`searchType`	`String`	`false`	JSON-encoded `TracingSearchType[]` — e.g. `["id"]`, `["content"]`, or both. `id` matches small identifier columns; `content` substring-matches input/output text.
`filter`	`String`	`false`	JSON-encoded `OnlineEvalTraceFilterSet` — `{ must?, any? }`. See the schema in components for the shape.
`orderBy`	`String`	`false`	JSON-encoded `{ column: TraceFilterColumn, order: "ASC" \| "DESC" }`.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id.
`Page`	`*int`	`false`	1-indexed page number. Default: `1`
`Limit`	`*int`	`false`	Page size, max 100. Default: `50`
`SearchQuery`	`*string`	`false`	Substring match against the columns configured by `searchType`.
`SearchType`	`[]ListTracesSearchType`	`false`	JSON-encoded `TracingSearchType[]` — e.g. `["id"]`, `["content"]`, or both. `id` matches small identifier columns; `content` substring-matches input/output text.
`Filter`	`*OnlineEvalTraceFilterSet`	`false`	JSON-encoded `OnlineEvalTraceFilterSet` — `{ must?, any? }`. See the schema in components for the shape.
`OrderBy`	`*ListTracesOrderBy`	`false`	JSON-encoded `{ column: TraceFilterColumn, order: "ASC" \| "DESC" }`.

Trace detail (single trace + evaluator scores)

Returns a single trace with all its evaluator scores attached — the same view you'd see when expanding a row in the dashboard.

const traceDetail = await client.onlineEval.trace(traceId);

const traceDetail = await client.onlineEval.trace(traceId);

Fetches a single trace by id, along with its observations and scores. The trace doesn't need to belong to any specific rule — any trace id in your project works. Useful for chaining off the trace list.

Param	Type	Required	Description
`traceId`	`string`	`true`	The trace's id (`.id` from a `listTraces` page or any other source in the project).

Param	Type	Required	Description
`trace_id`	`str`	`true`	The trace's id (`.id` from a `listTraces` page or any other source in the project).

Param	Type	Required	Description
`traceId`	`String`	`true`	The trace's id (`.id` from a `listTraces` page or any other source in the project).

Param	Type	Required	Description
`traceId`	`string`	`true`	The trace's id (`.id` from a `listTraces` page or any other source in the project).

Complete example

Every step above, stitched together into one block you can copy and adapt. The ruleId is grabbed off the create response; the traceId is picked from the first row of the list-traces response (fall back to a real trace id if the list is empty on a fresh rule).

// 1. Set up the client
import { AISDK } from '@browserstack/ai-sdk';

const publicKey = process.env.AISDK_PUBLIC_KEY!;
const secretKey = process.env.AISDK_SECRET_KEY!;

const client = new AISDK({
  publicKey,
  secretKey,
});

// 2. Per-run variables
const name = 'Production faithfulness scoring';
const evaluatorListId = 'YOUR_EVALUATOR_LIST_ID';

// 3. Create a rule
const rule = await client.onlineEval.create({
  name,
  samplingRate: 10,
  delay: 60,
  status: true,
  evaluatorListIds: [evaluatorListId],
});

// 4. Grab the id for later steps
const ruleId = rule.id;

// 5. List rules
const res = await client.onlineEval.list();

// 6. Fetch a rule by id
const fetchedRule = await client.onlineEval.get(ruleId);

// 7. Update the rule
const updated = await client.onlineEval.update(ruleId, {
  samplingRate: 80,
  status: false,
});

// 8. Aggregated evaluator scores
const scoresResult = await client.onlineEval.scores(ruleId, {
  fromTimestamp: new Date(Date.now() - 24 * 60 * 60 * 1000),
  toTimestamp: new Date(),
  granularity: 'hour',
});

// 9. Observation metrics
const metricsResult = await client.onlineEval.metrics(ruleId, { granularity: 'hour' });

// 10. Traces matched by a rule
const tracesPage = await client.onlineEval.listTraces(ruleId, { page: 1, limit: 50 });

// 11. Pick a traceId from the list
const traceId = tracesPage.data[0]?.id ?? '<TRACE_ID>';

// 12. Trace detail with evaluator scores
const traceDetail = await client.onlineEval.trace(traceId);

// 13. Delete the rule
const deleteResult = await client.onlineEval.delete(rule.id);

// 1. Set up the clientimport { AISDK } from '@browserstack/ai-sdk';const publicKey = process.env.AISDK_PUBLIC_KEY!;const secretKey = process.env.AISDK_SECRET_KEY!;const client = new AISDK({  publicKey,  secretKey,});// 2. Per-run variablesconst name = 'Production faithfulness scoring';const evaluatorListId = 'YOUR_EVALUATOR_LIST_ID';// 3. Create a ruleconst rule = await client.onlineEval.create({  name,  samplingRate: 10,  delay: 60,  status: true,  evaluatorListIds: [evaluatorListId],});// 4. Grab the id for later stepsconst ruleId = rule.id;// 5. List rulesconst res = await client.onlineEval.list();// 6. Fetch a rule by idconst fetchedRule = await client.onlineEval.get(ruleId);// 7. Update the ruleconst updated = await client.onlineEval.update(ruleId, {  samplingRate: 80,  status: false,});// 8. Aggregated evaluator scoresconst scoresResult = await client.onlineEval.scores(ruleId, {  fromTimestamp: new Date(Date.now() - 24 * 60 * 60 * 1000),  toTimestamp: new Date(),  granularity: 'hour',});// 9. Observation metricsconst metricsResult = await client.onlineEval.metrics(ruleId, { granularity: 'hour' });// 10. Traces matched by a ruleconst tracesPage = await client.onlineEval.listTraces(ruleId, { page: 1, limit: 50 });// 11. Pick a traceId from the listconst traceId = tracesPage.data[0]?.id ?? '<TRACE_ID>';// 12. Trace detail with evaluator scoresconst traceDetail = await client.onlineEval.trace(traceId);// 13. Delete the ruleconst deleteResult = await client.onlineEval.delete(rule.id);

Setup

import os

from browserstack_ai_sdk import AISDK

client = AISDK(
    public_key=os.environ["AISDK_PUBLIC_KEY"],
    secret_key=os.environ["AISDK_SECRET_KEY"],
)

import osfrom browserstack_ai_sdk import AISDKclient = AISDK(    public_key=os.environ["AISDK_PUBLIC_KEY"],    secret_key=os.environ["AISDK_SECRET_KEY"],)

Create a rule

rule = client.online_eval.create(
    name=name,
    sampling_rate=10,
    delay=60,
    status=True,
    evaluator_list_ids=[evaluator_list_id],
)

rule = client.online_eval.create(    name=name,    sampling_rate=10,    delay=60,    status=True,    evaluator_list_ids=[evaluator_list_id],)

Creates a new online-evaluation rule in your project. You can have up to 5 rules per project — going over that returns a 400 error.

Param	Type	Required	Description
`name`	`string`	`true`	Human-readable name for the rule, 1-255 characters.
`description`	`string \| null`	`false`	Optional free-text description shown alongside the rule in the UI.
`sourceType`	`EventSchedulerType`	`false`	Defaults to `TRACE` if you don't set it. You can't change this after the rule is created. Session rules unlock `triggerTraceCount` and support a smaller set of filter fields (see `OnlineEvalFilterField`).
`datasetId`	`string`	`false`	The dataset to add matches to. Setting this turns on the dataset-addition action. If you use `LLM_OUTPUT` mapping, you also need to set `datasetTag`.
`datasetMappingType`	`DatasetMappingType`	`false`	How the trace becomes a dataset item.
`datasetExtractionMode`	`DatasetExtractionMode`	`false`	Whether to use the whole trace or a specific observation as the item.
`outputObservationName`	`string \| null`	`false`	If `extractionMode` targets a specific observation, this is its span name.
`datasetTag`	`string \| null`	`false`	Required when `datasetMappingType` is `LLM_OUTPUT`.
`samplingRate`	`number`	`false`	Percent of matching traces the rule fires on (1–100). Default: `100`
`delay`	`number`	`false`	Seconds to wait before firing, so late-arriving spans can complete. Default: `30`
`triggerTraceCount`	`number`	`false`	Session rules only. Set to `0` to disable (the rule fires on the idle `delay` instead). Passing a non-zero value on a trace rule returns an error. Default: `0`
`status`	`boolean`	`false`	Set to `false` to create the rule paused instead of active. Default: `true`
`filters`	`OnlineEvalFilter[]`	`false`	Conditions that decide which traces the rule applies to. Leave empty to match every trace or session of this `sourceType`. Default: `[]`
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluatorListIds`	`string[]`	`false`	The evaluator lists to run on matched traces. Each list bundles one or more evaluators — this is what the dashboard calls a "Metric". You can create and list them via the `/api/public/evaluator-lists` endpoint. Passing a non-empty array turns on the online-evaluation action. Default: `[]`

Param	Type	Required	Description
`name`	`str`	`true`	Human-readable name for the rule, 1-255 characters.
`description`	`Optional[str]`	`false`	Optional free-text description shown alongside the rule in the UI.
`source_type`	`Optional[EventSchedulerType]`	`false`	Defaults to `TRACE` if you don't set it. You can't change this after the rule is created. Session rules unlock `triggerTraceCount` and support a smaller set of filter fields (see `OnlineEvalFilterField`).
`dataset_id`	`Optional[str]`	`false`	The dataset to add matches to. Setting this turns on the dataset-addition action. If you use `LLM_OUTPUT` mapping, you also need to set `datasetTag`.
`dataset_mapping_type`	`Optional[DatasetMappingType]`	`false`	How the trace becomes a dataset item.
`dataset_extraction_mode`	`Optional[DatasetExtractionMode]`	`false`	Whether to use the whole trace or a specific observation as the item.
`output_observation_name`	`Optional[str]`	`false`	If `extractionMode` targets a specific observation, this is its span name.
`dataset_tag`	`Optional[str]`	`false`	Required when `datasetMappingType` is `LLM_OUTPUT`.
`sampling_rate`	`Optional[int]`	`false`	Percent of matching traces the rule fires on (1–100). Default: `100`
`delay`	`Optional[int]`	`false`	Seconds to wait before firing, so late-arriving spans can complete. Default: `30`
`trigger_trace_count`	`Optional[int]`	`false`	Session rules only. Set to `0` to disable (the rule fires on the idle `delay` instead). Passing a non-zero value on a trace rule returns an error. Default: `0`
`status`	`Optional[bool]`	`false`	Set to `false` to create the rule paused instead of active. Default: `true`
`filters`	`Optional[list[OnlineEvalFilterInput]]`	`false`	Conditions that decide which traces the rule applies to. Leave empty to match every trace or session of this `sourceType`. Default: `[]`
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluator_list_ids`	`Optional[list[str]]`	`false`	The evaluator lists to run on matched traces. Each list bundles one or more evaluators — this is what the dashboard calls a "Metric". You can create and list them via the `/api/public/evaluator-lists` endpoint. Passing a non-empty array turns on the online-evaluation action. Default: `[]`

Param	Type	Required	Description
`name`	`String`	`true`	Human-readable name for the rule, 1-255 characters.
`description`	`String`	`false`	Optional free-text description shown alongside the rule in the UI.
`sourceType`	`EventSchedulerType`	`false`	Defaults to `TRACE` if you don't set it. You can't change this after the rule is created. Session rules unlock `triggerTraceCount` and support a smaller set of filter fields (see `OnlineEvalFilterField`).
`datasetId`	`String`	`false`	The dataset to add matches to. Setting this turns on the dataset-addition action. If you use `LLM_OUTPUT` mapping, you also need to set `datasetTag`.
`datasetMappingType`	`DatasetMappingType`	`false`	How the trace becomes a dataset item.
`datasetExtractionMode`	`DatasetExtractionMode`	`false`	Whether to use the whole trace or a specific observation as the item.
`outputObservationName`	`String`	`false`	If `extractionMode` targets a specific observation, this is its span name.
`datasetTag`	`String`	`false`	Required when `datasetMappingType` is `LLM_OUTPUT`.
`samplingRate`	`Integer`	`false`	Percent of matching traces the rule fires on (1–100). Default: `100`
`delay`	`Integer`	`false`	Seconds to wait before firing, so late-arriving spans can complete. Default: `30`
`triggerTraceCount`	`Integer`	`false`	Session rules only. Set to `0` to disable (the rule fires on the idle `delay` instead). Passing a non-zero value on a trace rule returns an error. Default: `0`
`status`	`Boolean`	`false`	Set to `false` to create the rule paused instead of active. Default: `true`
`filters`	`List<OnlineEvalFilter>`	`false`	Conditions that decide which traces the rule applies to. Leave empty to match every trace or session of this `sourceType`. Default: `[]`
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluatorListIds`	`List<String>`	`false`	The evaluator lists to run on matched traces. Each list bundles one or more evaluators — this is what the dashboard calls a "Metric". You can create and list them via the `/api/public/evaluator-lists` endpoint. Passing a non-empty array turns on the online-evaluation action. Default: `[]`

Param	Type	Required	Description
`Name`	`string`	`true`	Human-readable name for the rule, 1-255 characters.
`Description`	`*string`	`false`	Optional free-text description shown alongside the rule in the UI.
`SourceType`	`*EventSchedulerType`	`false`	Defaults to `TRACE` if you don't set it. You can't change this after the rule is created. Session rules unlock `triggerTraceCount` and support a smaller set of filter fields (see `OnlineEvalFilterField`).
`DatasetId`	`*string`	`false`	The dataset to add matches to. Setting this turns on the dataset-addition action. If you use `LLM_OUTPUT` mapping, you also need to set `datasetTag`.
`DatasetMappingType`	`*DatasetMappingType`	`false`	How the trace becomes a dataset item.
`DatasetExtractionMode`	`*DatasetExtractionMode`	`false`	Whether to use the whole trace or a specific observation as the item.
`OutputObservationName`	`*string`	`false`	If `extractionMode` targets a specific observation, this is its span name.
`DatasetTag`	`*string`	`false`	Required when `datasetMappingType` is `LLM_OUTPUT`.
`SamplingRate`	`*int`	`false`	Percent of matching traces the rule fires on (1–100). Default: `100`
`Delay`	`*int`	`false`	Seconds to wait before firing, so late-arriving spans can complete. Default: `30`
`TriggerTraceCount`	`*int`	`false`	Session rules only. Set to `0` to disable (the rule fires on the idle `delay` instead). Passing a non-zero value on a trace rule returns an error. Default: `0`
`Status`	`*bool`	`false`	Set to `false` to create the rule paused instead of active. Default: `true`
`Filters`	`*[]OnlineEvalFilter`	`false`	Conditions that decide which traces the rule applies to. Leave empty to match every trace or session of this `sourceType`. Default: `[]`
`Filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`Filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`Filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`Filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`EvaluatorListIds`	`*[]string`	`false`	The evaluator lists to run on matched traces. Each list bundles one or more evaluators — this is what the dashboard calls a "Metric". You can create and list them via the `/api/public/evaluator-lists` endpoint. Passing a non-empty array turns on the online-evaluation action. Default: `[]`

List rules

res = client.online_eval.list(page=1, limit=50)

res = client.online_eval.list(page=1, limit=50)

Each row also includes trace_count — the number of distinct traces the rule has evaluated so far.

Lists the rules in your project, paginated. You can page through results and search by name — richer filtering isn't available on this endpoint.

Param	Type	Required	Description
`page`	`number`	`false`	1-indexed page number (default 1) Default: `1`
`limit`	`number`	`false`	Page size, max 100 (default 50) Default: `50`
`search`	`string`	`false`	Substring match on name + description (case-insensitive)

Param	Type	Required	Description
`page`	`Optional[int]`	`false`	1-indexed page number (default 1) Default: `1`
`limit`	`Optional[int]`	`false`	Page size, max 100 (default 50) Default: `50`
`search`	`Optional[str]`	`false`	Substring match on name + description (case-insensitive)

Param	Type	Required	Description
`page`	`Integer`	`false`	1-indexed page number (default 1) Default: `1`
`limit`	`Integer`	`false`	Page size, max 100 (default 50) Default: `50`
`search`	`String`	`false`	Substring match on name + description (case-insensitive)

Param	Type	Required	Description
`Page`	`*int`	`false`	1-indexed page number (default 1) Default: `1`
`Limit`	`*int`	`false`	Page size, max 100 (default 50) Default: `50`
`Search`	`*string`	`false`	Substring match on name + description (case-insensitive)

Get a rule by id

rule = client.online_eval.get(rule_id)

rule = client.online_eval.get(rule_id)

Get one online evaluation rule by id

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id

Update a rule

updated = client.online_eval.update(
    rule_id, sampling_rate=80, status=False
)

updated = client.online_eval.update(    rule_id, sampling_rate=80, status=False)

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`name`	`string`	`false`	New rule name.
`description`	`string \| null`	`false`	New description. Pass `null` to clear it.
`onlineEvalEnabled`	`boolean`	`false`	Explicitly pause or resume the online-evaluation action. If you leave it out, the server derives it from `evaluatorListIds` (when that's part of this update).
`datasetEnabled`	`boolean`	`false`	Explicitly pause or resume the dataset-addition action. If you leave it out, the server derives it from `datasetId` (when that's part of this update).
`datasetId`	`string`	`false`	New dataset id.
`datasetMappingType`	`DatasetMappingType`	`false`	New dataset mapping type.
`datasetExtractionMode`	`DatasetExtractionMode`	`false`	New dataset extraction mode.
`outputObservationName`	`string \| null`	`false`	New target observation name for extraction.
`datasetTag`	`string \| null`	`false`	New tag applied to items added by this rule.
`samplingRate`	`number`	`false`	New sampling rate (1–100).
`delay`	`number`	`false`	New idle delay in seconds.
`triggerTraceCount`	`number`	`false`	Session rules only.
`status`	`boolean`	`false`	Set to `false` to pause the rule without deleting it.
`filters`	`OnlineEvalFilter[]`	`false`	New filter conditions. Pass an empty array to clear.
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluatorListIds`	`string[]`	`false`	The evaluator lists to attach — see the same field on the create body. This replaces what was there before; pass the full set you want, not a delta. Sending a non-empty array turns on online eval; an empty one turns it off.

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id
`name`	`Optional[str]`	`false`	New rule name.
`description`	`Optional[str]`	`false`	New description. Pass `null` to clear it.
`online_eval_enabled`	`Optional[bool]`	`false`	Explicitly pause or resume the online-evaluation action. If you leave it out, the server derives it from `evaluatorListIds` (when that's part of this update).
`dataset_enabled`	`Optional[bool]`	`false`	Explicitly pause or resume the dataset-addition action. If you leave it out, the server derives it from `datasetId` (when that's part of this update).
`dataset_id`	`Optional[str]`	`false`	New dataset id.
`dataset_mapping_type`	`Optional[DatasetMappingType]`	`false`	New dataset mapping type.
`dataset_extraction_mode`	`Optional[DatasetExtractionMode]`	`false`	New dataset extraction mode.
`output_observation_name`	`Optional[str]`	`false`	New target observation name for extraction.
`dataset_tag`	`Optional[str]`	`false`	New tag applied to items added by this rule.
`sampling_rate`	`Optional[int]`	`false`	New sampling rate (1–100).
`delay`	`Optional[int]`	`false`	New idle delay in seconds.
`trigger_trace_count`	`Optional[int]`	`false`	Session rules only.
`status`	`Optional[bool]`	`false`	Set to `false` to pause the rule without deleting it.
`filters`	`Optional[list[OnlineEvalFilterInput]]`	`false`	New filter conditions. Pass an empty array to clear.
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluator_list_ids`	`Optional[list[str]]`	`false`	The evaluator lists to attach — see the same field on the create body. This replaces what was there before; pass the full set you want, not a delta. Sending a non-empty array turns on online eval; an empty one turns it off.

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id
`name`	`String`	`false`	New rule name.
`description`	`String`	`false`	New description. Pass `null` to clear it.
`onlineEvalEnabled`	`Boolean`	`false`	Explicitly pause or resume the online-evaluation action. If you leave it out, the server derives it from `evaluatorListIds` (when that's part of this update).
`datasetEnabled`	`Boolean`	`false`	Explicitly pause or resume the dataset-addition action. If you leave it out, the server derives it from `datasetId` (when that's part of this update).
`datasetId`	`String`	`false`	New dataset id.
`datasetMappingType`	`DatasetMappingType`	`false`	New dataset mapping type.
`datasetExtractionMode`	`DatasetExtractionMode`	`false`	New dataset extraction mode.
`outputObservationName`	`String`	`false`	New target observation name for extraction.
`datasetTag`	`String`	`false`	New tag applied to items added by this rule.
`samplingRate`	`Integer`	`false`	New sampling rate (1–100).
`delay`	`Integer`	`false`	New idle delay in seconds.
`triggerTraceCount`	`Integer`	`false`	Session rules only.
`status`	`Boolean`	`false`	Set to `false` to pause the rule without deleting it.
`filters`	`List<OnlineEvalFilter>`	`false`	New filter conditions. Pass an empty array to clear.
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluatorListIds`	`List<String>`	`false`	The evaluator lists to attach — see the same field on the create body. This replaces what was there before; pass the full set you want, not a delta. Sending a non-empty array turns on online eval; an empty one turns it off.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`Name`	`*string`	`false`	New rule name.
`Description`	`*string`	`false`	New description. Pass `null` to clear it.
`OnlineEvalEnabled`	`*bool`	`false`	Explicitly pause or resume the online-evaluation action. If you leave it out, the server derives it from `evaluatorListIds` (when that's part of this update).
`DatasetEnabled`	`*bool`	`false`	Explicitly pause or resume the dataset-addition action. If you leave it out, the server derives it from `datasetId` (when that's part of this update).
`DatasetId`	`*string`	`false`	New dataset id.
`DatasetMappingType`	`*DatasetMappingType`	`false`	New dataset mapping type.
`DatasetExtractionMode`	`*DatasetExtractionMode`	`false`	New dataset extraction mode.
`OutputObservationName`	`*string`	`false`	New target observation name for extraction.
`DatasetTag`	`*string`	`false`	New tag applied to items added by this rule.
`SamplingRate`	`*int`	`false`	New sampling rate (1–100).
`Delay`	`*int`	`false`	New idle delay in seconds.
`TriggerTraceCount`	`*int`	`false`	Session rules only.
`Status`	`*bool`	`false`	Set to `false` to pause the rule without deleting it.
`Filters`	`*[]OnlineEvalFilter`	`false`	New filter conditions. Pass an empty array to clear.
`Filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`Filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`Filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`Filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`EvaluatorListIds`	`*[]string`	`false`	The evaluator lists to attach — see the same field on the create body. This replaces what was there before; pass the full set you want, not a delta. Sending a non-empty array turns on online eval; an empty one turns it off.

Delete a rule

result = client.online_eval.delete(rule_id)

result = client.online_eval.delete(rule_id)

Delete an online evaluation rule

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id

Analytics

Four read-only endpoints for pulling evaluation results into your own reports or dashboards.

Evaluator scores

Returns aggregated evaluator scores over a time window — one row per evaluator, per time bucket.

result = client.online_eval.scores(
    rule_id,
    from_timestamp=datetime.now(timezone.utc) - timedelta(days=1),
    to_timestamp=datetime.now(timezone.utc),
    granularity="hour",
)

result = client.online_eval.scores(    rule_id,    from_timestamp=datetime.now(timezone.utc) - timedelta(days=1),    to_timestamp=datetime.now(timezone.utc),    granularity="hour",)

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`fromTimestamp`	`Date \| string`	`false`	Inclusive lower bound, ISO 8601. Defaults to `toTimestamp − 24h`.
`toTimestamp`	`Date \| string`	`false`	Inclusive upper bound, ISO 8601. Defaults to the server clock at request time.
`granularity`	`Granularity`	`false`	Bucket width. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id
`from_timestamp`	`Optional[str]`	`false`	Inclusive lower bound, ISO 8601. Defaults to `toTimestamp − 24h`.
`to_timestamp`	`Optional[str]`	`false`	Inclusive upper bound, ISO 8601. Defaults to the server clock at request time.
`granularity`	`Optional[str]`	`false`	Bucket width. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id
`fromTimestamp`	`java.time.Instant`	`false`	Inclusive lower bound, ISO 8601. Defaults to `toTimestamp − 24h`.
`toTimestamp`	`java.time.Instant`	`false`	Inclusive upper bound, ISO 8601. Defaults to the server clock at request time.
`granularity`	`Granularity`	`false`	Bucket width. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`FromTimestamp`	`*time.Time`	`false`	Inclusive lower bound, ISO 8601. Defaults to `toTimestamp − 24h`.
`ToTimestamp`	`*time.Time`	`false`	Inclusive upper bound, ISO 8601. Defaults to the server clock at request time.
`Granularity`	`*Granularity`	`false`	Bucket width. When omitted, points are returned ungrouped.

Observation metrics

result = client.online_eval.metrics(
    rule_id, granularity="hour"
)

result = client.online_eval.metrics(    rule_id, granularity="hour")

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`fromTimestamp`	`Date \| string`	`false`	ISO 8601. Defaults to `toTimestamp − 24h`.
`toTimestamp`	`Date \| string`	`false`	ISO 8601. Defaults to server clock.
`granularity`	`Granularity`	`false`	Bucket width for the timeseries. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id
`from_timestamp`	`Optional[str]`	`false`	ISO 8601. Defaults to `toTimestamp − 24h`.
`to_timestamp`	`Optional[str]`	`false`	ISO 8601. Defaults to server clock.
`granularity`	`Optional[str]`	`false`	Bucket width for the timeseries. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id
`fromTimestamp`	`java.time.Instant`	`false`	ISO 8601. Defaults to `toTimestamp − 24h`.
`toTimestamp`	`java.time.Instant`	`false`	ISO 8601. Defaults to server clock.
`granularity`	`Granularity`	`false`	Bucket width for the timeseries. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`FromTimestamp`	`*time.Time`	`false`	ISO 8601. Defaults to `toTimestamp − 24h`.
`ToTimestamp`	`*time.Time`	`false`	ISO 8601. Defaults to server clock.
`Granularity`	`*Granularity`	`false`	Bucket width for the timeseries. When omitted, points are returned ungrouped.

Trace list (matched by a rule)

Lists the traces a specific rule has evaluated. You can filter by score, environment, tags, and more.

result = client.online_eval.list_traces(
    rule_id,
    page=1, limit=10,
    filter={
        "must": [
            {"fieldName": "name", "operator": "IN", "value": ["no-match-by-design"]},
        ],
    },
)

result = client.online_eval.list_traces(    rule_id,    page=1, limit=10,    filter={        "must": [            {"fieldName": "name", "operator": "IN", "value": ["no-match-by-design"]},        ],    },)

Lists the traces a rule has evaluated. Supports pagination and rich filtering — see the SDK docs for the filter DSL, allowed columns, and search options.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id.
`page`	`number`	`false`	1-indexed page number. Default: `1`
`limit`	`number`	`false`	Page size, max 100. Default: `50`
`searchQuery`	`string`	`false`	Substring match against the columns configured by `searchType`.
`searchType`	`('id' \| 'content')[]`	`false`	JSON-encoded `TracingSearchType[]` — e.g. `["id"]`, `["content"]`, or both. `id` matches small identifier columns; `content` substring-matches input/output text.
`filter`	`OnlineEvalTraceFilterSet`	`false`	JSON-encoded `OnlineEvalTraceFilterSet` — `{ must?, any? }`. See the schema in components for the shape.
`orderBy`	`{ column: TraceFilterColumn`	`false`	JSON-encoded `{ column: TraceFilterColumn, order: "ASC" \| "DESC" }`.

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id.
`page`	`Optional[int]`	`false`	1-indexed page number. Default: `1`
`limit`	`Optional[int]`	`false`	Page size, max 100. Default: `50`
`search_query`	`Optional[str]`	`false`	Substring match against the columns configured by `searchType`.
`search_type`	`Optional[str]`	`false`	JSON-encoded `TracingSearchType[]` — e.g. `["id"]`, `["content"]`, or both. `id` matches small identifier columns; `content` substring-matches input/output text.
`filter`	`Optional[str]`	`false`	JSON-encoded `OnlineEvalTraceFilterSet` — `{ must?, any? }`. See the schema in components for the shape.
`order_by`	`Optional[str]`	`false`	JSON-encoded `{ column: TraceFilterColumn, order: "ASC" \| "DESC" }`.

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id.
`page`	`Integer`	`false`	1-indexed page number. Default: `1`
`limit`	`Integer`	`false`	Page size, max 100. Default: `50`
`searchQuery`	`String`	`false`	Substring match against the columns configured by `searchType`.
`searchType`	`String`	`false`	JSON-encoded `TracingSearchType[]` — e.g. `["id"]`, `["content"]`, or both. `id` matches small identifier columns; `content` substring-matches input/output text.
`filter`	`String`	`false`	JSON-encoded `OnlineEvalTraceFilterSet` — `{ must?, any? }`. See the schema in components for the shape.
`orderBy`	`String`	`false`	JSON-encoded `{ column: TraceFilterColumn, order: "ASC" \| "DESC" }`.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id.
`Page`	`*int`	`false`	1-indexed page number. Default: `1`
`Limit`	`*int`	`false`	Page size, max 100. Default: `50`
`SearchQuery`	`*string`	`false`	Substring match against the columns configured by `searchType`.
`SearchType`	`[]ListTracesSearchType`	`false`	JSON-encoded `TracingSearchType[]` — e.g. `["id"]`, `["content"]`, or both. `id` matches small identifier columns; `content` substring-matches input/output text.
`Filter`	`*OnlineEvalTraceFilterSet`	`false`	JSON-encoded `OnlineEvalTraceFilterSet` — `{ must?, any? }`. See the schema in components for the shape.
`OrderBy`	`*ListTracesOrderBy`	`false`	JSON-encoded `{ column: TraceFilterColumn, order: "ASC" \| "DESC" }`.

Trace detail (single trace + evaluator scores)

Returns a single trace with all its evaluator scores attached — the same view you'd see when expanding a row in the dashboard.

trace_detail = client.online_eval.trace(trace_id)

trace_detail = client.online_eval.trace(trace_id)

Param	Type	Required	Description
`traceId`	`string`	`true`	The trace's id (`.id` from a `listTraces` page or any other source in the project).

Param	Type	Required	Description
`trace_id`	`str`	`true`	The trace's id (`.id` from a `listTraces` page or any other source in the project).

Param	Type	Required	Description
`traceId`	`String`	`true`	The trace's id (`.id` from a `listTraces` page or any other source in the project).

Param	Type	Required	Description
`traceId`	`string`	`true`	The trace's id (`.id` from a `listTraces` page or any other source in the project).

Complete example

Every step above, stitched together into one block you can copy and adapt. The rule_id is grabbed off the create response; the trace_id is picked from the first row of the list-traces response (fall back to a real trace id if the list is empty on a fresh rule).

# 1. Set up the client
import os

from browserstack_ai_sdk import AISDK

client = AISDK(
    public_key=os.environ["AISDK_PUBLIC_KEY"],
    secret_key=os.environ["AISDK_SECRET_KEY"],
)

# 2. Per-run variables
name = "Production faithfulness scoring"
evaluator_list_id = "YOUR_EVALUATOR_LIST_ID"

# 3. Create a rule
rule = client.online_eval.create(
    name=name,
    sampling_rate=10,
    delay=60,
    status=True,
    evaluator_list_ids=[evaluator_list_id],
)

# 4. Grab the id for later steps
rule_id = rule.id

# 5. List rules
res = client.online_eval.list(page=1, limit=50)

# 6. Fetch a rule by id
rule = client.online_eval.get(rule_id)

# 7. Update the rule
updated = client.online_eval.update(
    rule_id, sampling_rate=80, status=False
)

# 8. Aggregated evaluator scores
result = client.online_eval.scores(
    rule_id,
    from_timestamp=datetime.now(timezone.utc) - timedelta(days=1),
    to_timestamp=datetime.now(timezone.utc),
    granularity="hour",
)

# 9. Observation metrics
result = client.online_eval.metrics(
    rule_id, granularity="hour"
)

# 10. Traces matched by a rule
result = client.online_eval.list_traces(
    rule_id,
    page=1, limit=10,
    filter={
        "must": [
            {"fieldName": "name", "operator": "IN", "value": ["no-match-by-design"]},
        ],
    },
)

# 11. Pick a trace_id from the list
trace_id = result.data[0].id if result.data else "<TRACE_ID>"

# 12. Trace detail with evaluator scores
trace_detail = client.online_eval.trace(trace_id)

# 13. Delete the rule
result = client.online_eval.delete(rule_id)

# 1. Set up the clientimport osfrom browserstack_ai_sdk import AISDKclient = AISDK(    public_key=os.environ["AISDK_PUBLIC_KEY"],    secret_key=os.environ["AISDK_SECRET_KEY"],)# 2. Per-run variablesname = "Production faithfulness scoring"evaluator_list_id = "YOUR_EVALUATOR_LIST_ID"# 3. Create a rulerule = client.online_eval.create(    name=name,    sampling_rate=10,    delay=60,    status=True,    evaluator_list_ids=[evaluator_list_id],)# 4. Grab the id for later stepsrule_id = rule.id# 5. List rulesres = client.online_eval.list(page=1, limit=50)# 6. Fetch a rule by idrule = client.online_eval.get(rule_id)# 7. Update the ruleupdated = client.online_eval.update(    rule_id, sampling_rate=80, status=False)# 8. Aggregated evaluator scoresresult = client.online_eval.scores(    rule_id,    from_timestamp=datetime.now(timezone.utc) - timedelta(days=1),    to_timestamp=datetime.now(timezone.utc),    granularity="hour",)# 9. Observation metricsresult = client.online_eval.metrics(    rule_id, granularity="hour")# 10. Traces matched by a ruleresult = client.online_eval.list_traces(    rule_id,    page=1, limit=10,    filter={        "must": [            {"fieldName": "name", "operator": "IN", "value": ["no-match-by-design"]},        ],    },)# 11. Pick a trace_id from the listtrace_id = result.data[0].id if result.data else "<TRACE_ID>"# 12. Trace detail with evaluator scorestrace_detail = client.online_eval.trace(trace_id)# 13. Delete the ruleresult = client.online_eval.delete(rule_id)

Setup

import com.browserstack.aisdk.AISDK;
import com.browserstack.aisdk.clients.online_eval.EvaluatorScoresResponse;
import com.browserstack.aisdk.clients.online_eval.Granularity;
import com.browserstack.aisdk.clients.online_eval.ListOnlineEvalTracesResponse;
import com.browserstack.aisdk.clients.online_eval.ListTracesParams;
import com.browserstack.aisdk.clients.online_eval.MetricsParams;
import com.browserstack.aisdk.clients.online_eval.ObservationMetricsResponse;
import com.browserstack.aisdk.clients.online_eval.OnlineEval;
import com.browserstack.aisdk.clients.online_eval.OnlineEvalFiltersCustom;
import com.browserstack.aisdk.clients.online_eval.ScoresParams;

import java.time.Instant;
import java.util.Arrays;
import java.util.LinkedHashMap;
import java.util.Map;

AISDK sdk = AISDK.builder()
    .publicKey(System.getenv("BROWSERSTACK_PUBLIC_KEY"))
    .secretKey(System.getenv("BROWSERSTACK_SECRET_KEY"))
    .build();
OnlineEval onlineEval = sdk.onlineEval();

import com.browserstack.aisdk.AISDK;import com.browserstack.aisdk.clients.online_eval.EvaluatorScoresResponse;import com.browserstack.aisdk.clients.online_eval.Granularity;import com.browserstack.aisdk.clients.online_eval.ListOnlineEvalTracesResponse;import com.browserstack.aisdk.clients.online_eval.ListTracesParams;import com.browserstack.aisdk.clients.online_eval.MetricsParams;import com.browserstack.aisdk.clients.online_eval.ObservationMetricsResponse;import com.browserstack.aisdk.clients.online_eval.OnlineEval;import com.browserstack.aisdk.clients.online_eval.OnlineEvalFiltersCustom;import com.browserstack.aisdk.clients.online_eval.ScoresParams;import java.time.Instant;import java.util.Arrays;import java.util.LinkedHashMap;import java.util.Map;AISDK sdk = AISDK.builder()    .publicKey(System.getenv("BROWSERSTACK_PUBLIC_KEY"))    .secretKey(System.getenv("BROWSERSTACK_SECRET_KEY"))    .build();OnlineEval onlineEval = sdk.onlineEval();

Create a rule

// Create an online-eval rule that scores 25% of production traces
// against the "Faithfulness evaluator" list with a 120s delay.
CreateOnlineEvalBody body = new CreateOnlineEvalBody();
body.name = vars.name;
body.description = "Score production traces on the faithfulness evaluator";
body.sourceType = EventSchedulerType.TRACE;
body.samplingRate = 25;
body.delay = 120;
body.triggerTraceCount = 0;
body.status = true;
body.filters = Collections.emptyList();
body.evaluatorListIds = Collections.singletonList(vars.evaluatorListId);

OnlineEvalConfig rule = onlineEval.create(body);

// Create an online-eval rule that scores 25% of production traces// against the "Faithfulness evaluator" list with a 120s delay.CreateOnlineEvalBody body = new CreateOnlineEvalBody();body.name = vars.name;body.description = "Score production traces on the faithfulness evaluator";body.sourceType = EventSchedulerType.TRACE;body.samplingRate = 25;body.delay = 120;body.triggerTraceCount = 0;body.status = true;body.filters = Collections.emptyList();body.evaluatorListIds = Collections.singletonList(vars.evaluatorListId);OnlineEvalConfig rule = onlineEval.create(body);

Creates a new online-evaluation rule in your project. You can have up to 5 rules per project — going over that returns a 400 error.

Param	Type	Required	Description
`name`	`string`	`true`	Human-readable name for the rule, 1-255 characters.
`description`	`string \| null`	`false`	Optional free-text description shown alongside the rule in the UI.
`sourceType`	`EventSchedulerType`	`false`	Defaults to `TRACE` if you don't set it. You can't change this after the rule is created. Session rules unlock `triggerTraceCount` and support a smaller set of filter fields (see `OnlineEvalFilterField`).
`datasetId`	`string`	`false`	The dataset to add matches to. Setting this turns on the dataset-addition action. If you use `LLM_OUTPUT` mapping, you also need to set `datasetTag`.
`datasetMappingType`	`DatasetMappingType`	`false`	How the trace becomes a dataset item.
`datasetExtractionMode`	`DatasetExtractionMode`	`false`	Whether to use the whole trace or a specific observation as the item.
`outputObservationName`	`string \| null`	`false`	If `extractionMode` targets a specific observation, this is its span name.
`datasetTag`	`string \| null`	`false`	Required when `datasetMappingType` is `LLM_OUTPUT`.
`samplingRate`	`number`	`false`	Percent of matching traces the rule fires on (1–100). Default: `100`
`delay`	`number`	`false`	Seconds to wait before firing, so late-arriving spans can complete. Default: `30`
`triggerTraceCount`	`number`	`false`	Session rules only. Set to `0` to disable (the rule fires on the idle `delay` instead). Passing a non-zero value on a trace rule returns an error. Default: `0`
`status`	`boolean`	`false`	Set to `false` to create the rule paused instead of active. Default: `true`
`filters`	`OnlineEvalFilter[]`	`false`	Conditions that decide which traces the rule applies to. Leave empty to match every trace or session of this `sourceType`. Default: `[]`
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluatorListIds`	`string[]`	`false`	The evaluator lists to run on matched traces. Each list bundles one or more evaluators — this is what the dashboard calls a "Metric". You can create and list them via the `/api/public/evaluator-lists` endpoint. Passing a non-empty array turns on the online-evaluation action. Default: `[]`

Param	Type	Required	Description
`name`	`str`	`true`	Human-readable name for the rule, 1-255 characters.
`description`	`Optional[str]`	`false`	Optional free-text description shown alongside the rule in the UI.
`source_type`	`Optional[EventSchedulerType]`	`false`	Defaults to `TRACE` if you don't set it. You can't change this after the rule is created. Session rules unlock `triggerTraceCount` and support a smaller set of filter fields (see `OnlineEvalFilterField`).
`dataset_id`	`Optional[str]`	`false`	The dataset to add matches to. Setting this turns on the dataset-addition action. If you use `LLM_OUTPUT` mapping, you also need to set `datasetTag`.
`dataset_mapping_type`	`Optional[DatasetMappingType]`	`false`	How the trace becomes a dataset item.
`dataset_extraction_mode`	`Optional[DatasetExtractionMode]`	`false`	Whether to use the whole trace or a specific observation as the item.
`output_observation_name`	`Optional[str]`	`false`	If `extractionMode` targets a specific observation, this is its span name.
`dataset_tag`	`Optional[str]`	`false`	Required when `datasetMappingType` is `LLM_OUTPUT`.
`sampling_rate`	`Optional[int]`	`false`	Percent of matching traces the rule fires on (1–100). Default: `100`
`delay`	`Optional[int]`	`false`	Seconds to wait before firing, so late-arriving spans can complete. Default: `30`
`trigger_trace_count`	`Optional[int]`	`false`	Session rules only. Set to `0` to disable (the rule fires on the idle `delay` instead). Passing a non-zero value on a trace rule returns an error. Default: `0`
`status`	`Optional[bool]`	`false`	Set to `false` to create the rule paused instead of active. Default: `true`
`filters`	`Optional[list[OnlineEvalFilterInput]]`	`false`	Conditions that decide which traces the rule applies to. Leave empty to match every trace or session of this `sourceType`. Default: `[]`
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluator_list_ids`	`Optional[list[str]]`	`false`	The evaluator lists to run on matched traces. Each list bundles one or more evaluators — this is what the dashboard calls a "Metric". You can create and list them via the `/api/public/evaluator-lists` endpoint. Passing a non-empty array turns on the online-evaluation action. Default: `[]`

Param	Type	Required	Description
`name`	`String`	`true`	Human-readable name for the rule, 1-255 characters.
`description`	`String`	`false`	Optional free-text description shown alongside the rule in the UI.
`sourceType`	`EventSchedulerType`	`false`	Defaults to `TRACE` if you don't set it. You can't change this after the rule is created. Session rules unlock `triggerTraceCount` and support a smaller set of filter fields (see `OnlineEvalFilterField`).
`datasetId`	`String`	`false`	The dataset to add matches to. Setting this turns on the dataset-addition action. If you use `LLM_OUTPUT` mapping, you also need to set `datasetTag`.
`datasetMappingType`	`DatasetMappingType`	`false`	How the trace becomes a dataset item.
`datasetExtractionMode`	`DatasetExtractionMode`	`false`	Whether to use the whole trace or a specific observation as the item.
`outputObservationName`	`String`	`false`	If `extractionMode` targets a specific observation, this is its span name.
`datasetTag`	`String`	`false`	Required when `datasetMappingType` is `LLM_OUTPUT`.
`samplingRate`	`Integer`	`false`	Percent of matching traces the rule fires on (1–100). Default: `100`
`delay`	`Integer`	`false`	Seconds to wait before firing, so late-arriving spans can complete. Default: `30`
`triggerTraceCount`	`Integer`	`false`	Session rules only. Set to `0` to disable (the rule fires on the idle `delay` instead). Passing a non-zero value on a trace rule returns an error. Default: `0`
`status`	`Boolean`	`false`	Set to `false` to create the rule paused instead of active. Default: `true`
`filters`	`List<OnlineEvalFilter>`	`false`	Conditions that decide which traces the rule applies to. Leave empty to match every trace or session of this `sourceType`. Default: `[]`
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluatorListIds`	`List<String>`	`false`	The evaluator lists to run on matched traces. Each list bundles one or more evaluators — this is what the dashboard calls a "Metric". You can create and list them via the `/api/public/evaluator-lists` endpoint. Passing a non-empty array turns on the online-evaluation action. Default: `[]`

Param	Type	Required	Description
`Name`	`string`	`true`	Human-readable name for the rule, 1-255 characters.
`Description`	`*string`	`false`	Optional free-text description shown alongside the rule in the UI.
`SourceType`	`*EventSchedulerType`	`false`	Defaults to `TRACE` if you don't set it. You can't change this after the rule is created. Session rules unlock `triggerTraceCount` and support a smaller set of filter fields (see `OnlineEvalFilterField`).
`DatasetId`	`*string`	`false`	The dataset to add matches to. Setting this turns on the dataset-addition action. If you use `LLM_OUTPUT` mapping, you also need to set `datasetTag`.
`DatasetMappingType`	`*DatasetMappingType`	`false`	How the trace becomes a dataset item.
`DatasetExtractionMode`	`*DatasetExtractionMode`	`false`	Whether to use the whole trace or a specific observation as the item.
`OutputObservationName`	`*string`	`false`	If `extractionMode` targets a specific observation, this is its span name.
`DatasetTag`	`*string`	`false`	Required when `datasetMappingType` is `LLM_OUTPUT`.
`SamplingRate`	`*int`	`false`	Percent of matching traces the rule fires on (1–100). Default: `100`
`Delay`	`*int`	`false`	Seconds to wait before firing, so late-arriving spans can complete. Default: `30`
`TriggerTraceCount`	`*int`	`false`	Session rules only. Set to `0` to disable (the rule fires on the idle `delay` instead). Passing a non-zero value on a trace rule returns an error. Default: `0`
`Status`	`*bool`	`false`	Set to `false` to create the rule paused instead of active. Default: `true`
`Filters`	`*[]OnlineEvalFilter`	`false`	Conditions that decide which traces the rule applies to. Leave empty to match every trace or session of this `sourceType`. Default: `[]`
`Filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`Filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`Filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`Filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`EvaluatorListIds`	`*[]string`	`false`	The evaluator lists to run on matched traces. Each list bundles one or more evaluators — this is what the dashboard calls a "Metric". You can create and list them via the `/api/public/evaluator-lists` endpoint. Passing a non-empty array turns on the online-evaluation action. Default: `[]`

List rules

ListOnlineEvalsParams params = new ListOnlineEvalsParams();
params.page = 1;
params.limit = 50;
ListOnlineEvalsResponse page = onlineEval.list(params);
java.util.List<OnlineEvalConfig> rules = page.data;

ListOnlineEvalsParams params = new ListOnlineEvalsParams();params.page = 1;params.limit = 50;ListOnlineEvalsResponse page = onlineEval.list(params);java.util.List<OnlineEvalConfig> rules = page.data;

Each row also includes traceCount — the number of distinct traces the rule has evaluated so far.

Lists the rules in your project, paginated. You can page through results and search by name — richer filtering isn't available on this endpoint.

Param	Type	Required	Description
`page`	`number`	`false`	1-indexed page number (default 1) Default: `1`
`limit`	`number`	`false`	Page size, max 100 (default 50) Default: `50`
`search`	`string`	`false`	Substring match on name + description (case-insensitive)

Param	Type	Required	Description
`page`	`Optional[int]`	`false`	1-indexed page number (default 1) Default: `1`
`limit`	`Optional[int]`	`false`	Page size, max 100 (default 50) Default: `50`
`search`	`Optional[str]`	`false`	Substring match on name + description (case-insensitive)

Param	Type	Required	Description
`page`	`Integer`	`false`	1-indexed page number (default 1) Default: `1`
`limit`	`Integer`	`false`	Page size, max 100 (default 50) Default: `50`
`search`	`String`	`false`	Substring match on name + description (case-insensitive)

Param	Type	Required	Description
`Page`	`*int`	`false`	1-indexed page number (default 1) Default: `1`
`Limit`	`*int`	`false`	Page size, max 100 (default 50) Default: `50`
`Search`	`*string`	`false`	Substring match on name + description (case-insensitive)

Get a rule by id

OnlineEvalConfig rule = onlineEval.get(ruleId);

OnlineEvalConfig rule = onlineEval.get(ruleId);

Get one online evaluation rule by id

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id

Update a rule

// Partial update — only non-null fields are sent on the wire.
// sourceType is intentionally omitted from UpdateOnlineEvalBody
// (immutable after creation).
UpdateOnlineEvalBody update = new UpdateOnlineEvalBody();
update.samplingRate = 80;
update.status = false;

OnlineEvalConfig updated = onlineEval.update(ruleId, update);

// Partial update — only non-null fields are sent on the wire.// sourceType is intentionally omitted from UpdateOnlineEvalBody// (immutable after creation).UpdateOnlineEvalBody update = new UpdateOnlineEvalBody();update.samplingRate = 80;update.status = false;OnlineEvalConfig updated = onlineEval.update(ruleId, update);

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`name`	`string`	`false`	New rule name.
`description`	`string \| null`	`false`	New description. Pass `null` to clear it.
`onlineEvalEnabled`	`boolean`	`false`	Explicitly pause or resume the online-evaluation action. If you leave it out, the server derives it from `evaluatorListIds` (when that's part of this update).
`datasetEnabled`	`boolean`	`false`	Explicitly pause or resume the dataset-addition action. If you leave it out, the server derives it from `datasetId` (when that's part of this update).
`datasetId`	`string`	`false`	New dataset id.
`datasetMappingType`	`DatasetMappingType`	`false`	New dataset mapping type.
`datasetExtractionMode`	`DatasetExtractionMode`	`false`	New dataset extraction mode.
`outputObservationName`	`string \| null`	`false`	New target observation name for extraction.
`datasetTag`	`string \| null`	`false`	New tag applied to items added by this rule.
`samplingRate`	`number`	`false`	New sampling rate (1–100).
`delay`	`number`	`false`	New idle delay in seconds.
`triggerTraceCount`	`number`	`false`	Session rules only.
`status`	`boolean`	`false`	Set to `false` to pause the rule without deleting it.
`filters`	`OnlineEvalFilter[]`	`false`	New filter conditions. Pass an empty array to clear.
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluatorListIds`	`string[]`	`false`	The evaluator lists to attach — see the same field on the create body. This replaces what was there before; pass the full set you want, not a delta. Sending a non-empty array turns on online eval; an empty one turns it off.

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id
`name`	`Optional[str]`	`false`	New rule name.
`description`	`Optional[str]`	`false`	New description. Pass `null` to clear it.
`online_eval_enabled`	`Optional[bool]`	`false`	Explicitly pause or resume the online-evaluation action. If you leave it out, the server derives it from `evaluatorListIds` (when that's part of this update).
`dataset_enabled`	`Optional[bool]`	`false`	Explicitly pause or resume the dataset-addition action. If you leave it out, the server derives it from `datasetId` (when that's part of this update).
`dataset_id`	`Optional[str]`	`false`	New dataset id.
`dataset_mapping_type`	`Optional[DatasetMappingType]`	`false`	New dataset mapping type.
`dataset_extraction_mode`	`Optional[DatasetExtractionMode]`	`false`	New dataset extraction mode.
`output_observation_name`	`Optional[str]`	`false`	New target observation name for extraction.
`dataset_tag`	`Optional[str]`	`false`	New tag applied to items added by this rule.
`sampling_rate`	`Optional[int]`	`false`	New sampling rate (1–100).
`delay`	`Optional[int]`	`false`	New idle delay in seconds.
`trigger_trace_count`	`Optional[int]`	`false`	Session rules only.
`status`	`Optional[bool]`	`false`	Set to `false` to pause the rule without deleting it.
`filters`	`Optional[list[OnlineEvalFilterInput]]`	`false`	New filter conditions. Pass an empty array to clear.
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluator_list_ids`	`Optional[list[str]]`	`false`	The evaluator lists to attach — see the same field on the create body. This replaces what was there before; pass the full set you want, not a delta. Sending a non-empty array turns on online eval; an empty one turns it off.

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id
`name`	`String`	`false`	New rule name.
`description`	`String`	`false`	New description. Pass `null` to clear it.
`onlineEvalEnabled`	`Boolean`	`false`	Explicitly pause or resume the online-evaluation action. If you leave it out, the server derives it from `evaluatorListIds` (when that's part of this update).
`datasetEnabled`	`Boolean`	`false`	Explicitly pause or resume the dataset-addition action. If you leave it out, the server derives it from `datasetId` (when that's part of this update).
`datasetId`	`String`	`false`	New dataset id.
`datasetMappingType`	`DatasetMappingType`	`false`	New dataset mapping type.
`datasetExtractionMode`	`DatasetExtractionMode`	`false`	New dataset extraction mode.
`outputObservationName`	`String`	`false`	New target observation name for extraction.
`datasetTag`	`String`	`false`	New tag applied to items added by this rule.
`samplingRate`	`Integer`	`false`	New sampling rate (1–100).
`delay`	`Integer`	`false`	New idle delay in seconds.
`triggerTraceCount`	`Integer`	`false`	Session rules only.
`status`	`Boolean`	`false`	Set to `false` to pause the rule without deleting it.
`filters`	`List<OnlineEvalFilter>`	`false`	New filter conditions. Pass an empty array to clear.
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluatorListIds`	`List<String>`	`false`	The evaluator lists to attach — see the same field on the create body. This replaces what was there before; pass the full set you want, not a delta. Sending a non-empty array turns on online eval; an empty one turns it off.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`Name`	`*string`	`false`	New rule name.
`Description`	`*string`	`false`	New description. Pass `null` to clear it.
`OnlineEvalEnabled`	`*bool`	`false`	Explicitly pause or resume the online-evaluation action. If you leave it out, the server derives it from `evaluatorListIds` (when that's part of this update).
`DatasetEnabled`	`*bool`	`false`	Explicitly pause or resume the dataset-addition action. If you leave it out, the server derives it from `datasetId` (when that's part of this update).
`DatasetId`	`*string`	`false`	New dataset id.
`DatasetMappingType`	`*DatasetMappingType`	`false`	New dataset mapping type.
`DatasetExtractionMode`	`*DatasetExtractionMode`	`false`	New dataset extraction mode.
`OutputObservationName`	`*string`	`false`	New target observation name for extraction.
`DatasetTag`	`*string`	`false`	New tag applied to items added by this rule.
`SamplingRate`	`*int`	`false`	New sampling rate (1–100).
`Delay`	`*int`	`false`	New idle delay in seconds.
`TriggerTraceCount`	`*int`	`false`	Session rules only.
`Status`	`*bool`	`false`	Set to `false` to pause the rule without deleting it.
`Filters`	`*[]OnlineEvalFilter`	`false`	New filter conditions. Pass an empty array to clear.
`Filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`Filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`Filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`Filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`EvaluatorListIds`	`*[]string`	`false`	The evaluator lists to attach — see the same field on the create body. This replaces what was there before; pass the full set you want, not a delta. Sending a non-empty array turns on online eval; an empty one turns it off.

Delete a rule

DeleteOnlineEvalResponse res = onlineEval.delete(ruleId);

DeleteOnlineEvalResponse res = onlineEval.delete(ruleId);

Delete an online evaluation rule

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id

Analytics

Four read-only endpoints for pulling evaluation results into your own reports or dashboards.

Evaluator scores

Returns aggregated evaluator scores over a time window — one row per evaluator, per time bucket.

// Fetch per-evaluator score time-series for an online-eval rule.
// Server applies a last-24h default window when timestamps are omitted.
ScoresParams params = new ScoresParams();
params.fromTimestamp = Instant.parse("2026-06-24T00:00:00Z");
params.toTimestamp = Instant.parse("2026-06-25T00:00:00Z");
params.granularity = Granularity.hour;
EvaluatorScoresResponse resp = onlineEval.scores(ruleId, params);

// Fetch per-evaluator score time-series for an online-eval rule.// Server applies a last-24h default window when timestamps are omitted.ScoresParams params = new ScoresParams();params.fromTimestamp = Instant.parse("2026-06-24T00:00:00Z");params.toTimestamp = Instant.parse("2026-06-25T00:00:00Z");params.granularity = Granularity.hour;EvaluatorScoresResponse resp = onlineEval.scores(ruleId, params);

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`fromTimestamp`	`Date \| string`	`false`	Inclusive lower bound, ISO 8601. Defaults to `toTimestamp − 24h`.
`toTimestamp`	`Date \| string`	`false`	Inclusive upper bound, ISO 8601. Defaults to the server clock at request time.
`granularity`	`Granularity`	`false`	Bucket width. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id
`from_timestamp`	`Optional[str]`	`false`	Inclusive lower bound, ISO 8601. Defaults to `toTimestamp − 24h`.
`to_timestamp`	`Optional[str]`	`false`	Inclusive upper bound, ISO 8601. Defaults to the server clock at request time.
`granularity`	`Optional[str]`	`false`	Bucket width. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id
`fromTimestamp`	`java.time.Instant`	`false`	Inclusive lower bound, ISO 8601. Defaults to `toTimestamp − 24h`.
`toTimestamp`	`java.time.Instant`	`false`	Inclusive upper bound, ISO 8601. Defaults to the server clock at request time.
`granularity`	`Granularity`	`false`	Bucket width. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`FromTimestamp`	`*time.Time`	`false`	Inclusive lower bound, ISO 8601. Defaults to `toTimestamp − 24h`.
`ToTimestamp`	`*time.Time`	`false`	Inclusive upper bound, ISO 8601. Defaults to the server clock at request time.
`Granularity`	`*Granularity`	`false`	Bucket width. When omitted, points are returned ungrouped.

Observation metrics

// Fetch cost/token/provider-distribution metrics for the rule's
// sampled traces over the selected granularity bucket.
MetricsParams params = new MetricsParams();
params.granularity = Granularity.day;
ObservationMetricsResponse resp = onlineEval.metrics(ruleId, params);

// Fetch cost/token/provider-distribution metrics for the rule's// sampled traces over the selected granularity bucket.MetricsParams params = new MetricsParams();params.granularity = Granularity.day;ObservationMetricsResponse resp = onlineEval.metrics(ruleId, params);

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`fromTimestamp`	`Date \| string`	`false`	ISO 8601. Defaults to `toTimestamp − 24h`.
`toTimestamp`	`Date \| string`	`false`	ISO 8601. Defaults to server clock.
`granularity`	`Granularity`	`false`	Bucket width for the timeseries. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id
`from_timestamp`	`Optional[str]`	`false`	ISO 8601. Defaults to `toTimestamp − 24h`.
`to_timestamp`	`Optional[str]`	`false`	ISO 8601. Defaults to server clock.
`granularity`	`Optional[str]`	`false`	Bucket width for the timeseries. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id
`fromTimestamp`	`java.time.Instant`	`false`	ISO 8601. Defaults to `toTimestamp − 24h`.
`toTimestamp`	`java.time.Instant`	`false`	ISO 8601. Defaults to server clock.
`granularity`	`Granularity`	`false`	Bucket width for the timeseries. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`FromTimestamp`	`*time.Time`	`false`	ISO 8601. Defaults to `toTimestamp − 24h`.
`ToTimestamp`	`*time.Time`	`false`	ISO 8601. Defaults to server clock.
`Granularity`	`*Granularity`	`false`	Bucket width for the timeseries. When omitted, points are returned ungrouped.

Trace list (matched by a rule)

Lists the traces a specific rule has evaluated. You can filter by score, environment, tags, and more.

// Paginate the traces this online-eval rule has sampled.
ListTracesParams params = new ListTracesParams();
params.page = 1;
params.limit = 50;
ListOnlineEvalTracesResponse resp = onlineEval.listTraces(ruleId, params);

// Paginate the traces this online-eval rule has sampled.ListTracesParams params = new ListTracesParams();params.page = 1;params.limit = 50;ListOnlineEvalTracesResponse resp = onlineEval.listTraces(ruleId, params);

Lists the traces a rule has evaluated. Supports pagination and rich filtering — see the SDK docs for the filter DSL, allowed columns, and search options.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id.
`page`	`number`	`false`	1-indexed page number. Default: `1`
`limit`	`number`	`false`	Page size, max 100. Default: `50`
`searchQuery`	`string`	`false`	Substring match against the columns configured by `searchType`.
`searchType`	`('id' \| 'content')[]`	`false`	JSON-encoded `TracingSearchType[]` — e.g. `["id"]`, `["content"]`, or both. `id` matches small identifier columns; `content` substring-matches input/output text.
`filter`	`OnlineEvalTraceFilterSet`	`false`	JSON-encoded `OnlineEvalTraceFilterSet` — `{ must?, any? }`. See the schema in components for the shape.
`orderBy`	`{ column: TraceFilterColumn`	`false`	JSON-encoded `{ column: TraceFilterColumn, order: "ASC" \| "DESC" }`.

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id.
`page`	`Optional[int]`	`false`	1-indexed page number. Default: `1`
`limit`	`Optional[int]`	`false`	Page size, max 100. Default: `50`
`search_query`	`Optional[str]`	`false`	Substring match against the columns configured by `searchType`.
`search_type`	`Optional[str]`	`false`	JSON-encoded `TracingSearchType[]` — e.g. `["id"]`, `["content"]`, or both. `id` matches small identifier columns; `content` substring-matches input/output text.
`filter`	`Optional[str]`	`false`	JSON-encoded `OnlineEvalTraceFilterSet` — `{ must?, any? }`. See the schema in components for the shape.
`order_by`	`Optional[str]`	`false`	JSON-encoded `{ column: TraceFilterColumn, order: "ASC" \| "DESC" }`.

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id.
`page`	`Integer`	`false`	1-indexed page number. Default: `1`
`limit`	`Integer`	`false`	Page size, max 100. Default: `50`
`searchQuery`	`String`	`false`	Substring match against the columns configured by `searchType`.
`searchType`	`String`	`false`	JSON-encoded `TracingSearchType[]` — e.g. `["id"]`, `["content"]`, or both. `id` matches small identifier columns; `content` substring-matches input/output text.
`filter`	`String`	`false`	JSON-encoded `OnlineEvalTraceFilterSet` — `{ must?, any? }`. See the schema in components for the shape.
`orderBy`	`String`	`false`	JSON-encoded `{ column: TraceFilterColumn, order: "ASC" \| "DESC" }`.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id.
`Page`	`*int`	`false`	1-indexed page number. Default: `1`
`Limit`	`*int`	`false`	Page size, max 100. Default: `50`
`SearchQuery`	`*string`	`false`	Substring match against the columns configured by `searchType`.
`SearchType`	`[]ListTracesSearchType`	`false`	JSON-encoded `TracingSearchType[]` — e.g. `["id"]`, `["content"]`, or both. `id` matches small identifier columns; `content` substring-matches input/output text.
`Filter`	`*OnlineEvalTraceFilterSet`	`false`	JSON-encoded `OnlineEvalTraceFilterSet` — `{ must?, any? }`. See the schema in components for the shape.
`OrderBy`	`*ListTracesOrderBy`	`false`	JSON-encoded `{ column: TraceFilterColumn, order: "ASC" \| "DESC" }`.

Trace detail (single trace + evaluator scores)

Returns a single trace with all its evaluator scores attached — the same view you'd see when expanding a row in the dashboard.

// Fetch the full trace payload for a scored trace surfaced by the
// online-eval rule (delegates to the shared /api/public/traces facade).
Object trace = onlineEval.trace(traceId);

// Fetch the full trace payload for a scored trace surfaced by the// online-eval rule (delegates to the shared /api/public/traces facade).Object trace = onlineEval.trace(traceId);

Param	Type	Required	Description
`traceId`	`string`	`true`	The trace's id (`.id` from a `listTraces` page or any other source in the project).

Param	Type	Required	Description
`trace_id`	`str`	`true`	The trace's id (`.id` from a `listTraces` page or any other source in the project).

Param	Type	Required	Description
`traceId`	`String`	`true`	The trace's id (`.id` from a `listTraces` page or any other source in the project).

Param	Type	Required	Description
`traceId`	`string`	`true`	The trace's id (`.id` from a `listTraces` page or any other source in the project).

Complete example

// 1. Set up the SDK
import com.browserstack.aisdk.AISDK;
import com.browserstack.aisdk.clients.online_eval.EvaluatorScoresResponse;
import com.browserstack.aisdk.clients.online_eval.Granularity;
import com.browserstack.aisdk.clients.online_eval.ListOnlineEvalTracesResponse;
import com.browserstack.aisdk.clients.online_eval.ListTracesParams;
import com.browserstack.aisdk.clients.online_eval.MetricsParams;
import com.browserstack.aisdk.clients.online_eval.ObservationMetricsResponse;
import com.browserstack.aisdk.clients.online_eval.OnlineEval;
import com.browserstack.aisdk.clients.online_eval.OnlineEvalFiltersCustom;
import com.browserstack.aisdk.clients.online_eval.ScoresParams;

import java.time.Instant;
import java.util.Arrays;
import java.util.LinkedHashMap;
import java.util.Map;

AISDK sdk = AISDK.builder()
    .publicKey(System.getenv("BROWSERSTACK_PUBLIC_KEY"))
    .secretKey(System.getenv("BROWSERSTACK_SECRET_KEY"))
    .build();
OnlineEval onlineEval = sdk.onlineEval();

// 2. Per-run variables
String name = "Production faithfulness scoring";
String evaluatorListId = "YOUR_EVALUATOR_LIST_ID";

// 3. Create a rule
// Create an online-eval rule that scores 25% of production traces
// against the "Faithfulness evaluator" list with a 120s delay.
CreateOnlineEvalBody body = new CreateOnlineEvalBody();
body.name = vars.name;
body.description = "Score production traces on the faithfulness evaluator";
body.sourceType = EventSchedulerType.TRACE;
body.samplingRate = 25;
body.delay = 120;
body.triggerTraceCount = 0;
body.status = true;
body.filters = Collections.emptyList();
body.evaluatorListIds = Collections.singletonList(vars.evaluatorListId);

OnlineEvalConfig rule = onlineEval.create(body);

// 4. Grab the id for later steps
String ruleId = rule.id;

// 5. List rules
ListOnlineEvalsParams params = new ListOnlineEvalsParams();
params.page = 1;
params.limit = 50;
ListOnlineEvalsResponse page = onlineEval.list(params);
java.util.List<OnlineEvalConfig> rules = page.data;

// 6. Fetch a rule by id
OnlineEvalConfig rule = onlineEval.get(ruleId);

// 7. Update the rule
// Partial update — only non-null fields are sent on the wire.
// sourceType is intentionally omitted from UpdateOnlineEvalBody
// (immutable after creation).
UpdateOnlineEvalBody update = new UpdateOnlineEvalBody();
update.samplingRate = 80;
update.status = false;

OnlineEvalConfig updated = onlineEval.update(ruleId, update);

// 8. Aggregated evaluator scores
// Fetch per-evaluator score time-series for an online-eval rule.
// Server applies a last-24h default window when timestamps are omitted.
ScoresParams params = new ScoresParams();
params.fromTimestamp = Instant.parse("2026-06-24T00:00:00Z");
params.toTimestamp = Instant.parse("2026-06-25T00:00:00Z");
params.granularity = Granularity.hour;
EvaluatorScoresResponse resp = onlineEval.scores(ruleId, params);

// 9. Observation metrics
// Fetch cost/token/provider-distribution metrics for the rule's
// sampled traces over the selected granularity bucket.
MetricsParams params = new MetricsParams();
params.granularity = Granularity.day;
ObservationMetricsResponse resp = onlineEval.metrics(ruleId, params);

// 10. Traces matched by a rule
// Paginate the traces this online-eval rule has sampled.
ListTracesParams params = new ListTracesParams();
params.page = 1;
params.limit = 50;
ListOnlineEvalTracesResponse resp = onlineEval.listTraces(ruleId, params);

// 11. Pick a traceId from the list
String traceId = resp.data.isEmpty() ? "<TRACE_ID>" : resp.data.get(0).id;

// 12. Trace detail with evaluator scores
// Fetch the full trace payload for a scored trace surfaced by the
// online-eval rule (delegates to the shared /api/public/traces facade).
Object trace = onlineEval.trace(traceId);

// 13. Delete the rule
DeleteOnlineEvalResponse res = onlineEval.delete(ruleId);

// 1. Set up the SDKimport com.browserstack.aisdk.AISDK;import com.browserstack.aisdk.clients.online_eval.EvaluatorScoresResponse;import com.browserstack.aisdk.clients.online_eval.Granularity;import com.browserstack.aisdk.clients.online_eval.ListOnlineEvalTracesResponse;import com.browserstack.aisdk.clients.online_eval.ListTracesParams;import com.browserstack.aisdk.clients.online_eval.MetricsParams;import com.browserstack.aisdk.clients.online_eval.ObservationMetricsResponse;import com.browserstack.aisdk.clients.online_eval.OnlineEval;import com.browserstack.aisdk.clients.online_eval.OnlineEvalFiltersCustom;import com.browserstack.aisdk.clients.online_eval.ScoresParams;import java.time.Instant;import java.util.Arrays;import java.util.LinkedHashMap;import java.util.Map;AISDK sdk = AISDK.builder()    .publicKey(System.getenv("BROWSERSTACK_PUBLIC_KEY"))    .secretKey(System.getenv("BROWSERSTACK_SECRET_KEY"))    .build();OnlineEval onlineEval = sdk.onlineEval();// 2. Per-run variablesString name = "Production faithfulness scoring";String evaluatorListId = "YOUR_EVALUATOR_LIST_ID";// 3. Create a rule// Create an online-eval rule that scores 25% of production traces// against the "Faithfulness evaluator" list with a 120s delay.CreateOnlineEvalBody body = new CreateOnlineEvalBody();body.name = vars.name;body.description = "Score production traces on the faithfulness evaluator";body.sourceType = EventSchedulerType.TRACE;body.samplingRate = 25;body.delay = 120;body.triggerTraceCount = 0;body.status = true;body.filters = Collections.emptyList();body.evaluatorListIds = Collections.singletonList(vars.evaluatorListId);OnlineEvalConfig rule = onlineEval.create(body);// 4. Grab the id for later stepsString ruleId = rule.id;// 5. List rulesListOnlineEvalsParams params = new ListOnlineEvalsParams();params.page = 1;params.limit = 50;ListOnlineEvalsResponse page = onlineEval.list(params);java.util.List<OnlineEvalConfig> rules = page.data;// 6. Fetch a rule by idOnlineEvalConfig rule = onlineEval.get(ruleId);// 7. Update the rule// Partial update — only non-null fields are sent on the wire.// sourceType is intentionally omitted from UpdateOnlineEvalBody// (immutable after creation).UpdateOnlineEvalBody update = new UpdateOnlineEvalBody();update.samplingRate = 80;update.status = false;OnlineEvalConfig updated = onlineEval.update(ruleId, update);// 8. Aggregated evaluator scores// Fetch per-evaluator score time-series for an online-eval rule.// Server applies a last-24h default window when timestamps are omitted.ScoresParams params = new ScoresParams();params.fromTimestamp = Instant.parse("2026-06-24T00:00:00Z");params.toTimestamp = Instant.parse("2026-06-25T00:00:00Z");params.granularity = Granularity.hour;EvaluatorScoresResponse resp = onlineEval.scores(ruleId, params);// 9. Observation metrics// Fetch cost/token/provider-distribution metrics for the rule's// sampled traces over the selected granularity bucket.MetricsParams params = new MetricsParams();params.granularity = Granularity.day;ObservationMetricsResponse resp = onlineEval.metrics(ruleId, params);// 10. Traces matched by a rule// Paginate the traces this online-eval rule has sampled.ListTracesParams params = new ListTracesParams();params.page = 1;params.limit = 50;ListOnlineEvalTracesResponse resp = onlineEval.listTraces(ruleId, params);// 11. Pick a traceId from the listString traceId = resp.data.isEmpty() ? "<TRACE_ID>" : resp.data.get(0).id;// 12. Trace detail with evaluator scores// Fetch the full trace payload for a scored trace surfaced by the// online-eval rule (delegates to the shared /api/public/traces facade).Object trace = onlineEval.trace(traceId);// 13. Delete the ruleDeleteOnlineEvalResponse res = onlineEval.delete(ruleId);

Setup

import (
	"context"
	"log"

	aisdk "github.com/browserstack/ai-sdk-go"
)

ctx := context.Background()

sdk, err := aisdk.New(aisdk.WithCredentials(
	"pk-lf-your-public-key",
	"sk-lf-your-secret-key",
))
if err != nil {
	log.Fatalf("aisdk.New: %v", err)
}

client := sdk.OnlineEval()

import (	"context"	"log"	aisdk "github.com/browserstack/ai-sdk-go")ctx := context.Background()sdk, err := aisdk.New(aisdk.WithCredentials(	"pk-lf-your-public-key",	"sk-lf-your-secret-key",))if err != nil {	log.Fatalf("aisdk.New: %v", err)}client := sdk.OnlineEval()

Create a rule

desc := "Score 25% of production traces on faithfulness evaluator"
sr, delay := 25, 120
status := true
rule, err := client.Create(context.Background(), &CreateOnlineEvalBody{
	Name:             vars.name,
	Description:      &desc,
	SamplingRate:     &sr,
	Delay:            &delay,
	Status:           &status,
	Filters:          &[]OnlineEvalFilter{},
	EvaluatorListIds: ptr([]string{evaluatorListID}),
})

desc := "Score 25% of production traces on faithfulness evaluator"sr, delay := 25, 120status := truerule, err := client.Create(context.Background(), &CreateOnlineEvalBody{	Name:             vars.name,	Description:      &desc,	SamplingRate:     &sr,	Delay:            &delay,	Status:           &status,	Filters:          &[]OnlineEvalFilter{},	EvaluatorListIds: ptr([]string{evaluatorListID}),})

Creates a new online-evaluation rule in your project. You can have up to 5 rules per project — going over that returns a 400 error.

Param	Type	Required	Description
`name`	`string`	`true`	Human-readable name for the rule, 1-255 characters.
`description`	`string \| null`	`false`	Optional free-text description shown alongside the rule in the UI.
`sourceType`	`EventSchedulerType`	`false`	Defaults to `TRACE` if you don't set it. You can't change this after the rule is created. Session rules unlock `triggerTraceCount` and support a smaller set of filter fields (see `OnlineEvalFilterField`).
`datasetId`	`string`	`false`	The dataset to add matches to. Setting this turns on the dataset-addition action. If you use `LLM_OUTPUT` mapping, you also need to set `datasetTag`.
`datasetMappingType`	`DatasetMappingType`	`false`	How the trace becomes a dataset item.
`datasetExtractionMode`	`DatasetExtractionMode`	`false`	Whether to use the whole trace or a specific observation as the item.
`outputObservationName`	`string \| null`	`false`	If `extractionMode` targets a specific observation, this is its span name.
`datasetTag`	`string \| null`	`false`	Required when `datasetMappingType` is `LLM_OUTPUT`.
`samplingRate`	`number`	`false`	Percent of matching traces the rule fires on (1–100). Default: `100`
`delay`	`number`	`false`	Seconds to wait before firing, so late-arriving spans can complete. Default: `30`
`triggerTraceCount`	`number`	`false`	Session rules only. Set to `0` to disable (the rule fires on the idle `delay` instead). Passing a non-zero value on a trace rule returns an error. Default: `0`
`status`	`boolean`	`false`	Set to `false` to create the rule paused instead of active. Default: `true`
`filters`	`OnlineEvalFilter[]`	`false`	Conditions that decide which traces the rule applies to. Leave empty to match every trace or session of this `sourceType`. Default: `[]`
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluatorListIds`	`string[]`	`false`	The evaluator lists to run on matched traces. Each list bundles one or more evaluators — this is what the dashboard calls a "Metric". You can create and list them via the `/api/public/evaluator-lists` endpoint. Passing a non-empty array turns on the online-evaluation action. Default: `[]`

Param	Type	Required	Description
`name`	`str`	`true`	Human-readable name for the rule, 1-255 characters.
`description`	`Optional[str]`	`false`	Optional free-text description shown alongside the rule in the UI.
`source_type`	`Optional[EventSchedulerType]`	`false`	Defaults to `TRACE` if you don't set it. You can't change this after the rule is created. Session rules unlock `triggerTraceCount` and support a smaller set of filter fields (see `OnlineEvalFilterField`).
`dataset_id`	`Optional[str]`	`false`	The dataset to add matches to. Setting this turns on the dataset-addition action. If you use `LLM_OUTPUT` mapping, you also need to set `datasetTag`.
`dataset_mapping_type`	`Optional[DatasetMappingType]`	`false`	How the trace becomes a dataset item.
`dataset_extraction_mode`	`Optional[DatasetExtractionMode]`	`false`	Whether to use the whole trace or a specific observation as the item.
`output_observation_name`	`Optional[str]`	`false`	If `extractionMode` targets a specific observation, this is its span name.
`dataset_tag`	`Optional[str]`	`false`	Required when `datasetMappingType` is `LLM_OUTPUT`.
`sampling_rate`	`Optional[int]`	`false`	Percent of matching traces the rule fires on (1–100). Default: `100`
`delay`	`Optional[int]`	`false`	Seconds to wait before firing, so late-arriving spans can complete. Default: `30`
`trigger_trace_count`	`Optional[int]`	`false`	Session rules only. Set to `0` to disable (the rule fires on the idle `delay` instead). Passing a non-zero value on a trace rule returns an error. Default: `0`
`status`	`Optional[bool]`	`false`	Set to `false` to create the rule paused instead of active. Default: `true`
`filters`	`Optional[list[OnlineEvalFilterInput]]`	`false`	Conditions that decide which traces the rule applies to. Leave empty to match every trace or session of this `sourceType`. Default: `[]`
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluator_list_ids`	`Optional[list[str]]`	`false`	The evaluator lists to run on matched traces. Each list bundles one or more evaluators — this is what the dashboard calls a "Metric". You can create and list them via the `/api/public/evaluator-lists` endpoint. Passing a non-empty array turns on the online-evaluation action. Default: `[]`

Param	Type	Required	Description
`name`	`String`	`true`	Human-readable name for the rule, 1-255 characters.
`description`	`String`	`false`	Optional free-text description shown alongside the rule in the UI.
`sourceType`	`EventSchedulerType`	`false`	Defaults to `TRACE` if you don't set it. You can't change this after the rule is created. Session rules unlock `triggerTraceCount` and support a smaller set of filter fields (see `OnlineEvalFilterField`).
`datasetId`	`String`	`false`	The dataset to add matches to. Setting this turns on the dataset-addition action. If you use `LLM_OUTPUT` mapping, you also need to set `datasetTag`.
`datasetMappingType`	`DatasetMappingType`	`false`	How the trace becomes a dataset item.
`datasetExtractionMode`	`DatasetExtractionMode`	`false`	Whether to use the whole trace or a specific observation as the item.
`outputObservationName`	`String`	`false`	If `extractionMode` targets a specific observation, this is its span name.
`datasetTag`	`String`	`false`	Required when `datasetMappingType` is `LLM_OUTPUT`.
`samplingRate`	`Integer`	`false`	Percent of matching traces the rule fires on (1–100). Default: `100`
`delay`	`Integer`	`false`	Seconds to wait before firing, so late-arriving spans can complete. Default: `30`
`triggerTraceCount`	`Integer`	`false`	Session rules only. Set to `0` to disable (the rule fires on the idle `delay` instead). Passing a non-zero value on a trace rule returns an error. Default: `0`
`status`	`Boolean`	`false`	Set to `false` to create the rule paused instead of active. Default: `true`
`filters`	`List<OnlineEvalFilter>`	`false`	Conditions that decide which traces the rule applies to. Leave empty to match every trace or session of this `sourceType`. Default: `[]`
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluatorListIds`	`List<String>`	`false`	The evaluator lists to run on matched traces. Each list bundles one or more evaluators — this is what the dashboard calls a "Metric". You can create and list them via the `/api/public/evaluator-lists` endpoint. Passing a non-empty array turns on the online-evaluation action. Default: `[]`

Param	Type	Required	Description
`Name`	`string`	`true`	Human-readable name for the rule, 1-255 characters.
`Description`	`*string`	`false`	Optional free-text description shown alongside the rule in the UI.
`SourceType`	`*EventSchedulerType`	`false`	Defaults to `TRACE` if you don't set it. You can't change this after the rule is created. Session rules unlock `triggerTraceCount` and support a smaller set of filter fields (see `OnlineEvalFilterField`).
`DatasetId`	`*string`	`false`	The dataset to add matches to. Setting this turns on the dataset-addition action. If you use `LLM_OUTPUT` mapping, you also need to set `datasetTag`.
`DatasetMappingType`	`*DatasetMappingType`	`false`	How the trace becomes a dataset item.
`DatasetExtractionMode`	`*DatasetExtractionMode`	`false`	Whether to use the whole trace or a specific observation as the item.
`OutputObservationName`	`*string`	`false`	If `extractionMode` targets a specific observation, this is its span name.
`DatasetTag`	`*string`	`false`	Required when `datasetMappingType` is `LLM_OUTPUT`.
`SamplingRate`	`*int`	`false`	Percent of matching traces the rule fires on (1–100). Default: `100`
`Delay`	`*int`	`false`	Seconds to wait before firing, so late-arriving spans can complete. Default: `30`
`TriggerTraceCount`	`*int`	`false`	Session rules only. Set to `0` to disable (the rule fires on the idle `delay` instead). Passing a non-zero value on a trace rule returns an error. Default: `0`
`Status`	`*bool`	`false`	Set to `false` to create the rule paused instead of active. Default: `true`
`Filters`	`*[]OnlineEvalFilter`	`false`	Conditions that decide which traces the rule applies to. Leave empty to match every trace or session of this `sourceType`. Default: `[]`
`Filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`Filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`Filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`Filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`EvaluatorListIds`	`*[]string`	`false`	The evaluator lists to run on matched traces. Each list bundles one or more evaluators — this is what the dashboard calls a "Metric". You can create and list them via the `/api/public/evaluator-lists` endpoint. Passing a non-empty array turns on the online-evaluation action. Default: `[]`

List rules

page, limit := 1, 1
res, err := client.List(context.Background(), &ListOnlineEvalsParams{
	Page:  &page,
	Limit: &limit,
})

page, limit := 1, 1res, err := client.List(context.Background(), &ListOnlineEvalsParams{	Page:  &page,	Limit: &limit,})

Each row also includes TraceCount — the number of distinct traces the rule has evaluated so far.

Lists the rules in your project, paginated. You can page through results and search by name — richer filtering isn't available on this endpoint.

Param	Type	Required	Description
`page`	`number`	`false`	1-indexed page number (default 1) Default: `1`
`limit`	`number`	`false`	Page size, max 100 (default 50) Default: `50`
`search`	`string`	`false`	Substring match on name + description (case-insensitive)

Param	Type	Required	Description
`page`	`Optional[int]`	`false`	1-indexed page number (default 1) Default: `1`
`limit`	`Optional[int]`	`false`	Page size, max 100 (default 50) Default: `50`
`search`	`Optional[str]`	`false`	Substring match on name + description (case-insensitive)

Param	Type	Required	Description
`page`	`Integer`	`false`	1-indexed page number (default 1) Default: `1`
`limit`	`Integer`	`false`	Page size, max 100 (default 50) Default: `50`
`search`	`String`	`false`	Substring match on name + description (case-insensitive)

Param	Type	Required	Description
`Page`	`*int`	`false`	1-indexed page number (default 1) Default: `1`
`Limit`	`*int`	`false`	Page size, max 100 (default 50) Default: `50`
`Search`	`*string`	`false`	Substring match on name + description (case-insensitive)

Get a rule by id

rule, err := client.Get(context.Background(), ruleID)

rule, err := client.Get(context.Background(), ruleID)

Get one online evaluation rule by id

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id

Update a rule

sr := 80
status := false
rule, err := client.Update(context.Background(), ruleID, &UpdateOnlineEvalBody{
	SamplingRate: &sr,
	Status:       &status,
})

sr := 80status := falserule, err := client.Update(context.Background(), ruleID, &UpdateOnlineEvalBody{	SamplingRate: &sr,	Status:       &status,})

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`name`	`string`	`false`	New rule name.
`description`	`string \| null`	`false`	New description. Pass `null` to clear it.
`onlineEvalEnabled`	`boolean`	`false`	Explicitly pause or resume the online-evaluation action. If you leave it out, the server derives it from `evaluatorListIds` (when that's part of this update).
`datasetEnabled`	`boolean`	`false`	Explicitly pause or resume the dataset-addition action. If you leave it out, the server derives it from `datasetId` (when that's part of this update).
`datasetId`	`string`	`false`	New dataset id.
`datasetMappingType`	`DatasetMappingType`	`false`	New dataset mapping type.
`datasetExtractionMode`	`DatasetExtractionMode`	`false`	New dataset extraction mode.
`outputObservationName`	`string \| null`	`false`	New target observation name for extraction.
`datasetTag`	`string \| null`	`false`	New tag applied to items added by this rule.
`samplingRate`	`number`	`false`	New sampling rate (1–100).
`delay`	`number`	`false`	New idle delay in seconds.
`triggerTraceCount`	`number`	`false`	Session rules only.
`status`	`boolean`	`false`	Set to `false` to pause the rule without deleting it.
`filters`	`OnlineEvalFilter[]`	`false`	New filter conditions. Pass an empty array to clear.
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluatorListIds`	`string[]`	`false`	The evaluator lists to attach — see the same field on the create body. This replaces what was there before; pass the full set you want, not a delta. Sending a non-empty array turns on online eval; an empty one turns it off.

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id
`name`	`Optional[str]`	`false`	New rule name.
`description`	`Optional[str]`	`false`	New description. Pass `null` to clear it.
`online_eval_enabled`	`Optional[bool]`	`false`	Explicitly pause or resume the online-evaluation action. If you leave it out, the server derives it from `evaluatorListIds` (when that's part of this update).
`dataset_enabled`	`Optional[bool]`	`false`	Explicitly pause or resume the dataset-addition action. If you leave it out, the server derives it from `datasetId` (when that's part of this update).
`dataset_id`	`Optional[str]`	`false`	New dataset id.
`dataset_mapping_type`	`Optional[DatasetMappingType]`	`false`	New dataset mapping type.
`dataset_extraction_mode`	`Optional[DatasetExtractionMode]`	`false`	New dataset extraction mode.
`output_observation_name`	`Optional[str]`	`false`	New target observation name for extraction.
`dataset_tag`	`Optional[str]`	`false`	New tag applied to items added by this rule.
`sampling_rate`	`Optional[int]`	`false`	New sampling rate (1–100).
`delay`	`Optional[int]`	`false`	New idle delay in seconds.
`trigger_trace_count`	`Optional[int]`	`false`	Session rules only.
`status`	`Optional[bool]`	`false`	Set to `false` to pause the rule without deleting it.
`filters`	`Optional[list[OnlineEvalFilterInput]]`	`false`	New filter conditions. Pass an empty array to clear.
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluator_list_ids`	`Optional[list[str]]`	`false`	The evaluator lists to attach — see the same field on the create body. This replaces what was there before; pass the full set you want, not a delta. Sending a non-empty array turns on online eval; an empty one turns it off.

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id
`name`	`String`	`false`	New rule name.
`description`	`String`	`false`	New description. Pass `null` to clear it.
`onlineEvalEnabled`	`Boolean`	`false`	Explicitly pause or resume the online-evaluation action. If you leave it out, the server derives it from `evaluatorListIds` (when that's part of this update).
`datasetEnabled`	`Boolean`	`false`	Explicitly pause or resume the dataset-addition action. If you leave it out, the server derives it from `datasetId` (when that's part of this update).
`datasetId`	`String`	`false`	New dataset id.
`datasetMappingType`	`DatasetMappingType`	`false`	New dataset mapping type.
`datasetExtractionMode`	`DatasetExtractionMode`	`false`	New dataset extraction mode.
`outputObservationName`	`String`	`false`	New target observation name for extraction.
`datasetTag`	`String`	`false`	New tag applied to items added by this rule.
`samplingRate`	`Integer`	`false`	New sampling rate (1–100).
`delay`	`Integer`	`false`	New idle delay in seconds.
`triggerTraceCount`	`Integer`	`false`	Session rules only.
`status`	`Boolean`	`false`	Set to `false` to pause the rule without deleting it.
`filters`	`List<OnlineEvalFilter>`	`false`	New filter conditions. Pass an empty array to clear.
`filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`evaluatorListIds`	`List<String>`	`false`	The evaluator lists to attach — see the same field on the create body. This replaces what was there before; pass the full set you want, not a delta. Sending a non-empty array turns on online eval; an empty one turns it off.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`Name`	`*string`	`false`	New rule name.
`Description`	`*string`	`false`	New description. Pass `null` to clear it.
`OnlineEvalEnabled`	`*bool`	`false`	Explicitly pause or resume the online-evaluation action. If you leave it out, the server derives it from `evaluatorListIds` (when that's part of this update).
`DatasetEnabled`	`*bool`	`false`	Explicitly pause or resume the dataset-addition action. If you leave it out, the server derives it from `datasetId` (when that's part of this update).
`DatasetId`	`*string`	`false`	New dataset id.
`DatasetMappingType`	`*DatasetMappingType`	`false`	New dataset mapping type.
`DatasetExtractionMode`	`*DatasetExtractionMode`	`false`	New dataset extraction mode.
`OutputObservationName`	`*string`	`false`	New target observation name for extraction.
`DatasetTag`	`*string`	`false`	New tag applied to items added by this rule.
`SamplingRate`	`*int`	`false`	New sampling rate (1–100).
`Delay`	`*int`	`false`	New idle delay in seconds.
`TriggerTraceCount`	`*int`	`false`	Session rules only.
`Status`	`*bool`	`false`	Set to `false` to pause the rule without deleting it.
`Filters`	`*[]OnlineEvalFilter`	`false`	New filter conditions. Pass an empty array to clear.
`Filters.fieldName`	`OnlineEvalFilterField \| string`	`true`	One of the trace / observation / session fields (see `OnlineEvalFilterField` for the list) or a custom `metadata.<key>` path. Invalid combinations for the rule's `sourceType` return `400`.
`Filters.operator`	`"EQUALS" \| "NOT_EQUALS" \| "CONTAINS" \| "NOT_CONTAINS" \| "STARTS_WITH" \| "ENDS_WITH" \| "REGEX" \| "IN" \| "NOT_IN" \| "GREATER_THAN" \| "LESS_THAN" \| "GREATER_THAN_OR_EQUAL" \| "LESS_THAN_OR_EQUAL" \| "IS_EMPTY" \| "IS_NOT_EMPTY"`	`true`	The comparison to run.
`Filters.value`	`string`	`true`	What to compare against. For `IN` / `NOT_IN`, pass a comma-separated list. For `IS_EMPTY` / `IS_NOT_EMPTY`, this is ignored.
`Filters.logicalOperator`	`"AND" \| "OR"`	`false`	How this filter combines with the one before it. Default: `"AND"`
`EvaluatorListIds`	`*[]string`	`false`	The evaluator lists to attach — see the same field on the create body. This replaces what was there before; pass the full set you want, not a delta. Sending a non-empty array turns on online eval; an empty one turns it off.

Delete a rule

res, err := client.Delete(context.Background(), ruleID)

res, err := client.Delete(context.Background(), ruleID)

Delete an online evaluation rule

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id

Analytics

Four read-only endpoints for pulling evaluation results into your own reports or dashboards.

Evaluator scores

Returns aggregated evaluator scores over a time window — one row per evaluator, per time bucket.

from := time.Date(2026, 6, 24, 0, 0, 0, 0, time.UTC)
to := time.Date(2026, 6, 25, 0, 0, 0, 0, time.UTC)
granularity := Granularity("hour")

resp, err := client.Scores(context.Background(), ruleID, &ScoresParams{
	FromTimestamp: &from,
	ToTimestamp:   &to,
	Granularity:   &granularity,
})

from := time.Date(2026, 6, 24, 0, 0, 0, 0, time.UTC)to := time.Date(2026, 6, 25, 0, 0, 0, 0, time.UTC)granularity := Granularity("hour")resp, err := client.Scores(context.Background(), ruleID, &ScoresParams{	FromTimestamp: &from,	ToTimestamp:   &to,	Granularity:   &granularity,})

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`fromTimestamp`	`Date \| string`	`false`	Inclusive lower bound, ISO 8601. Defaults to `toTimestamp − 24h`.
`toTimestamp`	`Date \| string`	`false`	Inclusive upper bound, ISO 8601. Defaults to the server clock at request time.
`granularity`	`Granularity`	`false`	Bucket width. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id
`from_timestamp`	`Optional[str]`	`false`	Inclusive lower bound, ISO 8601. Defaults to `toTimestamp − 24h`.
`to_timestamp`	`Optional[str]`	`false`	Inclusive upper bound, ISO 8601. Defaults to the server clock at request time.
`granularity`	`Optional[str]`	`false`	Bucket width. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id
`fromTimestamp`	`java.time.Instant`	`false`	Inclusive lower bound, ISO 8601. Defaults to `toTimestamp − 24h`.
`toTimestamp`	`java.time.Instant`	`false`	Inclusive upper bound, ISO 8601. Defaults to the server clock at request time.
`granularity`	`Granularity`	`false`	Bucket width. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`FromTimestamp`	`*time.Time`	`false`	Inclusive lower bound, ISO 8601. Defaults to `toTimestamp − 24h`.
`ToTimestamp`	`*time.Time`	`false`	Inclusive upper bound, ISO 8601. Defaults to the server clock at request time.
`Granularity`	`*Granularity`	`false`	Bucket width. When omitted, points are returned ungrouped.

Observation metrics

granularity := Granularity("day")
resp, err := client.Metrics(context.Background(), ruleID, &MetricsParams{
	Granularity: &granularity,
})

granularity := Granularity("day")resp, err := client.Metrics(context.Background(), ruleID, &MetricsParams{	Granularity: &granularity,})

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`fromTimestamp`	`Date \| string`	`false`	ISO 8601. Defaults to `toTimestamp − 24h`.
`toTimestamp`	`Date \| string`	`false`	ISO 8601. Defaults to server clock.
`granularity`	`Granularity`	`false`	Bucket width for the timeseries. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id
`from_timestamp`	`Optional[str]`	`false`	ISO 8601. Defaults to `toTimestamp − 24h`.
`to_timestamp`	`Optional[str]`	`false`	ISO 8601. Defaults to server clock.
`granularity`	`Optional[str]`	`false`	Bucket width for the timeseries. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id
`fromTimestamp`	`java.time.Instant`	`false`	ISO 8601. Defaults to `toTimestamp − 24h`.
`toTimestamp`	`java.time.Instant`	`false`	ISO 8601. Defaults to server clock.
`granularity`	`Granularity`	`false`	Bucket width for the timeseries. When omitted, points are returned ungrouped.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id
`FromTimestamp`	`*time.Time`	`false`	ISO 8601. Defaults to `toTimestamp − 24h`.
`ToTimestamp`	`*time.Time`	`false`	ISO 8601. Defaults to server clock.
`Granularity`	`*Granularity`	`false`	Bucket width for the timeseries. When omitted, points are returned ungrouped.

Trace list (matched by a rule)

Lists the traces a specific rule has evaluated. You can filter by score, environment, tags, and more.

filterSet := &OnlineEvalTraceFilterSet{
	Must: &[]OnlineEvalTraceFilter{},
	Any:  &[]OnlineEvalTraceFilter{},
}
_, err := client.ListTraces(context.Background(), ruleID, &ListTracesParams{
	Filter: filterSet,
})

filterSet := &OnlineEvalTraceFilterSet{	Must: &[]OnlineEvalTraceFilter{},	Any:  &[]OnlineEvalTraceFilter{},}_, err := client.ListTraces(context.Background(), ruleID, &ListTracesParams{	Filter: filterSet,})

Lists the traces a rule has evaluated. Supports pagination and rich filtering — see the SDK docs for the filter DSL, allowed columns, and search options.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id.
`page`	`number`	`false`	1-indexed page number. Default: `1`
`limit`	`number`	`false`	Page size, max 100. Default: `50`
`searchQuery`	`string`	`false`	Substring match against the columns configured by `searchType`.
`searchType`	`('id' \| 'content')[]`	`false`	JSON-encoded `TracingSearchType[]` — e.g. `["id"]`, `["content"]`, or both. `id` matches small identifier columns; `content` substring-matches input/output text.
`filter`	`OnlineEvalTraceFilterSet`	`false`	JSON-encoded `OnlineEvalTraceFilterSet` — `{ must?, any? }`. See the schema in components for the shape.
`orderBy`	`{ column: TraceFilterColumn`	`false`	JSON-encoded `{ column: TraceFilterColumn, order: "ASC" \| "DESC" }`.

Param	Type	Required	Description
`rule_id`	`str`	`true`	The rule's id.
`page`	`Optional[int]`	`false`	1-indexed page number. Default: `1`
`limit`	`Optional[int]`	`false`	Page size, max 100. Default: `50`
`search_query`	`Optional[str]`	`false`	Substring match against the columns configured by `searchType`.
`search_type`	`Optional[str]`	`false`	JSON-encoded `TracingSearchType[]` — e.g. `["id"]`, `["content"]`, or both. `id` matches small identifier columns; `content` substring-matches input/output text.
`filter`	`Optional[str]`	`false`	JSON-encoded `OnlineEvalTraceFilterSet` — `{ must?, any? }`. See the schema in components for the shape.
`order_by`	`Optional[str]`	`false`	JSON-encoded `{ column: TraceFilterColumn, order: "ASC" \| "DESC" }`.

Param	Type	Required	Description
`ruleId`	`String`	`true`	The rule's id.
`page`	`Integer`	`false`	1-indexed page number. Default: `1`
`limit`	`Integer`	`false`	Page size, max 100. Default: `50`
`searchQuery`	`String`	`false`	Substring match against the columns configured by `searchType`.
`searchType`	`String`	`false`	JSON-encoded `TracingSearchType[]` — e.g. `["id"]`, `["content"]`, or both. `id` matches small identifier columns; `content` substring-matches input/output text.
`filter`	`String`	`false`	JSON-encoded `OnlineEvalTraceFilterSet` — `{ must?, any? }`. See the schema in components for the shape.
`orderBy`	`String`	`false`	JSON-encoded `{ column: TraceFilterColumn, order: "ASC" \| "DESC" }`.

Param	Type	Required	Description
`ruleId`	`string`	`true`	The rule's id.
`Page`	`*int`	`false`	1-indexed page number. Default: `1`
`Limit`	`*int`	`false`	Page size, max 100. Default: `50`
`SearchQuery`	`*string`	`false`	Substring match against the columns configured by `searchType`.
`SearchType`	`[]ListTracesSearchType`	`false`	JSON-encoded `TracingSearchType[]` — e.g. `["id"]`, `["content"]`, or both. `id` matches small identifier columns; `content` substring-matches input/output text.
`Filter`	`*OnlineEvalTraceFilterSet`	`false`	JSON-encoded `OnlineEvalTraceFilterSet` — `{ must?, any? }`. See the schema in components for the shape.
`OrderBy`	`*ListTracesOrderBy`	`false`	JSON-encoded `{ column: TraceFilterColumn, order: "ASC" \| "DESC" }`.

Trace detail (single trace + evaluator scores)

Returns a single trace with all its evaluator scores attached — the same view you'd see when expanding a row in the dashboard.

_, err := client.Trace(context.Background(), traceID)

_, err := client.Trace(context.Background(), traceID)

Param	Type	Required	Description
`traceId`	`string`	`true`	The trace's id (`.id` from a `listTraces` page or any other source in the project).

Param	Type	Required	Description
`trace_id`	`str`	`true`	The trace's id (`.id` from a `listTraces` page or any other source in the project).

Param	Type	Required	Description
`traceId`	`String`	`true`	The trace's id (`.id` from a `listTraces` page or any other source in the project).

Param	Type	Required	Description
`traceId`	`string`	`true`	The trace's id (`.id` from a `listTraces` page or any other source in the project).

Complete example

Every step above, stitched together into one block you can copy and adapt. The ruleID is grabbed off the create response; the traceID is picked from the first row of the list-traces response (fall back to a real trace id if the list is empty on a fresh rule).

// 1. Set up the SDK
import (
	"context"
	"log"

	aisdk "github.com/browserstack/ai-sdk-go"
)

ctx := context.Background()

sdk, err := aisdk.New(aisdk.WithCredentials(
	"pk-lf-your-public-key",
	"sk-lf-your-secret-key",
))
if err != nil {
	log.Fatalf("aisdk.New: %v", err)
}

client := sdk.OnlineEval()

// 2. Per-run variables
name := "Production faithfulness scoring"
evaluatorListID := "YOUR_EVALUATOR_LIST_ID"

// 3. Create a rule
desc := "Score 25% of production traces on faithfulness evaluator"
sr, delay := 25, 120
status := true
rule, err := client.Create(context.Background(), &CreateOnlineEvalBody{
	Name:             vars.name,
	Description:      &desc,
	SamplingRate:     &sr,
	Delay:            &delay,
	Status:           &status,
	Filters:          &[]OnlineEvalFilter{},
	EvaluatorListIds: ptr([]string{evaluatorListID}),
})

// 4. Grab the id for later steps
ruleID := rule.Id

// 5. List rules
page, limit := 1, 1
res, err := client.List(context.Background(), &ListOnlineEvalsParams{
	Page:  &page,
	Limit: &limit,
})

// 6. Fetch a rule by id
rule, err := client.Get(context.Background(), ruleID)

// 7. Update the rule
sr := 80
status := false
rule, err := client.Update(context.Background(), ruleID, &UpdateOnlineEvalBody{
	SamplingRate: &sr,
	Status:       &status,
})

// 8. Aggregated evaluator scores
from := time.Date(2026, 6, 24, 0, 0, 0, 0, time.UTC)
to := time.Date(2026, 6, 25, 0, 0, 0, 0, time.UTC)
granularity := Granularity("hour")

resp, err := client.Scores(context.Background(), ruleID, &ScoresParams{
	FromTimestamp: &from,
	ToTimestamp:   &to,
	Granularity:   &granularity,
})

// 9. Observation metrics
granularity := Granularity("day")
resp, err := client.Metrics(context.Background(), ruleID, &MetricsParams{
	Granularity: &granularity,
})

// 10. Traces matched by a rule
filterSet := &OnlineEvalTraceFilterSet{
	Must: &[]OnlineEvalTraceFilter{},
	Any:  &[]OnlineEvalTraceFilter{},
}
_, err := client.ListTraces(context.Background(), ruleID, &ListTracesParams{
	Filter: filterSet,
})

// 11. Pick a traceID from the list
traceID := "<TRACE_ID>"
if len(resp.Data) > 0 {
	traceID = resp.Data[0].Id
}

// 12. Trace detail with evaluator scores
_, err := client.Trace(context.Background(), traceID)

// 13. Delete the rule
res, err := client.Delete(context.Background(), ruleID)

// 1. Set up the SDKimport (	"context"	"log"	aisdk "github.com/browserstack/ai-sdk-go")ctx := context.Background()sdk, err := aisdk.New(aisdk.WithCredentials(	"pk-lf-your-public-key",	"sk-lf-your-secret-key",))if err != nil {	log.Fatalf("aisdk.New: %v", err)}client := sdk.OnlineEval()// 2. Per-run variablesname := "Production faithfulness scoring"evaluatorListID := "YOUR_EVALUATOR_LIST_ID"// 3. Create a ruledesc := "Score 25% of production traces on faithfulness evaluator"sr, delay := 25, 120status := truerule, err := client.Create(context.Background(), &CreateOnlineEvalBody{	Name:             vars.name,	Description:      &desc,	SamplingRate:     &sr,	Delay:            &delay,	Status:           &status,	Filters:          &[]OnlineEvalFilter{},	EvaluatorListIds: ptr([]string{evaluatorListID}),})// 4. Grab the id for later stepsruleID := rule.Id// 5. List rulespage, limit := 1, 1res, err := client.List(context.Background(), &ListOnlineEvalsParams{	Page:  &page,	Limit: &limit,})// 6. Fetch a rule by idrule, err := client.Get(context.Background(), ruleID)// 7. Update the rulesr := 80status := falserule, err := client.Update(context.Background(), ruleID, &UpdateOnlineEvalBody{	SamplingRate: &sr,	Status:       &status,})// 8. Aggregated evaluator scoresfrom := time.Date(2026, 6, 24, 0, 0, 0, 0, time.UTC)to := time.Date(2026, 6, 25, 0, 0, 0, 0, time.UTC)granularity := Granularity("hour")resp, err := client.Scores(context.Background(), ruleID, &ScoresParams{	FromTimestamp: &from,	ToTimestamp:   &to,	Granularity:   &granularity,})// 9. Observation metricsgranularity := Granularity("day")resp, err := client.Metrics(context.Background(), ruleID, &MetricsParams{	Granularity: &granularity,})// 10. Traces matched by a rulefilterSet := &OnlineEvalTraceFilterSet{	Must: &[]OnlineEvalTraceFilter{},	Any:  &[]OnlineEvalTraceFilter{},}_, err := client.ListTraces(context.Background(), ruleID, &ListTracesParams{	Filter: filterSet,})// 11. Pick a traceID from the listtraceID := "<TRACE_ID>"if len(resp.Data) > 0 {	traceID = resp.Data[0].Id}// 12. Trace detail with evaluator scores_, err := client.Trace(context.Background(), traceID)// 13. Delete the ruleres, err := client.Delete(context.Background(), ruleID)

Online Evaluations

On this page