Experiments API

Experiments associate a dataset and an evaluator list (and optionally a prompt) to measure LLM pipeline quality systematically. Each experiment can have multiple runs.

For local experiment runs — running a task in your own code and pushing results from your machine — use the SDK or CLI; see Local Experiment Runs.

Create Experiment

POST /api/public/experiments

Two creation modes are supported:

Mode 1 — by dataset run tag (use a tagged dataset run as ground truth):

Field	Type	Required	Description
`name`	`string`	Yes	Experiment name
`description`	`string`	No	Description
`datasetRunTagId`	`string`	Yes	Tag ID from a dataset run
`evaluatorListId`	`string`	Yes	Evaluator list ID
`concurrency`	`integer`	No	Max concurrent eval executions

Mode 2 — by prompt + dataset (run evaluations against a prompt on a dataset):

Field	Type	Required	Description
`name`	`string`	Yes	Experiment name
`description`	`string`	No	Description
`promptId`	`string`	Yes	Prompt ID
`datasetId`	`string`	Yes	Dataset ID
`evaluatorListId`	`string`	Yes	Evaluator list ID
`concurrency`	`integer`	No	Max concurrent eval executions

cURL Example (tag-based)

curl -X POST https://evals-api.browserstack.com/api/public/experiments \
  -u "pk-lf-...:sk-lf-..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "rag-eval-baseline",
    "datasetRunTagId": "tag-uuid-1",
    "evaluatorListId": "eval-list-uuid-1"
  }'

cURL Example (prompt-based)

curl -X POST https://evals-api.browserstack.com/api/public/experiments \
  -u "pk-lf-...:sk-lf-..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "prompt-eval-v2",
    "promptId": "prompt-uuid-1",
    "datasetId": "ds-uuid-1",
    "evaluatorListId": "eval-list-uuid-1"
  }'

Response

{
  "id": "exp-uuid-1",
  "name": "rag-eval-baseline",
  "description": null,
  "concurrency": 10,
  "projectId": "proj-xyz",
  "datasetId": "ds-uuid-1",
  "promptId": null,
  "datasetRunTagId": "tag-uuid-1",
  "experimentEvaluatorId": "eval-list-uuid-1",
  "createdBy": "user-123",
  "createdAt": "2026-04-03T10:00:00.000Z",
  "updatedAt": "2026-04-03T10:00:00.000Z"
}

List Experiments

GET /api/public/experiments

Parameter	Type	Description
`page`	`integer`	Page number
`limit`	`integer`	Items per page

curl "https://evals-api.browserstack.com/api/public/experiments?page=1&limit=20" \
  -u "pk-lf-...:sk-lf-..."

Response:

{
  "experiments": [
    {
      "id": "exp-uuid-1",
      "name": "rag-eval-baseline",
      "projectId": "proj-xyz",
      "datasetId": "ds-uuid-1",
      "promptId": null,
      "datasetRunTagId": "tag-uuid-1",
      "experimentEvaluatorId": "eval-list-uuid-1",
      "concurrency": 10,
      "createdBy": "user-123",
      "createdAt": "2026-04-03T10:00:00.000Z",
      "updatedAt": "2026-04-03T10:00:00.000Z"
    }
  ],
  "totalCount": 1
}

Get Experiment

GET /api/public/experiments/{experimentId}

curl "https://evals-api.browserstack.com/api/public/experiments/exp-uuid-1" \
  -u "pk-lf-...:sk-lf-..."

Experiment Runs

Create Experiment Run

POST /api/public/experiment-runs

Field	Type	Required	Description
`experimentId`	`string`	Yes	Experiment ID
`name`	`string`	No	Run name
`description`	`string`	No	Description
`metadata`	`object`	No	Metadata

curl -X POST https://evals-api.browserstack.com/api/public/experiment-runs \
  -u "pk-lf-...:sk-lf-..." \
  -H "Content-Type: application/json" \
  -d '{
    "experimentId": "exp-uuid-1",
    "name": "run-2026-04-03"
  }'

Response:

{
  "id": "run-uuid-1",
  "experimentId": "exp-uuid-1",
  "name": "run-2026-04-03",
  "status": "PENDING",
  "createdAt": "2026-04-03T10:00:00.000Z",
  "updatedAt": "2026-04-03T10:00:00.000Z"
}

List Experiment Runs

GET /api/public/experiment-runs

Parameter	Type	Description
`experimentId`	`string`	Filter by experiment ID
`page`	`integer`	Page number
`limit`	`integer`	Items per page

curl "https://evals-api.browserstack.com/api/public/experiment-runs?experimentId=exp-uuid-1" \
  -u "pk-lf-...:sk-lf-..."

Get Experiment Run

GET /api/public/experiment-runs/{runId}

curl "https://evals-api.browserstack.com/api/public/experiment-runs/run-uuid-1" \
  -u "pk-lf-...:sk-lf-..."

Get Experiment Run Summary

GET /api/public/experiment-runs/{experimentRunId}/summary

Returns scores, metrics, diffs, and UI deep-links for a run in a single call. This endpoint composes data from multiple internal sources so you do not need to orchestrate separate requests for scores, latency, and cost.

Path Parameters

Parameter	Type	Required	Description
`experimentRunId`	`string`	Yes	ID of the experiment run to summarise

Query Parameters

Parameter	Type	Required	Default	Description
`compareToRunId`	`string`	No	—	ID of the baseline run to diff against. When omitted, the server uses the experiment's UI-pinned baseline (set via "Set as baseline" in the experiment view). If neither is set, no comparison is performed: `comparisonExperimentRunId` is `null` and every score's `diff` is `null`. The server does NOT auto-pick a baseline by recency.

cURL Examples

Minimal — uses the experiment's UI-pinned baseline (or returns no comparison if none):

curl "https://evals-api.browserstack.com/api/public/experiment-runs/run-uuid-1/summary" \
  -u "pk-lf-...:sk-lf-..."

With an explicit baseline:

curl "https://evals-api.browserstack.com/api/public/experiment-runs/run-uuid-1/summary\
?compareToRunId=run-uuid-0" \
  -u "pk-lf-...:sk-lf-..."

Response

{
  "projectId": "aa7557d1-0241-4a8e-91c4-184f55ab64a1",
  "projectName": "Demo Project",

  "experimentId": "cmpe0fezj01zrjx08m4wpr77y",
  "experimentName": "checkout-flow-v3",

  "experimentRunId": "cmpe0ff0g01zvjx08std8i28o",
  "experimentRunName": "run-2026-06-08-prompt-v3",

  "comparisonExperimentRunId": "cmpe0fabc01zvjx08abc1234",
  "comparisonExperimentRunName": "run-2026-06-07-prompt-v2",

  "urls": {
    "project":      "https://evals.browserstack.com/project/aa7557d1-...",
    "experiment":   "https://evals.browserstack.com/project/aa7557d1-.../experiments/cmpe0fezj...",
    "run":          "https://evals.browserstack.com/project/aa7557d1-.../experiments/cmpe0fezj.../runs?runId=cmpe0ff0g...",
    "scoresTab":    "https://evals.browserstack.com/project/aa7557d1-.../experiments/cmpe0fezj.../runs?runId=cmpe0ff0g...&tab=scores",
    "analyticsTab": "https://evals.browserstack.com/project/aa7557d1-.../experiments/cmpe0fezj.../runs?runId=cmpe0ff0g...&tab=analytics"
  },

  "scores": [
    {
      "name":            "exact_match",
      "dataType":        "NUMERIC",
      "score":           0.82,
      "passCount":       8,
      "failCount":       2,
      "totalItems":      10,
      "threshold":       0.70,
      "criteria":        "greater_than",
      "thresholdStatus": "pass",
      "diff":            0.07,
      "improvements":    5,
      "regressions":     2,
      "source":          "EVAL",
      "version":         3,
      "versionLabel":    "v3-prompt-tuned"
    },
    {
      "name":            "verdict",
      "dataType":        "CATEGORICAL",
      "score":           null,
      "categoryCounts":  { "pass": 8, "fail": 2, "skip": 0 },
      "passCategories":  ["pass"],
      "ignoreCategories":["skip"],
      "totalItems":      10,
      "threshold":       null,
      "criteria":        null,
      "thresholdStatus": "pass",
      "diff":            null,
      "improvements":    3,
      "regressions":     1,
      "source":          "EVAL"
    }
  ],

  "metrics": {
    "duration":          { "name": "duration",          "metric": 0.632,    "unit": "s",   "diff": -0.198 },
    "latency_avg":       { "name": "latency_avg",        "metric": 412,      "unit": "ms",  "diff": -85    },
    "latency_p50":       { "name": "latency_p50",        "metric": 380,      "unit": "ms",  "diff": -70    },
    "latency_p90":       { "name": "latency_p90",        "metric": 720,      "unit": "ms",  "diff": -120   },
    "latency_p99":       { "name": "latency_p99",        "metric": 1240,     "unit": "ms",  "diff": -200   },
    "prompt_tokens":     { "name": "prompt_tokens",      "metric": 14,       "unit": "tok", "diff": 0      },
    "completion_tokens": { "name": "completion_tokens",  "metric": 12,       "unit": "tok", "diff": -4     },
    "total_tokens":      { "name": "total_tokens",       "metric": 26,       "unit": "tok", "diff": -4     },
    "total_cost":        { "name": "total_cost",         "metric": 0.000074, "unit": "$",   "diff": -0.00002 },
    "cost_per_call":     { "name": "cost_per_call",      "metric": 0.000037, "unit": "$",   "diff": -0.00001 }
  },

  "summary": {
    "verdict":        "PASS",
    "failureCount":   0,
    "improvedCount":  6,
    "regressedCount": 2
  }
}

Score fields

scores is an array of score summaries, ordered by the server with the worst-regressed evaluators first (sort key: regressions desc, improvements desc, name asc). Each entry contains one of two shapes, discriminated by dataType. Look an evaluator up by iterating, e.g. scores.find(s => s.name === "exact_match").

Numeric score (dataType: "NUMERIC")

Field	Type	Description
`name`	`string`	Evaluator name
`dataType`	`"NUMERIC"`	Discriminator
`score`	`number \| null`	Mean score across all dataset items
`minScore`	`number`	Minimum score (when available)
`maxScore`	`number`	Maximum score (when available)
`p90Score`	`number`	90th-percentile score (when available)
`passCount`	`number`	Items meeting the threshold
`failCount`	`number`	Items below the threshold
`totalItems`	`number`	Total dataset items scored
`threshold`	`number \| null`	Configured pass threshold
`criteria`	`string \| null`	Threshold direction (e.g. `"greater_than"`)
`thresholdStatus`	`"pass" \| "fail" \| null`	Whether the mean score meets the threshold
`diff`	`number \| null`	Mean score change vs baseline (`null` when no baseline)
`improvements`	`number`	Items that moved toward passing vs baseline
`regressions`	`number`	Items that moved away from passing vs baseline
`source`	`"EVAL" \| "ANNOTATION"`	Score origin
`version`	`number`	Evaluator version (when available)
`versionLabel`	`string`	Evaluator version label (when available)

Note: minScore, maxScore, and p90Score are present when the server has per-item distribution data for the run. They may be absent in v1 for some evaluator configurations.

Categorical score (dataType: "CATEGORICAL")

Field	Type	Description
`name`	`string`	Evaluator name
`dataType`	`"CATEGORICAL"`	Discriminator
`score`	`null`	Always `null` for categorical scores
`categoryCounts`	`object`	Map of category label to item count
`passCategories`	`string[]`	Which categories count as passing
`ignoreCategories`	`string[]`	Categories excluded from pass/fail accounting
`neutralCount`	`number`	Items in ignored categories (when available)
`totalItems`	`number`	Total dataset items scored
`thresholdStatus`	`"pass" \| "fail" \| null`	Whether the run meets its categorical threshold
`diff`	`null`	Categorical diff is not computed in v1
`improvements`	`number`	Items that moved into a pass category vs baseline
`regressions`	`number`	Items that moved out of a pass category vs baseline
`source`	`"EVAL" \| "ANNOTATION"`	Score origin

Note: neutralCount is present when the server has per-item categorical classification data. It may be absent in v1 for some evaluator configurations.

Metric fields

Each entry in metrics is keyed by metric name and contains:

Field	Type	Description
`name`	`string`	Metric name
`metric`	`number`	Aggregate value for this run
`unit`	`string`	Unit code: `s`, `ms`, `tok`, `$`, or `""`
`diff`	`number \| null`	Change vs baseline run (`null` when no baseline)

Metrics are emitted only when the underlying data exists for the run. The full set of possible metric names is: duration, latency_avg, latency_p50, latency_p90, latency_p95, latency_p99, llm_latency_avg, llm_latency_sum, eval_latency_avg, prompt_tokens, completion_tokens, total_tokens, eval_prompt_tokens, eval_completion_tokens, total_cost, cost_per_call, eval_cost.

Comparison Verdict

GET /api/public/experiment-comparisons/verdict

Returns the AI Verdict for a comparison between two runs of the same experiment — which run is better, a confidence level, a reasoning sentence, and the per-scorer drivers behind the call. The verdict is produced by a fully deterministic (no-LLM) engine and is saved; this endpoint is read-only and returns the last generated verdict (or NOT_GENERATED if none exists yet).

Generate or refresh a verdict from the experiment compare page in the UI. See the AI Verdict platform guide.

This endpoint authenticates with a project API key (the same pk-lf-...:sk-lf-... pair used elsewhere).

Query Parameters

Parameter	Type	Required	Description
`experimentId`	`string`	Yes	Experiment the two runs belong to
`baseRunId`	`string`	Yes	Baseline run ID
`targetRunId`	`string`	Yes	Target run ID (compared against the baseline)

cURL Example

curl "https://evals-api.browserstack.com/api/public/experiment-comparisons/verdict\
?experimentId=exp-uuid-1&baseRunId=run-uuid-0&targetRunId=run-uuid-1" \
  -u "pk-lf-...:sk-lf-..."

Response

{
  "status": "READY",
  "baseRunId": "run-uuid-0",
  "targetRunId": "run-uuid-1",
  "winner": "TARGET",
  "confidence": "HIGH",
  "reasoning": "Target run wins: exact_match rose from 0.74 to 0.86 (+0.12) and faithfulness held steady; no scorer regressed.",
  "drivers": [
    {
      "name": "exact_match",
      "role": "FOR",
      "baseValue": 0.74,
      "targetValue": 0.86,
      "normalized": 0.12,
      "items": 50,
      "lowData": false
    },
    {
      "name": "faithfulness",
      "role": "NEUTRAL",
      "baseValue": 0.91,
      "targetValue": 0.91,
      "normalized": 0.0,
      "items": 50,
      "lowData": false
    }
  ],
  "generatedAt": "2026-06-24T10:00:00.000Z"
}

Response Fields

Field	Type	Description
`status`	`"READY" \| "STALE" \| "DECLINED" \| "NOT_GENERATED"`	Verdict state (see below)
`baseRunId`	`string`	Echo of the baseline run ID
`targetRunId`	`string`	Echo of the target run ID
`winner`	`"BASE" \| "TARGET" \| null`	Which run is better; `null` unless `status` is `READY` or `STALE`
`confidence`	`"HIGH" \| "MEDIUM" \| "LOW" \| null`	How decisive the result is; `null` when no winner
`reasoning`	`string \| null`	Templated, plain-English explanation citing scorer numbers; `null` when no winner
`drivers`	`array \| null`	Per-scorer breakdown (see below); `null` when no winner
`generatedAt`	`string \| null`	ISO timestamp the saved verdict was generated; `null` when `NOT_GENERATED`

Each entry in drivers:

Field	Type	Description
`name`	`string`	Evaluator name
`role`	`"FOR" \| "AGAINST" \| "NEUTRAL"`	Whether this scorer favored the winner, opposed it, or was neutral
`baseValue`	`number`	Scorer aggregate on the baseline run
`targetValue`	`number`	Scorer aggregate on the target run
`normalized`	`number`	Scaled gap, comparable across scorers on different scales
`items`	`number`	Comparable scored items behind this scorer
`lowData`	`boolean`	`true` when this scorer had too few items to count fully

Status values

Status	`winner` / `drivers`	Meaning
`READY`	populated	A verdict exists and matches the current scores
`STALE`	populated	A verdict exists but a run's scores changed since it was generated — refresh it from the UI
`DECLINED`	`null`	Not enough comparable data to call a winner
`NOT_GENERATED`	`null`	No verdict has been generated for this run pair yet

Errors

Status	When
`400 Bad Request`	A required query parameter (`experimentId`, `baseRunId`, `targetRunId`) is missing or malformed
`404 Not Found`	The experiment or one of the runs does not exist, or a run does not belong to the experiment

Typical Workflow

# 1. Create or identify a dataset
curl -X POST https://evals-api.browserstack.com/api/public/datasets \
  -u "pk-lf-...:sk-lf-..." \
  -d '{ "name": "my-dataset" }'

# 2. Upload items to the dataset (see Datasets API)

# 3. Create an evaluator list (see Evaluators API)

# 4. Create an experiment
curl -X POST https://evals-api.browserstack.com/api/public/experiments \
  -u "pk-lf-...:sk-lf-..." \
  -d '{
    "name": "my-experiment",
    "datasetId": "<dataset-id>",
    "promptId": "<prompt-id>",
    "evaluatorListId": "<eval-list-id>"
  }'

# 5. Trigger a run
curl -X POST https://evals-api.browserstack.com/api/public/experiment-runs \
  -u "pk-lf-...:sk-lf-..." \
  -d '{ "experimentId": "<experiment-id>", "name": "run-1" }'

Experiments API

On this page