Experiments API

Experiments associate a dataset and an evaluator list (and optionally a prompt) to measure LLM pipeline quality systematically. Each experiment can have multiple runs.

Create Experiment

POST /api/public/experiments

Two creation modes are supported:

Mode 1 — by dataset run tag (use a tagged dataset run as ground truth):

Field	Type	Required	Description
`name`	`string`	Yes	Experiment name
`description`	`string`	No	Description
`datasetRunTagId`	`string`	Yes	Tag ID from a dataset run
`evaluatorListId`	`string`	Yes	Evaluator list ID
`concurrency`	`integer`	No	Max concurrent eval executions

Mode 2 — by prompt + dataset (run evaluations against a prompt on a dataset):

Field	Type	Required	Description
`name`	`string`	Yes	Experiment name
`description`	`string`	No	Description
`promptId`	`string`	Yes	Prompt ID
`datasetId`	`string`	Yes	Dataset ID
`evaluatorListId`	`string`	Yes	Evaluator list ID
`concurrency`	`integer`	No	Max concurrent eval executions

cURL Example (tag-based)

curl -X POST https://evals-api.browserstack.com/api/public/experiments \
  -u "pk-lf-...:sk-lf-..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "rag-eval-baseline",
    "datasetRunTagId": "tag-uuid-1",
    "evaluatorListId": "eval-list-uuid-1"
  }'

cURL Example (prompt-based)

curl -X POST https://evals-api.browserstack.com/api/public/experiments \
  -u "pk-lf-...:sk-lf-..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "prompt-eval-v2",
    "promptId": "prompt-uuid-1",
    "datasetId": "ds-uuid-1",
    "evaluatorListId": "eval-list-uuid-1"
  }'

Response

{
  "id": "exp-uuid-1",
  "name": "rag-eval-baseline",
  "description": null,
  "concurrency": 10,
  "projectId": "proj-xyz",
  "datasetId": "ds-uuid-1",
  "promptId": null,
  "datasetRunTagId": "tag-uuid-1",
  "experimentEvaluatorId": "eval-list-uuid-1",
  "createdBy": "user-123",
  "createdAt": "2026-04-03T10:00:00.000Z",
  "updatedAt": "2026-04-03T10:00:00.000Z"
}

List Experiments

GET /api/public/experiments

Parameter	Type	Description
`page`	`integer`	Page number
`limit`	`integer`	Items per page

curl "https://evals-api.browserstack.com/api/public/experiments?page=1&limit=20" \
  -u "pk-lf-...:sk-lf-..."

Response:

{
  "experiments": [
    {
      "id": "exp-uuid-1",
      "name": "rag-eval-baseline",
      "projectId": "proj-xyz",
      "datasetId": "ds-uuid-1",
      "promptId": null,
      "datasetRunTagId": "tag-uuid-1",
      "experimentEvaluatorId": "eval-list-uuid-1",
      "concurrency": 10,
      "createdBy": "user-123",
      "createdAt": "2026-04-03T10:00:00.000Z",
      "updatedAt": "2026-04-03T10:00:00.000Z"
    }
  ],
  "totalCount": 1
}

Get Experiment

GET /api/public/experiments/{experimentId}

curl "https://evals-api.browserstack.com/api/public/experiments/exp-uuid-1" \
  -u "pk-lf-...:sk-lf-..."

Delete Experiment

DELETE /api/public/experiments/{experimentId}

curl -X DELETE "https://evals-api.browserstack.com/api/public/experiments/exp-uuid-1" \
  -u "pk-lf-...:sk-lf-..."

Experiment Runs

Create Experiment Run

POST /api/public/experiment-runs

Field	Type	Required	Description
`experimentId`	`string`	Yes	Experiment ID
`name`	`string`	No	Run name
`description`	`string`	No	Description
`metadata`	`object`	No	Metadata

curl -X POST https://evals-api.browserstack.com/api/public/experiment-runs \
  -u "pk-lf-...:sk-lf-..." \
  -H "Content-Type: application/json" \
  -d '{
    "experimentId": "exp-uuid-1",
    "name": "run-2026-04-03"
  }'

Response:

{
  "id": "run-uuid-1",
  "experimentId": "exp-uuid-1",
  "name": "run-2026-04-03",
  "status": "PENDING",
  "createdAt": "2026-04-03T10:00:00.000Z",
  "updatedAt": "2026-04-03T10:00:00.000Z"
}

List Experiment Runs

GET /api/public/experiment-runs

Parameter	Type	Description
`experimentId`	`string`	Filter by experiment ID
`page`	`integer`	Page number
`limit`	`integer`	Items per page

curl "https://evals-api.browserstack.com/api/public/experiment-runs?experimentId=exp-uuid-1" \
  -u "pk-lf-...:sk-lf-..."

Get Experiment Run

GET /api/public/experiment-runs/{runId}

curl "https://evals-api.browserstack.com/api/public/experiment-runs/run-uuid-1" \
  -u "pk-lf-...:sk-lf-..."

Delete Experiment Run

DELETE /api/public/experiment-runs/{runId}

curl -X DELETE "https://evals-api.browserstack.com/api/public/experiment-runs/run-uuid-1" \
  -u "pk-lf-...:sk-lf-..."

Get Experiment Run Summary

GET /api/public/experiment-runs/{experimentRunId}/summary

Returns scores, metrics, diffs, and UI deep-links for a run in a single call. This endpoint composes data from multiple internal sources so you do not need to orchestrate separate requests for scores, latency, and cost.

Path Parameters

Parameter	Type	Required	Description
`experimentRunId`	`string`	Yes	ID of the experiment run to summarise

Query Parameters

Parameter	Type	Required	Default	Description
`compareToRunId`	`string`	No	—	ID of the baseline run to diff against. When omitted, the server uses the experiment's UI-pinned baseline (set via "Set as baseline" in the experiment view). If neither is set, no comparison is performed: `comparisonExperimentRunId` is `null` and every score's `diff` is `null`. The server does NOT auto-pick a baseline by recency.

cURL Examples

Minimal — uses the experiment's UI-pinned baseline (or returns no comparison if none):

curl "https://evals-api.browserstack.com/api/public/experiment-runs/run-uuid-1/summary" \
  -u "pk-lf-...:sk-lf-..."

With an explicit baseline:

curl "https://evals-api.browserstack.com/api/public/experiment-runs/run-uuid-1/summary\
?compareToRunId=run-uuid-0" \
  -u "pk-lf-...:sk-lf-..."

Response

{
  "projectId": "aa7557d1-0241-4a8e-91c4-184f55ab64a1",
  "projectName": "Demo Project",

  "experimentId": "cmpe0fezj01zrjx08m4wpr77y",
  "experimentName": "checkout-flow-v3",

  "experimentRunId": "cmpe0ff0g01zvjx08std8i28o",
  "experimentRunName": "run-2026-06-08-prompt-v3",

  "comparisonExperimentRunId": "cmpe0fabc01zvjx08abc1234",
  "comparisonExperimentRunName": "run-2026-06-07-prompt-v2",

  "urls": {
    "project":      "https://evals.browserstack.com/project/aa7557d1-...",
    "experiment":   "https://evals.browserstack.com/project/aa7557d1-.../experiments/cmpe0fezj...",
    "run":          "https://evals.browserstack.com/project/aa7557d1-.../experiments/cmpe0fezj.../runs?runId=cmpe0ff0g...",
    "scoresTab":    "https://evals.browserstack.com/project/aa7557d1-.../experiments/cmpe0fezj.../runs?runId=cmpe0ff0g...&tab=scores",
    "analyticsTab": "https://evals.browserstack.com/project/aa7557d1-.../experiments/cmpe0fezj.../runs?runId=cmpe0ff0g...&tab=analytics"
  },

  "scores": [
    {
      "name":            "exact_match",
      "dataType":        "NUMERIC",
      "score":           0.82,
      "passCount":       8,
      "failCount":       2,
      "totalItems":      10,
      "threshold":       0.70,
      "criteria":        "greater_than",
      "thresholdStatus": "pass",
      "diff":            0.07,
      "improvements":    5,
      "regressions":     2,
      "source":          "EVAL",
      "version":         3,
      "versionLabel":    "v3-prompt-tuned"
    },
    {
      "name":            "verdict",
      "dataType":        "CATEGORICAL",
      "score":           null,
      "categoryCounts":  { "pass": 8, "fail": 2, "skip": 0 },
      "passCategories":  ["pass"],
      "ignoreCategories":["skip"],
      "totalItems":      10,
      "threshold":       null,
      "criteria":        null,
      "thresholdStatus": "pass",
      "diff":            null,
      "improvements":    3,
      "regressions":     1,
      "source":          "EVAL"
    }
  ],

  "metrics": {
    "duration":          { "name": "duration",          "metric": 0.632,    "unit": "s",   "diff": -0.198 },
    "latency_avg":       { "name": "latency_avg",        "metric": 412,      "unit": "ms",  "diff": -85    },
    "latency_p50":       { "name": "latency_p50",        "metric": 380,      "unit": "ms",  "diff": -70    },
    "latency_p90":       { "name": "latency_p90",        "metric": 720,      "unit": "ms",  "diff": -120   },
    "latency_p99":       { "name": "latency_p99",        "metric": 1240,     "unit": "ms",  "diff": -200   },
    "prompt_tokens":     { "name": "prompt_tokens",      "metric": 14,       "unit": "tok", "diff": 0      },
    "completion_tokens": { "name": "completion_tokens",  "metric": 12,       "unit": "tok", "diff": -4     },
    "total_tokens":      { "name": "total_tokens",       "metric": 26,       "unit": "tok", "diff": -4     },
    "total_cost":        { "name": "total_cost",         "metric": 0.000074, "unit": "$",   "diff": -0.00002 },
    "cost_per_call":     { "name": "cost_per_call",      "metric": 0.000037, "unit": "$",   "diff": -0.00001 }
  },

  "summary": {
    "verdict":        "PASS",
    "failureCount":   0,
    "improvedCount":  6,
    "regressedCount": 2
  }
}

Score fields

scores is an array of score summaries, ordered by the server with the worst-regressed evaluators first (sort key: regressions desc, improvements desc, name asc). Each entry contains one of two shapes, discriminated by dataType. Look an evaluator up by iterating, e.g. scores.find(s => s.name === "exact_match").

Numeric score (dataType: "NUMERIC")

Field	Type	Description
`name`	`string`	Evaluator name
`dataType`	`"NUMERIC"`	Discriminator
`score`	`number \| null`	Mean score across all dataset items
`minScore`	`number`	Minimum score (when available)
`maxScore`	`number`	Maximum score (when available)
`p90Score`	`number`	90th-percentile score (when available)
`passCount`	`number`	Items meeting the threshold
`failCount`	`number`	Items below the threshold
`totalItems`	`number`	Total dataset items scored
`threshold`	`number \| null`	Configured pass threshold
`criteria`	`string \| null`	Threshold direction (e.g. `"greater_than"`)
`thresholdStatus`	`"pass" \| "fail" \| null`	Whether the mean score meets the threshold
`diff`	`number \| null`	Mean score change vs baseline (`null` when no baseline)
`improvements`	`number`	Items that moved toward passing vs baseline
`regressions`	`number`	Items that moved away from passing vs baseline
`source`	`"EVAL" \| "ANNOTATION"`	Score origin
`version`	`number`	Evaluator version (when available)
`versionLabel`	`string`	Evaluator version label (when available)

Note: minScore, maxScore, and p90Score are present when the server has per-item distribution data for the run. They may be absent in v1 for some evaluator configurations.

Categorical score (dataType: "CATEGORICAL")

Field	Type	Description
`name`	`string`	Evaluator name
`dataType`	`"CATEGORICAL"`	Discriminator
`score`	`null`	Always `null` for categorical scores
`categoryCounts`	`object`	Map of category label to item count
`passCategories`	`string[]`	Which categories count as passing
`ignoreCategories`	`string[]`	Categories excluded from pass/fail accounting
`neutralCount`	`number`	Items in ignored categories (when available)
`totalItems`	`number`	Total dataset items scored
`thresholdStatus`	`"pass" \| "fail" \| null`	Whether the run meets its categorical threshold
`diff`	`null`	Categorical diff is not computed in v1
`improvements`	`number`	Items that moved into a pass category vs baseline
`regressions`	`number`	Items that moved out of a pass category vs baseline
`source`	`"EVAL" \| "ANNOTATION"`	Score origin

Note: neutralCount is present when the server has per-item categorical classification data. It may be absent in v1 for some evaluator configurations.

Metric fields

Each entry in metrics is keyed by metric name and contains:

Field	Type	Description
`name`	`string`	Metric name
`metric`	`number`	Aggregate value for this run
`unit`	`string`	Unit code: `s`, `ms`, `tok`, `$`, or `""`
`diff`	`number \| null`	Change vs baseline run (`null` when no baseline)

Metrics are emitted only when the underlying data exists for the run. The full set of possible metric names is: duration, latency_avg, latency_p50, latency_p90, latency_p95, latency_p99, llm_latency_avg, llm_latency_sum, eval_latency_avg, prompt_tokens, completion_tokens, total_tokens, eval_prompt_tokens, eval_completion_tokens, total_cost, cost_per_call, eval_cost.

Typical Workflow

# 1. Create or identify a dataset
curl -X POST https://evals-api.browserstack.com/api/public/datasets \
  -u "pk-lf-...:sk-lf-..." \
  -d '{ "name": "my-dataset" }'

# 2. Upload items to the dataset (see Datasets API)

# 3. Create an evaluator list (see Evaluators API)

# 4. Create an experiment
curl -X POST https://evals-api.browserstack.com/api/public/experiments \
  -u "pk-lf-...:sk-lf-..." \
  -d '{
    "name": "my-experiment",
    "datasetId": "<dataset-id>",
    "promptId": "<prompt-id>",
    "evaluatorListId": "<eval-list-id>"
  }'

# 5. Trigger a run
curl -X POST https://evals-api.browserstack.com/api/public/experiment-runs \
  -u "pk-lf-...:sk-lf-..." \
  -d '{ "experimentId": "<experiment-id>", "name": "run-1" }'

Experiments API

On this page