BrowserStack AI Evals
EvaluationEvaluation Datasets

Evaluation Datasets

Create and manage evaluation datasets — inputs paired with pre-captured outputs that evaluators score directly, without re-running a prompt or API.

Evaluation Datasets

Evaluation Datasets store collections of input + output pairs that have already been generated. Evaluators score the existing outputs directly — no prompt or API is re-run when an experiment is launched. Each item has an input, a required output (the captured response), and optional expectedOutput, context, and metadata.

Use them when you already have model outputs — for example, from production traces, an offline batch run, or an external system — and you want to score that frozen set against one or more evaluators.

You can manage evaluation datasets from:

  • Dashboard UI — create evaluation datasets, add items (single or CSV import), and launch experiments from Datasets → Evaluation Datasets in the left sidebar.
  • SDK — TypeScript, Python, and Java SDKs each expose a dedicated evaluationDataset client (testOps.evaluationDataset, client.evaluation_dataset, sdk.evaluationDataset()) that wraps the create / list / add-items / run-experiment flow.
  • REST API — see the Evaluation Datasets API reference.

Manage Evaluation Datasets

Create, list, rename, and delete evaluation datasets.

Evaluation Dataset Items

Add items with captured outputs, upload from CSV, and manage versions.

Run Experiments

Launch experiments that score the pre-captured outputs without re-running a model.

Dataset Runs

Create dataset runs and link traces to dataset items from the dashboard or SDK.

Manage Evaluation Datasets

Create, list, and manage evaluation datasets from the dashboard, SDK, or API.

On this page

Evaluation Datasets