Launch experiments that score the pre-captured outputs in an evaluation dataset — no model re-run.

Run Experiments on Evaluation Datasets

When you run an experiment on an evaluation dataset, the platform forwards each item's captured output straight into the run and asks the selected evaluators to score it. No prompt is executed, no API is called, and no LLM cost is incurred.

For the broader experiment concept — prompts, dataset runs, scheduling — see the Experiments section.

From the Dashboard

Open Experiments in the left sidebar and click New Experiment.

In the Source step, choose Evaluation Dataset. No prompt or API section is shown — outputs are forwarded from the dataset items.

Pick one of three modes:

Existing — select an evaluation dataset you've already created
Upload new — create a new evaluation dataset from a CSV during experiment creation (the CSV must include the output column)
Upload as new version — add a new version to an existing evaluation dataset

Pin a dataset version if needed:

Leave Latest selected to always run on the newest version of the dataset.
Pick a specific version number to freeze the experiment against a particular snapshot of the items.

Select the Evaluator List you want to score the outputs with.

Set Concurrency (1–10) — how many items to score in parallel. Higher values finish faster but use more evaluator capacity.

Click Run Experiment. The run appears under the experiment with status PROCESSING and updates to COMPLETED once all evaluators have scored every item. There are no model calls during this run.

Reviewing Results

The experiment detail page for an evaluation-dataset experiment differs from a prompt-based one in two ways:

The header reads Evaluation Dataset Details instead of Dataset Details and links to the evaluation dataset's detail page.
The prompt section is hidden — there is no prompt or model parameter chip in the Version Chip Strip. The strip still shows the dataset version chip and the evaluator metrics chip.

Click any item to see its input, captured output, expectedOutput, and the score(s) produced by each evaluator.

From the SDK

The dedicated evaluationDataset client exposes a runExperiment / run_experiment method that creates an experiment scoped to an evaluation dataset. It accepts name, datasetId, and evaluatorListId, plus optional description, concurrency, and datasetVersion.

Run an Experiment

const experiment = await evaluationDataset.runExperiment({
  name: 'replay-support-traces-v2',
  datasetId: dataset.id,
  evaluatorListId: 'evl_answer_quality',
  description: 'Re-score April production traces with the v2 rubric',
  concurrency: 5,
});

console.log(experiment.id);
console.log(experiment.status); // 'PROCESSING' → 'COMPLETED'

experiment = client.evaluation_dataset.run_experiment({
    "name": "replay-support-traces-v2",
    "datasetId": dataset["id"],
    "evaluatorListId": "evl_answer_quality",
    "description": "Re-score April production traces with the v2 rubric",
    "concurrency": 5,
})

print(experiment["id"])
print(experiment["status"])  # 'PROCESSING' -> 'COMPLETED'

ExperimentResponse experiment = evaluationDataset.runExperiment(
    "replay-support-traces-v2",
    dataset.getId(),
    "evl_answer_quality"
);

// Or with description, concurrency, and pinned version
ExperimentResponse experiment = evaluationDataset.runExperiment(
    "replay-support-traces-v2",
    dataset.getId(),
    "evl_answer_quality",
    "Re-score April production traces with the v2 rubric",
    5,    // concurrency
    null  // datasetVersion (null = latest)
);

System.out.println(experiment.getId());
System.out.println(experiment.getStatus());

Pinning a Dataset Version

Pass datasetVersion to freeze the experiment against a specific version of the evaluation dataset. Pass null (or omit) to always use the latest version.

await evaluationDataset.runExperiment({
  name: 'compare-rubric-v1-vs-v2',
  datasetId: dataset.id,
  evaluatorListId: 'evl_answer_quality',
  datasetVersion: 3, // pin to version 3
});

client.evaluation_dataset.run_experiment({
    "name": "compare-rubric-v1-vs-v2",
    "datasetId": dataset["id"],
    "evaluatorListId": "evl_answer_quality",
    "datasetVersion": 3,
})

evaluationDataset.runExperiment(
    "compare-rubric-v1-vs-v2",
    dataset.getId(),
    "evl_answer_quality",
    null,
    null,
    3 // pin to version 3
);

Complete Example

End-to-end example: create an evaluation dataset, add captured outputs, and run an experiment that scores them.

import { AISDK } from '@browserstack/ai-sdk';

const testOps = new AISDK({
  publicKey: process.env.AISDK_PUBLIC_KEY,
  secretKey: process.env.AISDK_SECRET_KEY,
});

const evaluationDataset = testOps.evaluationDataset;

async function rescoreCapturedOutputs() {
  // 1. Create an evaluation dataset
  const dataset = await evaluationDataset.create(
    'support-prod-traces',
    'Captured support responses from April'
  );

  // 2. Add items with pre-captured outputs
  await evaluationDataset.createItems({
    datasetName: 'support-prod-traces',
    items: [
      {
        input: { question: 'How do I cancel my subscription?' },
        output: { answer: 'Go to Settings → Billing → Cancel.' },
        expectedOutput: { answer: 'Cancel from Settings → Billing.' },
      },
      {
        input: { question: 'What is your refund policy?' },
        output: { answer: 'Refunds are issued within 14 days.' },
      },
    ],
  });

  // 3. Run an experiment that scores those outputs
  const experiment = await evaluationDataset.runExperiment({
    name: 'april-replay-with-v2-rubric',
    datasetId: dataset.id,
    evaluatorListId: 'evl_answer_quality',
    concurrency: 5,
  });

  console.log(`Experiment ${experiment.id} started — no model calls made`);
  await testOps.shutdown();
}

rescoreCapturedOutputs();

import os
from browserstack_ai_sdk import AISDK

client = AISDK(
    public_key=os.environ["AISDK_PUBLIC_KEY"],
    secret_key=os.environ["AISDK_SECRET_KEY"],
)

# 1. Create an evaluation dataset
dataset = client.evaluation_dataset.create(
    name="support-prod-traces",
    description="Captured support responses from April",
)

# 2. Add items with pre-captured outputs
client.evaluation_dataset.create_items(
    dataset_name="support-prod-traces",
    items=[
        {
            "input": {"question": "How do I cancel my subscription?"},
            "output": {"answer": "Go to Settings → Billing → Cancel."},
            "expectedOutput": {"answer": "Cancel from Settings → Billing."},
        },
        {
            "input": {"question": "What is your refund policy?"},
            "output": {"answer": "Refunds are issued within 14 days."},
        },
    ],
)

# 3. Run an experiment that scores those outputs
experiment = client.evaluation_dataset.run_experiment({
    "name": "april-replay-with-v2-rubric",
    "datasetId": dataset["id"],
    "evaluatorListId": "evl_answer_quality",
    "concurrency": 5,
})

print(f"Experiment {experiment['id']} started — no model calls made")
client.flush()

Run Experiments on Evaluation Datasets

Run Experiments on Evaluation Datasets

From the Dashboard

Reviewing Results

From the SDK

Run an Experiment

Pinning a Dataset Version

Complete Example

On this page