Manage Evaluation Datasets

You can manage evaluation datasets from the Dashboard UI, the SDK (TypeScript, Python, Java), or the REST API. Each SDK exposes a dedicated evaluationDataset client scoped to the operations that make sense for this kind — create, list, add items, and run experiments.

From the Dashboard

View Evaluation Datasets

Navigate to Datasets in the left sidebar.

Select the Evaluation Datasets tab at the top of the Dataset Library page. The URL deep-links via ?tab=evaluation.

The list shows one row per evaluation dataset with these columns:

Column	Description
Name	Dataset name (click to open detail page)
Items	Number of items in the dataset
Created	Creation date
Last updated	Date of the most recent item edit or upload

Use the search bar to filter by name.

Create an Evaluation Dataset

On the Evaluation Datasets tab, click New Evaluation Dataset in the top-right.

Fill in the form:

Name (required) — unique dataset name
Description (optional) — free-form notes about the dataset
Metadata (optional) — JSON key-value pairs for custom metadata

Click Create. You'll be taken to the new evaluation dataset's detail page.

Evaluation Dataset Detail Page

Click any evaluation dataset name to open its detail page. The page has three tabs:

Items — view, add, and manage items (see Evaluation Dataset Items)
Linked Experiments — experiments that reference this evaluation dataset
Linked Rules — automation rules attached to this evaluation dataset

The header shows a Last saved timestamp and a Versions button so you can pin experiments to a specific version of the dataset. There is no Create Dataset Run button — evaluation datasets feed experiments directly (see Run Experiments).

Dataset Actions

Click the more menu (three dots) on the evaluation dataset detail page for:

Rename — update the dataset name and description
Delete — permanently remove the evaluation dataset and all its items

From the SDK

Setup

import { AISDK } from '@browserstack/ai-sdk';

const testOps = new AISDK({
  publicKey: process.env.AISDK_PUBLIC_KEY,
  secretKey: process.env.AISDK_SECRET_KEY,
});

const evaluationDataset = testOps.evaluationDataset; // EvaluationDatasetsClient

import os
from browserstack_ai_sdk import AISDK

client = AISDK(
    public_key=os.environ["AISDK_PUBLIC_KEY"],
    secret_key=os.environ["AISDK_SECRET_KEY"],
)

evaluation_dataset = client.evaluation_dataset  # EvaluationDatasets

import com.browserstack.aisdk.TestOps;
import com.browserstack.aisdk.eval.EvaluationDatasetsClient;

TestOps sdk = TestOps.fromEnv();
EvaluationDatasetsClient evaluationDataset = sdk.evaluationDataset();

Create an Evaluation Dataset

const dataset = await evaluationDataset.create(
  'support-prod-traces',
  'Captured support responses from production',
  { source: 'prod-traces', window: '2026-04' } // optional metadata
);

console.log(dataset.id);   // evaluation dataset ID
console.log(dataset.name); // 'support-prod-traces'

dataset = client.evaluation_dataset.create(
    name="support-prod-traces",
    description="Captured support responses from production",
    metadata='{"source": "prod-traces"}',
)
print(dataset)

// Name only
DatasetResponse ds = evaluationDataset.create("support-prod-traces");

// With description
DatasetResponse ds = evaluationDataset.create(
    "support-prod-traces",
    "Captured support responses from production"
);

// With metadata
DatasetResponse ds = evaluationDataset.create(
    "support-prod-traces",
    "Captured support responses from production",
    Map.of("source", "prod-traces", "window", "2026-04")
);

System.out.println("Created: " + ds.getId() + " — " + ds.getName());

List Evaluation Datasets

The list call is scoped to kind=EVALUATION automatically — only evaluation datasets are returned.

const result = await evaluationDataset.list(
  1,      // page (1-indexed)
  20,     // limit per page
  'qa'    // optional name filter
);

for (const ds of result.data) {
  console.log(ds.id, ds.name);
}

result = client.evaluation_dataset.list(page=1, limit=20)

for dataset in result["data"]:
    print(f"{dataset.get('name')}: {dataset.get('id')}")

Filter by name:

datasets = client.evaluation_dataset.list(name="support-prod-traces")

// Default (page 1, limit 50)
ListDatasetsResponse list = evaluationDataset.list();

// With pagination
ListDatasetsResponse list = evaluationDataset.list(1, 20);

// Filtered by name
ListDatasetsResponse list = evaluationDataset.list(1, 20, "support");

list.getData().forEach(d -> System.out.println(d.getId() + " " + d.getName()));

Manage Evaluation Datasets

On this page