Add items with captured outputs to an evaluation dataset from the dashboard or SDK — single items, batch upload, or CSV import.

Evaluation Dataset Items

Every item in an evaluation dataset must include an input and an output — the captured response that evaluators will score. expectedOutput, context, and metadata are optional.

From the Dashboard

Open any evaluation dataset and select the Items tab to view and manage items.

Items Table

Each row shows one item with these columns:

Column	Description
Item ID	Unique item identifier (click to open item detail)
Source	Link to the originating trace, observation, or session (if the item was created from a trace)
Status	`ACTIVE` or `ARCHIVED`
Input	JSON viewer (expandable)
Output	JSON viewer (expandable) — the captured response that evaluators score
Expected Output	JSON viewer (expandable) — reference answer for evaluators that need ground truth
Context	JSON viewer (hidden by default)
Metadata	JSON viewer (hidden by default)
Added by	Automation rule name or user who created the item

Use the column visibility toggle to show/hide columns.

Add a Single Item

Click New Item in the top-right of the Items tab.

Fill in the form:

Input (required) — JSON object, array, or double-quoted string
Output (required) — the captured response to be scored. JSON or string.
Expected Output (optional) — the reference answer used by evaluators that need ground truth
Metadata (optional) — JSON key-value pairs

Click Add to Dataset.

Upload CSV

Click Upload CSV in the top-right of the Items tab. It is the primary action on this page.

Drag and drop a CSV file or click to browse. Maximum file size is 100 MB.

The CSV must contain both an input column and an output column. Optional columns: expectedOutput, context, metadata. The upload is rejected if the output header is missing.

Preview the parsed data and confirm to import. Items are added to the dataset in bulk.

Item Actions

Click the actions menu on any item row for:

Archive / Unarchive — toggle the item's status between ACTIVE and ARCHIVED
Delete — permanently remove the item

Bulk Actions

Select multiple items using the checkboxes, then use the Actions dropdown:

Compare — compare selected items side by side (requires 2+ items)
Export Selected — export selected items

Use the Batch Export button in the toolbar to export all visible items as CSV or JSON.

From the SDK

The evaluationDataset.createItems / create_items / createItems method on the dedicated evaluation client adds items to an evaluation dataset. Every item must include output.

Create Items (Batch)

await evaluationDataset.createItems({
  datasetName: 'support-prod-traces',
  items: [
    {
      input: { question: 'How do I cancel my subscription?' },
      output: { answer: 'Go to Settings → Billing → Cancel.' },
      expectedOutput: { answer: 'Cancel from Settings → Billing.' },
      metadata: { source_trace: 'trace_123' },
    },
    {
      input: { question: 'What is your refund policy?' },
      output: { answer: 'Refunds are issued within 14 days.' },
    },
  ],
});

result = client.evaluation_dataset.create_items(
    dataset_name="support-prod-traces",
    items=[
        {
            "input": {"question": "How do I cancel my subscription?"},
            "output": {"answer": "Go to Settings → Billing → Cancel."},
            "expectedOutput": {"answer": "Cancel from Settings → Billing."},
            "metadata": {"source_trace": "trace_123"},
        },
        {
            "input": {"question": "What is your refund policy?"},
            "output": {"answer": "Refunds are issued within 14 days."},
        },
    ],
)
print(f"Added {result['itemCount']} items")

Items are sent in batches of 100.

import com.browserstack.aisdk.eval.model.CreateDatasetItemRequest;
import java.util.List;
import java.util.Map;

List<CreateDatasetItemRequest> items = List.of(
    CreateDatasetItemRequest.builder()
        .input("How do I cancel my subscription?")
        .output("Go to Settings → Billing → Cancel.")
        .expectedOutput("Cancel from Settings → Billing.")
        .metadata(Map.of("source_trace", "trace_123"))
        .build(),

    CreateDatasetItemRequest.builder()
        .input("What is your refund policy?")
        .output("Refunds are issued within 14 days.")
        .build()
);

CreateDatasetItemsResponse result = evaluationDataset.createItems("support-prod-traces", items);
System.out.println("Added " + result.getItemCount() + " items");

Import from CSV

Your CSV must include both input and output columns. Optional columns: expectedOutput, context, metadata, id, sourceTraceId, sourceObservationId, status.

input,output,expectedOutput,context,metadata
"What is 2+2?","2+2 is 4.","4","math textbook","{""difficulty"": ""easy""}"
"Capital of France?","The capital of France is Paris.","Paris","geography quiz","{""difficulty"": ""medium""}"

const result = await evaluationDataset.createItems({
  datasetName: 'support-prod-traces',
  fileUrl: '/path/to/captured-outputs.csv',
});
console.log(`Imported ${result.itemCount} items`);

result = client.evaluation_dataset.create_items(
    dataset_name="support-prod-traces",
    file_url="/path/to/captured-outputs.csv",
    options={"batchSize": 50},
)
print(f"Imported {result['itemCount']} items")

CreateDatasetItemsResponse result = evaluationDataset.createItemsFromCsv(
    "support-prod-traces",
    "/path/to/captured-outputs.csv"
);
System.out.println("Imported " + result.getItemCount() + " items");

Override batch size:

evaluationDataset.createItemsFromCsv("support-prod-traces", "/path/to/captured-outputs.csv", 50);

Evaluation Dataset Items

On this page