aievals CLI

CLI for the AI Evals platform — create datasets, run experiments, score LLM responses, and manage evaluators from the terminal.

Step 1: Install

# One-line install — no sudo needed
curl -fsSL https://static.testops.ai/cli/install.sh | sh

irm https://static.testops.ai/cli/install.ps1 | iex

# Verify
aievals --version

Installs to ~/.local/bin and auto-adds to PATH. Tab completion (zsh, bash, fish) is installed automatically.

Step 2: Login

aievals auth login \
  --public-key <YOUR_PUBLIC_KEY> \
  --secret-key <YOUR_SECRET_KEY> \
  --host https://evals.browserstack.com

Keys are pk-to-... and sk-to-... from your project's API settings page: Settings → API Keys in the AI Evals web UI.

Step 3: Verify

aievals doctor

You should see API reachable, authenticated, and your project name.

Step 4: Try it

# List datasets
aievals dataset list --format json

# List traces
aievals trace list --limit 5 --format json

# List experiments
aievals experiment list --limit 5 --format json

# Discover available LLM providers and models
aievals provider list --format json

# Full help
aievals --help

Step 5: Let an AI agent drive it

If you use Claude Code or Cursor, install the skill file so the agent knows how to drive this CLI without guessing at commands:

# Run inside your project — auto-detects Claude Code / Cursor
aievals skill install

Then just ask the agent in plain English — "list all datasets and create one called my-tests" — and it runs the right aievals commands for you. See AI Agents for targeting a specific agent, install paths, and CI/CD usage.

Key things to know

Output format: Table in the terminal, JSON when piped. Use --format json to force JSON.

Aliases: aievals metrics = aievals evaluator-list (same command, UI name).

Next steps