AI Agents

Run this inside your project directory:

aievals agent setup

This auto-detects Claude Code or Cursor and writes a skill file so the agent knows how to use the CLI. After this, you can give Claude Code prompts like:

List all datasets in the project and create a new one called "my-test-dataset"

Or a full experiment workflow:

Using the aievals CLI, create a dataset with 3 items, create a code evaluator, create a metrics group, create a prompt with model params, create an experiment linking them all, and run it with --wait

Without agent setup, Claude Code still works — it'll discover the CLI via --help. The agent setup just makes it faster (skips the --help discovery step).