Agent Endpoints

aievals agent manages agents — the deployed API you want to test. You register the endpoint (URL, method, headers, auth, body) and evals calls out to it as the target of experiments, dataset runs, and the Conversation Simulator.

There is no delete verb in the CLI. Deleting an agent is still available from the dashboard (Manage Agents) and through the SDK — it is the CLI that deliberately omits it.

List and read

aievals agent list --format json
aievals agent list --api-environment-id env123 --format json
aievals agent get agt_123 --format json

list accepts --api-environment-id to filter by API environment. Unlike most list commands in the CLI, it is not paginated — it returns the full array, so there are no --page / --limit flags.

Create

--name, --type, --method, and --url are required.

aievals agent create --name "Support bot" --type HTTP \
  --method POST --url https://api.example.com/chat

With headers, query parameters, and a body template:

aievals agent create --name Bot --type HTTP --method POST --url https://api.example.com/chat \
  --header X-Api-Key=secret \
  --query-param locale=en-US \
  --body '{"prompt":"{{input}}"}'

Update

update is a PATCH — only the flags you pass are sent. --method and --url are required on every update; the server validates this.

aievals agent update agt_123 --method POST --url https://api.example.com/v2/chat
aievals agent update agt_123 --method POST --url https://api.example.com/v2/chat --header X-Api-Key=rotated

Flags

create and update share the same body flags.

Scalars

Name=Value pairs — repeatable, one occurrence per entry:

--header X-Api-Key=secret --header X-Trace-Id=abc

Nested JSON — these have no flat flag equivalent, so they take inline JSON:

Related