Trace Management
Publish, tag, and convert traces into dataset items from the terminal with the aievals CLI trace write commands.
Trace Management
Four write verbs on aievals trace, alongside the existing read commands
(trace list, trace get):
| Command | What it does |
|---|---|
trace publish | Make a trace publicly viewable, or revoke that |
trace tag | Replace a trace's tags |
trace add-to-dataset | Convert traces into dataset items |
trace add-to-eval-dataset | Convert traces into eval-dataset items |
Publish a trace
Sets a trace's public visibility.
aievals trace publish tr_123 --yes
aievals trace publish tr_123 --public=falseA public trace is readable by anyone with the link, without a login. Making a
trace public therefore asks for confirmation — pass --yes to skip the prompt
in scripts. Revoking does not prompt.
Tag a trace
aievals trace tag tr_123 --tag production --tag reviewed
aievals trace tag tr_123 --clear # removes all tags--tag is repeatable. Tagging replaces the tag list wholesale — the server
does not merge, so the tags you pass become the complete set.
Clearing every tag requires an explicit --clear; omitting --tag on its own
is an error. That way a script that computes zero tags cannot silently wipe a
trace.
This is the write verb. trace list --tag is a read filter — different command,
different meaning.
Convert traces into dataset items
add-to-dataset enqueues matching traces as items on a dataset. Select traces
either explicitly with repeatable --trace-id, or by --filter. The two
are mutually exclusive, and one of them is required.
aievals trace add-to-dataset --dataset-id ds_1 --trace-id tr_1 --trace-id tr_2aievals trace add-to-dataset --dataset-id ds_1 --mapping-type LLM_OUTPUT \
--filter '{"must":[{"fieldName":"environment","operator":"IN","value":["prod"]}]}'| Flag | Notes |
|---|---|
--dataset-id | Destination dataset ID (required) |
--trace-id | Trace to add (repeatable; exclusive with --filter) |
--filter | Trace filter as JSON (exclusive with --trace-id) |
--dataset-tag | Tag applied to the created items |
--mapping-type | EXPECTED_OUTPUT, LLM_OUTPUT |
Filter shape. --filter is inline JSON:
{"must":[{"fieldName":"environment","operator":"IN","value":["prod"]}],"any":[]}operator is the server enum (EQUALS, IN, CONTAINS, GREATER_THAN, and
so on). Some columns accept only certain operators — environment, for one,
requires IN or NOT_IN with an array value.
On success the command returns the destination, not the items:
{"datasetId":"ds_1","datasetName":"qa-golden-set"}The conversion is asynchronous — the command enqueues the work and returns
straight away. The items appear once a worker processes the job, so an
immediate dataset-item list can legitimately come back empty.
This is a bulk trace-to-dataset conversion. To add a single hand-written item
instead, use dataset-item create.
Convert traces into eval-dataset items
The eval-dataset counterpart. It takes no --dataset-tag and no
--mapping-type — the eval flow derives both.
aievals trace add-to-eval-dataset --dataset-id ds_1 --trace-id tr_1| Flag | Notes |
|---|---|
--dataset-id | Destination eval dataset ID (required) |
--trace-id | Trace to add (repeatable; exclusive with --filter) |
--filter | Trace filter as JSON (exclusive with --trace-id) |
Success returns the destination plus how many traces were accepted:
{"datasetId":"ds_1","datasetName":"qa-golden-set","traceCount":1}Two server-side constraints apply here:
- Every named trace must already have an output. Traces without one are rejected.
--filtersupports onlymust, notany.
Related
Alerts & Rules
Manage alerts, automation rules, and online-eval rules from the terminal — create thresholds on trace metrics, evaluate traces automatically, and read back rule results.
Agent Endpoints
Manage agents — HTTP webhook and voice endpoints the platform calls during dataset runs — with the aievals CLI.