Troubleshooting

Known limitations (beta)

Windows: the collector command is not available.
score list does not return experiment/eval scores — use trace list → score get instead.
eval execute may time out on slow LLM providers (30s default per-request timeout).
dataset-run get-by-tag is deprecated — use Evaluation Datasets instead.
Per-request timeout is 30s — long LLM calls may fail. Adjust with --timeout 60.

Problem	Fix
Table output looks garbled/truncated	Use `--format json` for readable output
`command not found: aievals`	Run `curl -fsSL https://static.testops.ai/cli/install.sh \| sh`
`No credentials found`	Run `aievals auth login` (see Quick Start)
Tab completion not working	Restart terminal or run `exec zsh`
`Prompt configuration is missing LLM adapter details`	Add `--model-params` when creating prompts
Evaluator scores all 0	Code: must use `function main(...)`. LLM: check provider name matches `provider list` exactly (case-sensitive)
`API key for provider X not found`	Run `aievals provider list` and use the exact `name` field for `--model-params-provider`
LLM evaluator silent 0 scores	Wrong provider name — use `name` field from provider list, not `adapter`
`score list` returns empty for eval scores	Use `trace list` → `score get <score-id>` instead
Code evaluator `Invalid or unexpected token`	Shell escaping issue — write code to a file, use `--code "$(cat file.js)"`
`Permission denied` on rc file during install	Script detects your active shell and skips unrelated rc files. If your shell's rc file is read-only, add PATH manually: `export PATH="$HOME/.local/bin:$PATH"`

rm -f ~/.local/bin/aievals
rm -f ~/.zsh/completions/_aievals

aievals auth logout --yes