EvaluationEvaluators

Evaluators

Create evaluator lists and run inline evaluations across TypeScript, Python, and Java SDKs.

Evaluator lists group one or more evaluation metrics that can be attached to experiments or run inline against LLM outputs.

Build evaluator lists with metrics and parameter configurations.

Run evaluations directly against LLM outputs without an experiment.

Experiment Runs

Start experiment runs, poll for completion, and view results across TypeScript, Python, and Java SDKs.

Evaluator Types

Reference for all evaluator types in TestAI — LLM-as-a-Judge, code-based, aggregate, and tool call evaluators.