LLM Connections

Access via Settings → LLM Connections in your project.

LLM Connections store provider API keys at the project level. These keys are used by:

• LLM-as-judge evaluators — the platform calls the LLM on your behalf to score traces
• Playground — interactive prompt testing using your provider credentials
• Default evaluation model — the model selected for evaluations that don't specify a model

Supported Providers

Adding an LLM Connection

1. Go to Settings → LLM Connections
2. Click Add LLM API Key
3. Select your provider from the dropdown
4. Enter the required fields:
   • Display name — a label for this connection (e.g., "OpenAI Production")
   • API key — your provider API key
   • Base URL — for Azure or custom endpoints
   • Extra headers — optional additional headers (e.g., for proxy authentication)
5. Click Save

For Azure OpenAI, you also need to specify the deployment name and API version in the base URL.

Default Evaluation Model

Select which LLM connection and model to use by default when running evaluators. This setting is project-wide and can be overridden on individual evaluator configurations.

To set the default:

1. Go to Settings → LLM Connections
2. Find the Default Evaluation Model section
3. Select the provider connection and model name
4. Click Save

Security

LLM Connection API keys are:

• Encrypted at rest using AES-256
• Never returned in plaintext after creation — only the key name and creation date are shown
• Scoped to the project — they cannot be used to access other projects

To rotate a key:

1. Add a new connection with the updated key
2. Update any evaluators that reference the old connection
3. Delete the old connection

Permissions