LLM Connections

Access via Settings → LLM Connections in your project.

LLM Connections store provider API keys at the project level. These keys are used by:

• LLM-as-judge evaluators — the platform calls the LLM on your behalf to score traces
• Playground — interactive prompt testing using your provider credentials
• Default evaluation model — the model selected for evaluations that don't specify a model

Supported Providers

Adding an LLM Connection

1. Go to Settings → LLM Connections
2. Click Add LLM API Key
3. Select your provider from the dropdown
4. Enter the required fields:
   • Display name — a label for this connection (e.g., "OpenAI Production")
   • API key — your provider API key
   • Base URL — for Azure or custom endpoints
   • Extra headers — optional additional headers (e.g., for proxy authentication)
5. Click Save

For Azure OpenAI, you also need to specify the deployment name and API version in the base URL.

To reach a model that isn't publicly accessible (running on localhost, on-prem, or behind a VPN), see Private and self-hosted models.

Private and self-hosted models

If your model isn't publicly reachable, route the connection through a private environment backed by BrowserStack Local. The LLM Connections page has two tabs — LLM API keys and Private Environments.

Create a private environment

On the Private Environments tab, click Create environment. In the dialog:

• BrowserStack Local username (email) — identifies which BrowserStack Local binary handles the tunnel. Pre-filled with your account email; change it to a teammate's if their binary is the one running.
• Name — for example On-prem inference (laptop).
• Description (optional).
• Tunnel ID (optional) — the --local-identifier value you pass when starting the binary.

Run the BrowserStack Local binary on the same machine as your model, then use Test connection to confirm it's reachable and click Create.

./BrowserStackLocal --key <your-access-key> --local-identifier my-local-tunnel

Route a connection through it

When adding an LLM connection on the LLM API keys tab:

• Set API Base URL to your private endpoint, for example http://localhost:11434/v1.
• Under Advanced settings, set Connection to your private environment. The default, Public (direct), calls the provider directly.

You can also link an existing key later — open the environment and click Link existing LLM API key. Connections are grouped into Public connections (direct to the provider) and Private connections (routed through a Local environment); use Unlink from environment to switch a key back to public.

Once a connection is private, the tunnel is used wherever it runs — the playground, LLM-as-judge evaluators, and experiment or dataset runs.

Default Evaluation Model

Select which LLM connection and model to use by default when running evaluators. This setting is project-wide and can be overridden on individual evaluator configurations.

To set the default:

1. Go to Settings → LLM Connections
2. Find the Default Evaluation Model section
3. Select the provider connection and model name
4. Click Save

Security

LLM Connection API keys are:

• Encrypted at rest using AES-256
• Never returned in plaintext after creation — only the key name and creation date are shown
• Scoped to the project — they cannot be used to access other projects

To rotate a key:

1. Add a new connection with the updated key
2. Update any evaluators that reference the old connection
3. Delete the old connection

Permissions