BrowserStack AI Evals
Evaluation

Prompts

Fetch versioned prompts and compile Mustache templates across TypeScript, Python, and Java SDKs.

Prompts

Prompts are version-controlled templates stored in BrowserStack AI Evals. The SDK lets you fetch prompts at runtime, compile Mustache variables, and create new prompt versions programmatically.

Fetch a Prompt

Prompts are fetched directly on the AISDK instance. The client caches fetched prompts to reduce API calls.

import { AISDK } from '@browserstack/ai-sdk';

const testOps = new AISDK({
  publicKey: process.env.AISDK_PUBLIC_KEY,
  secretKey: process.env.AISDK_SECRET_KEY,
});

// By name (latest version)
const prompt = await testOps.getPrompt('summarizer');

// By version number
const prompt = await testOps.getPrompt('summarizer', 3);

// By label
const prompt = await testOps.getPrompt('summarizer', undefined, {
  label: 'production',
});
ParameterTypeDescription
namestringPrompt name as defined in the dashboard
versionnumberSpecific version number. Omit for latest
options.labelstringFetch by label (e.g. 'production', 'staging')
options.cacheTtlSecondsnumberOverride default cache TTL

Compile a Prompt

Both text and chat prompts support Mustache template compilation via .compile().

Text Prompts

const prompt = await testOps.getPrompt('summarizer');
// prompt.type === 'text'
// prompt.prompt === "Summarize the following text in {{language}}: {{text}}"

const compiled: string = prompt.compile({
  language: 'French',
  text: 'Paris is the capital of France...',
});

console.log(compiled);
// "Summarize the following text in French: Paris is the capital of France..."

Chat Prompts

const prompt = await testOps.getPrompt('qa-chat');
// prompt.type === 'chat'

const messages = prompt.compile({
  topic: 'quantum computing',
  userQuestion: 'What is superposition?',
});

// messages is a ChatMessage[] ready to pass to an LLM provider

Using with OpenAI

import { AISDK } from '@browserstack/ai-sdk';
import OpenAI from 'openai';

const testOps = new AISDK({
  publicKey: process.env.AISDK_PUBLIC_KEY,
  secretKey: process.env.AISDK_SECRET_KEY,
});
const openai = new OpenAI();

const prompt = await testOps.getPrompt('qa-chat', undefined, {
  label: 'production',
});

const messages = prompt.compile({
  userQuestion: 'What is the boiling point of water?',
});

const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: messages as any,
});

console.log(response.choices[0].message.content);

Prompt Tools

If a prompt has tools attached in the dashboard, they are available via prompt.tools.

const prompt = await testOps.getPrompt('agent-prompt');
const tools = prompt.tools.compile();

const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: prompt.compile({ task: 'Search for recent AI news' }) as any,
  tools: tools as any,
  tool_choice: 'auto',
});

Create a Prompt

Text Prompt

const prompt = await testOps.createPrompt({
  name: 'summarizer',
  type: 'text',
  prompt: 'Summarize the following in {{language}}: {{text}}',
  labels: ['staging'],
  config: {
    model: 'gpt-4o',
    temperature: 0.3,
  },
});

Chat Prompt

const prompt = await testOps.createPrompt({
  name: 'qa-assistant',
  type: 'chat',
  prompt: [
    { role: 'system', content: 'You are a helpful assistant for {{domain}}.' },
    { role: 'user', content: '{{userQuestion}}' },
  ],
  labels: ['production'],
});

Fetch a Prompt

The SDK provides two prompt interfaces:

  • Prompt (standalone class) — static class methods that work without an AISDK instance
  • client.get_prompt() — same API, available on an AISDK instance
import os
from browserstack_ai_sdk import Prompt

# Standalone usage (reads AISDK_PUBLIC_KEY / AISDK_SECRET_KEY from env)
prompt_client = Prompt.get(
    "my-prompt",
    type="text",         # "text" (default) or "chat"
    label="production",  # label or version — defaults to "latest"
)

# Compile with Mustache variables
text = prompt_client.compile(name="Alice", topic="quantum computing")
print(text)

Via AISDK instance

import os
from browserstack_ai_sdk import AISDK

client = AISDK(
    public_key=os.environ["AISDK_PUBLIC_KEY"],
    secret_key=os.environ["AISDK_SECRET_KEY"],
)

prompt_client = client.get_prompt(
    "my-prompt",
    type="text",
    label="production",
)
text = prompt_client.compile(name="Bob")
print(text)

get() Parameters

ParameterTypeDefaultDescription
namestrrequiredPrompt name
type"text" | "chat""text"Prompt type
versionintNonePin to a specific version
labelstr"latest"Pin to a label ("production", "staging", etc.)
cache_ttl_secondsintNoneClient-side cache TTL
fallbackstr | listNoneFallback value if fetch fails

Chat Prompts

import os
from browserstack_ai_sdk import Prompt
import openai

openai_client = openai.OpenAI(api_key=os.environ["OPENAI_API_KEY"])

chat_prompt = Prompt.get("support-assistant", type="chat", label="production")
messages = chat_prompt.compile(user_name="Alice", product="Acme Widget")

response = openai_client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
)
print(response.choices[0].message.content)

Create a Prompt

Text Prompt

import os
from browserstack_ai_sdk import Prompt

result = Prompt.create(
    name="summarizer",
    prompt="Summarize the following article in 3 bullet points:\n\n{{article}}",
    type="text",
    labels=["production"],
    tags=["summarization"],
    model_params={"model": "gpt-4o", "temperature": 0.3},
    commit_message="Initial version",
)
print(result)

Chat Prompt

result = Prompt.create(
    name="support-assistant",
    type="chat",
    prompt=[
        {"role": "system", "content": "You are a helpful support assistant for {{product}}."},
        {"role": "user", "content": "{{user_message}}"},
    ],
    labels=["staging"],
    model_params={"model": "gpt-4o-mini"},
)

List Prompts

from browserstack_ai_sdk import Prompt

prompts = Prompt.list(
    label="production",
    page=1,
    limit=20,
)

for p in prompts:
    print(p["name"], p["version"])

Prompt Versioning

Each call to create() creates a new version. Labels control which version is fetched by get().

# Publish version 3 to production
Prompt.create(
    name="my-prompt",
    prompt="Updated template: {{topic}}",
    labels=["production"],
    commit_message="Improve clarity",
)

# Always fetches the latest "production" version
prompt = Prompt.get("my-prompt", label="production")

PromptsClient

import com.browserstack.aisdk.TestOps;
import com.browserstack.aisdk.eval.PromptsClient;
import com.browserstack.aisdk.eval.model.PromptResponse;

TestOps sdk = TestOps.fromEnv();
PromptsClient prompts = sdk.prompts();

Create a Prompt

Text Prompt

// Simple text prompt
PromptResponse prompt = prompts.createText(
    "aurora-explainer",
    "You are a science educator. Explain the following in simple terms: {{input}}"
);

// With labels
PromptResponse prompt = prompts.createText(
    "aurora-explainer",
    "You are a science educator. Explain the following in simple terms: {{input}}",
    List.of("production", "v1")
);

Chat Prompt

import java.util.List;
import java.util.Map;

List<Map<String, Object>> messages = List.of(
    Map.of("role", "system", "content", "You are a science educator."),
    Map.of("role", "user", "content", "Explain the following: {{input}}")
);

PromptResponse prompt = prompts.createChat("aurora-explainer-chat", messages);

// With labels
PromptResponse prompt = prompts.createChat(
    "aurora-explainer-chat",
    messages,
    List.of("production")
);

Retrieve a Prompt

// Gets the prompt labeled "production", cached for 60 seconds
PromptResponse prompt = prompts.get("aurora-explainer");

// Specific version
PromptResponse v1 = prompts.get("aurora-explainer", 1, null);

// Specific label
PromptResponse staging = prompts.get("aurora-explainer", null, "staging");

Cache Control

// Custom TTL (120 seconds)
PromptResponse prompt = prompts.get("aurora-explainer", null, "production", 120);

// Bypass cache entirely
PromptResponse fresh = prompts.get("aurora-explainer", null, "production", 0);

The cache uses stale-while-revalidate semantics — expired entries are returned immediately while a background refresh runs.

Use a Prompt

PromptResponse prompt = prompts.get("aurora-explainer");
String template = (String) prompt.getPrompt();

// Simple variable substitution
String userInput = "What causes Northern Lights?";
String finalPrompt = template.replace("{{input}}", userInput);

// Use with OpenAI
var response = openai.chat().completions().create(
    ChatCompletionCreateParams.builder()
        .model("gpt-4o")
        .addMessage(ChatCompletionMessageParam.ofUser(finalPrompt))
        .build()
);

Tracking Prompt Usage in Traces

PromptResponse prompt = prompts.get("aurora-explainer");

var gen = trace.generation(GenerationBody.builder()
    .name("llm-call")
    .model("gpt-4o")
    .promptName(prompt.getName())
    .promptVersion(prompt.getVersion())
    .input(finalPrompt)
    .output(answer)
    .build());

List Prompts

// Default (page 1, limit 50)
ListPromptsResponse list = prompts.list();

// With filters
ListPromptsResponse list = prompts.list(1, 20, "aurora", "production", null);

Update Labels

// Promote version 2 to production
PromptResponse updated = prompts.updateLabels("aurora-explainer", 2, List.of("production"));