BrowserStack AI Evals
Evaluation

Prompts

Fetch versioned prompts and compile Mustache templates across TypeScript, Python, and Java SDKs.

Prompts

Prompts are version-controlled templates stored in BrowserStack AI Evals. The SDK lets you fetch prompts at runtime, compile Mustache variables, and create new prompt versions programmatically.

Prompts are accessed via the client.prompt namespace on an AISDK instance, or the standalone Prompt class (no client needed). The client caches fetched prompts to reduce API calls.

ApproachUsageWhen to use
client.prompt.get() / client.prompt.create()Via an AISDK instanceWhen you already have a client for tracing, datasets, etc.
Prompt.get() / Prompt.create()Static import, no client neededStandalone scripts, quick access from env vars
import { AISDK, Prompt } from '@browserstack/ai-sdk';

const client = new AISDK({
  publicKey: process.env.AISDK_PUBLIC_KEY,
  secretKey: process.env.AISDK_SECRET_KEY,
});

Create a Prompt

Text Prompt

const prompt = await client.prompt.create({
  name: 'summarizer',
  type: 'text',
  prompt: 'Summarize the following in {{language}}: {{text}}',
  labels: ['staging'],
  config: {
    model: 'gpt-4o',
    temperature: 0.3,
  },
});

Chat Prompt

const prompt = await client.prompt.create({
  name: 'qa-assistant',
  type: 'chat',
  prompt: [
    { role: 'system', content: 'You are a helpful assistant for {{domain}}.' },
    { role: 'user', content: '{{userQuestion}}' },
  ],
  labels: ['production'],
});

Fetch a Prompt

// By name (latest version)
const prompt = await client.prompt.get('summarizer');

// By version number
const promptV3 = await client.prompt.get('summarizer', 3);

// By label
const productionPrompt = await client.prompt.get('summarizer', undefined, {
  label: 'production',
});

// Standalone — no client needed (reads AISDK_PUBLIC_KEY / AISDK_SECRET_KEY from env)
const standalonePrompt = await Prompt.get('summarizer');
ParameterTypeDescription
namestringPrompt name as defined in the dashboard
versionnumberSpecific version number. Omit for latest
options.labelstringFetch by label (e.g. 'production', 'staging')
options.type"text" | "chat"Prompt type (defaults to "text")
options.cacheTtlSecondsnumberOverride default cache TTL
options.fallbackstring | ChatMessage[]Fallback value if fetch fails
options.maxRetriesnumberNumber of fetch retries
options.fetchTimeoutMsnumberFetch timeout in milliseconds

Compile a Prompt

Both text and chat prompts support Mustache template compilation via .compile().

Text Prompts

const prompt = await client.prompt.get('summarizer');
// prompt.type === 'text'
// prompt.prompt === "Summarize the following text in {{language}}: {{text}}"

const compiled: string = prompt.compile({
  language: 'French',
  text: 'Paris is the capital of France...',
});

console.log(compiled);
// "Summarize the following text in French: Paris is the capital of France..."

Chat Prompts

const prompt = await client.prompt.get('qa-chat');
// prompt.type === 'chat'

const messages = prompt.compile({
  topic: 'quantum computing',
  userQuestion: 'What is superposition?',
});

// messages is a ChatMessage[] ready to pass to an LLM provider

Using with OpenAI

import { AISDK } from '@browserstack/ai-sdk';
import OpenAI from 'openai';

const client = new AISDK({
  publicKey: process.env.AISDK_PUBLIC_KEY,
  secretKey: process.env.AISDK_SECRET_KEY,
});
const openai = new OpenAI();

const prompt = await client.prompt.get('qa-chat', undefined, {
  label: 'production',
});

const messages = prompt.compile({
  userQuestion: 'What is the boiling point of water?',
});

const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: messages,
});

console.log(response.choices[0].message.content);

Prompt Tools

If a prompt has tools attached in the dashboard, they are available via prompt.tools.

const prompt = await client.prompt.get('agent-prompt');
const tools = prompt.tools.compile();

const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: prompt.compile({ task: 'Search for recent AI news' }),
  tools: tools,
  tool_choice: 'auto',
});

Prompts are accessed via the client.prompt namespace on an AISDK instance, or the standalone Prompt class (no client needed). The client caches fetched prompts to reduce API calls.

ApproachUsageWhen to use
client.prompt.get() / client.prompt.create()Via an AISDK instanceWhen you already have a client for tracing, datasets, etc.
Prompt.get() / Prompt.create()Static class methods, no client neededStandalone scripts, quick access from env vars
import os
from browserstack_ai_sdk import AISDK, Prompt

client = AISDK(
    public_key=os.environ["AISDK_PUBLIC_KEY"],
    secret_key=os.environ["AISDK_SECRET_KEY"],
)

Create a Prompt

Text Prompt

result = client.prompt.create(
    name="summarizer",
    prompt="Summarize the following article in 3 bullet points:\n\n{{article}}",
    type="text",
    labels=["production"],
)
print(result)

Chat Prompt

result = client.prompt.create(
    name="support-assistant",
    type="chat",
    prompt=[
        {"role": "system", "content": "You are a helpful support assistant for {{product}}."},
        {"role": "user", "content": "{{user_message}}"},
    ],
    labels=["staging"],
)

create() Parameters

ParameterTypeDefaultDescription
namestrrequiredUnique prompt name
promptstr | list[dict]requiredTemplate string (text) or messages list (chat)
type"text" | "chat""text"Prompt type
labelslist[str][]Labels (e.g., ["production"])

Fetch a Prompt

# By name (latest version)
prompt_client = client.prompt.get("summarizer")

# By version number
prompt_client = client.prompt.get("summarizer", version=3)

# By label
prompt_client = client.prompt.get("summarizer", label="production")

# With cache control
prompt_client = client.prompt.get(
    "summarizer",
    label="production",
    type="text",
    cache_ttl_seconds=120,
)

# Standalone — no client needed (reads AISDK_PUBLIC_KEY / AISDK_SECRET_KEY from env)
standalone_prompt = Prompt.get("summarizer")

get() Parameters

ParameterTypeDefaultDescription
namestrrequiredPrompt name
type"text" | "chat""text"Prompt type
versionintNonePin to a specific version
labelstr"latest"Pin to a label ("production", "staging", etc.)
cache_ttl_secondsintNoneClient-side cache TTL
fallbackstr | listNoneFallback value if fetch fails

Compile a Prompt

Text Prompts

prompt_client = client.prompt.get("summarizer", label="production")

compiled = prompt_client.compile(
    article="Paris is the capital of France...",
)

print(compiled)
# "Summarize the following article in 3 bullet points:\n\nParis is the capital of France..."

Chat Prompts

prompt_client = client.prompt.get(
    "support-assistant",
    type="chat",
    label="production",
)

messages = prompt_client.compile(
    product="Acme Widget",
    user_message="How do I reset my password?",
)

print(messages)
# [
#   {"role": "system", "content": "You are a helpful support assistant for Acme Widget."},
#   {"role": "user", "content": "How do I reset my password?"},
# ]

Using with OpenAI

import openai

openai_client = openai.OpenAI()

prompt_client = client.prompt.get("support-assistant", type="chat", label="production")
messages = prompt_client.compile(product="Acme Widget", user_message="What are your hours?")

response = openai_client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
)
print(response.choices[0].message.content)

Prompt Versioning

Each call to create() creates a new version. Labels control which version is fetched by get().

# Publish a new version to production
client.prompt.create(
    name="my-prompt",
    prompt="Updated template: {{topic}}",
    labels=["production"],
    commit_message="Improve clarity",
)

# Always fetches the latest "production" version
prompt = client.prompt.get("my-prompt", label="production")

PromptsClient

import com.browserstack.aisdk.TestOps;
import com.browserstack.aisdk.eval.PromptsClient;
import com.browserstack.aisdk.eval.model.PromptResponse;

TestOps sdk = TestOps.fromEnv();
PromptsClient prompts = sdk.prompts();

Create a Prompt

Text Prompt

// Simple text prompt
PromptResponse prompt = prompts.createText(
    "aurora-explainer",
    "You are a science educator. Explain the following in simple terms: {{input}}"
);

// With labels
PromptResponse prompt = prompts.createText(
    "aurora-explainer",
    "You are a science educator. Explain the following in simple terms: {{input}}",
    List.of("production", "v1")
);

Chat Prompt

import java.util.List;
import java.util.Map;

List<Map<String, Object>> messages = List.of(
    Map.of("role", "system", "content", "You are a science educator."),
    Map.of("role", "user", "content", "Explain the following: {{input}}")
);

PromptResponse prompt = prompts.createChat("aurora-explainer-chat", messages);

// With labels
PromptResponse prompt = prompts.createChat(
    "aurora-explainer-chat",
    messages,
    List.of("production")
);

Retrieve a Prompt

// Gets the prompt labeled "production", cached for 60 seconds
PromptResponse prompt = prompts.get("aurora-explainer");

// Specific version
PromptResponse v1 = prompts.get("aurora-explainer", 1, null);

// Specific label
PromptResponse staging = prompts.get("aurora-explainer", null, "staging");

Cache Control

// Custom TTL (120 seconds)
PromptResponse prompt = prompts.get("aurora-explainer", null, "production", 120);

// Bypass cache entirely
PromptResponse fresh = prompts.get("aurora-explainer", null, "production", 0);

The cache uses stale-while-revalidate semantics — expired entries are returned immediately while a background refresh runs.

Use a Prompt

PromptResponse prompt = prompts.get("aurora-explainer");
String template = (String) prompt.getPrompt();

// Simple variable substitution
String userInput = "What causes Northern Lights?";
String finalPrompt = template.replace("{{input}}", userInput);

// Use with OpenAI
var response = openai.chat().completions().create(
    ChatCompletionCreateParams.builder()
        .model("gpt-4o")
        .addMessage(ChatCompletionMessageParam.ofUser(finalPrompt))
        .build()
);

Tracking Prompt Usage in Traces

PromptResponse prompt = prompts.get("aurora-explainer");

var gen = trace.generation(GenerationBody.builder()
    .name("llm-call")
    .model("gpt-4o")
    .promptName(prompt.getName())
    .promptVersion(prompt.getVersion())
    .input(finalPrompt)
    .output(answer)
    .build());

Update Labels

// Promote version 2 to production
PromptResponse updated = prompts.updateLabels("aurora-explainer", 2, List.of("production"));