Tools

Tools are versioned LLM function definitions stored in BrowserStack AI Evals. The SDK lets you create, fetch, compile, and update tools programmatically, then pass them directly to LLM API calls.

Setup

import { AISDK } from '@browserstack/ai-sdk';

const testOps = new AISDK({
  publicKey: process.env.AISDK_PUBLIC_KEY,
  secretKey: process.env.AISDK_SECRET_KEY,
});

const tools = testOps.tools;

Create a Tool

const toolInstance = await tools.create({
  name: 'search_web',
  description: 'Search the web for up-to-date information on a topic.',
  parameters: {
    type: 'object',
    properties: {
      query: { type: 'string', description: 'The search query.' },
      maxResults: { type: 'integer', description: 'Maximum results to return.', default: 5 },
    },
    required: ['query'],
  },
  isRunnable: true,
  labels: ['production'],
});

console.log(toolInstance.id, toolInstance.version);

Get and Compile a Tool

The second argument selects the provider format ("openai" by default). Supported providers: "openai", "anthropic", "gemini".

// Get latest version (OpenAI format, default)
const result = await tools.get('search_web');

// Get in Anthropic format
const result = await tools.get('search_web', 'anthropic');

// Get by version
const result = await tools.get('search_web', undefined, { version: 2 });

// Get by label
const result = await tools.get('search_web', undefined, { label: 'production' });

ProviderToolResult.compile() returns a provider-agnostic tool definition ready to pass to an LLM SDK:

const result = await tools.get('search_web');
const compiled = result.compile();

const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'What happened in the news today?' }],
  tools: [compiled as any],
  tool_choice: 'auto',
});

List Tools

const listing = await tools.list(
  20,             // limit
  undefined,      // cursor for pagination
  'production'    // optional label filter
);

for (const tool of listing.data) {
  console.log(tool.name, 'v' + tool.version);
}

if (listing.meta.nextCursor) {
  const next = await tools.list(20, listing.meta.nextCursor);
}

Update a Tool

Updating creates a new version:

const updated = await tools.update({
  name: 'search_web',
  description: 'Search the web for current information.',
  parameters: {
    type: 'object',
    properties: {
      query: { type: 'string' },
      maxResults: { type: 'integer', default: 10 },
      language: { type: 'string', default: 'en' },
    },
    required: ['query'],
  },
  labels: ['production'],
  commitMessage: 'Add language parameter',
});

console.log('New version:', updated.version);

Static Methods

All methods are also available as static calls without an instance:

import { Tool } from '@browserstack/ai-sdk';

const toolInstance = await Tool.create({ name: 'my-tool', description: '...' });
const result = await Tool.get('my-tool');
const listing = await Tool.list(10);

Create a Tool

import os
from browserstack_ai_sdk import Tool

tool_instance = Tool.create(
    name="get_weather",
    description="Get the current weather for a given location.",
    parameters={
        "type": "object",
        "properties": {
            "location": {
                "type": "string",
                "description": "City and state, e.g. 'San Francisco, CA'",
            },
            "unit": {
                "type": "string",
                "enum": ["celsius", "fahrenheit"],
            },
        },
        "required": ["location"],
    },
    labels=["production"],
    commit_message="Initial version",
)

print(tool_instance.name)     # "get_weather"
print(tool_instance.id)       # registry ID
print(tool_instance.version)  # 1

`create()` Parameters

Parameter	Type	Default	Description
`name`	`str`	required	Tool name (alphanumeric, hyphens, underscores)
`description`	`str`	required	Human-readable description
`parameters`	`dict`	`{}`	JSON Schema for tool parameters
`sample_output`	`dict`	`None`	Example output for documentation
`is_runnable`	`bool`	`None`	Whether the tool can be executed
`labels`	`list[str]`	`None`	Labels (e.g., `["production"]`)
`commit_message`	`str`	`None`	Version commit message

Fetch and Compile a Tool

from browserstack_ai_sdk import Tool

# Fetch with default provider (OpenAI format)
tool = Tool.get("get_weather")

# Fetch for a specific provider
tool = Tool.get("get_weather", provider="openai")

# Fetch a specific version
tool = Tool.get("get_weather", version=2)

# Fetch by label
tool = Tool.get("get_weather", label="production")

ProviderToolResult.compile() resolves Mustache placeholders and strips internal metadata:

from browserstack_ai_sdk import Tool
import openai

tool = Tool.get("search_products", provider="openai")
compiled = tool.compile()

client = openai.OpenAI()
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Find me a good laptop."}],
    tools=[compiled],
    tool_choice="auto",
)

List and Update Tools

from browserstack_ai_sdk import Tool

# List tools
response = Tool.list(limit=20)
for tool_data in response.get("data", []):
    print(tool_data["name"])

# Filter by label
response = Tool.list(label="production")

# Update (creates a new version)
updated = Tool.update(
    name="get_weather",
    description="Get the current weather and 5-day forecast for a location.",
    parameters={
        "type": "object",
        "properties": {
            "location": {"type": "string"},
            "days": {"type": "integer", "minimum": 1, "maximum": 5},
        },
        "required": ["location"],
    },
    labels=["production"],
    commit_message="Add forecast support",
)
print(updated.version)  # 2

ToolList (from Prompts)

When you fetch a prompt that has an attached tool list, prompt.tools is a ToolList:

from browserstack_ai_sdk import Prompt
import openai

prompt = Prompt.get("my-prompt-with-tools", type="chat")

compiled_tools = prompt.tools.compile(strings={"env": "production"})

client = openai.OpenAI()
response = client.chat.completions.create(
    model="gpt-4o",
    messages=prompt.compile(user_message="What tools do you have?"),
    tools=compiled_tools,
    tool_choice="auto",
)

Via AISDK Instance

import os
from browserstack_ai_sdk import AISDK

client = AISDK(
    public_key=os.environ["AISDK_PUBLIC_KEY"],
    secret_key=os.environ["AISDK_SECRET_KEY"],
)

tool = client.tools.get("get_weather", provider="openai")
compiled = tool.compile()

Setup

import com.browserstack.aisdk.AISDK;
import com.browserstack.aisdk.eval.ToolsClient;

AISDK sdk = AISDK.fromEnv();
ToolsClient tools = sdk.tools();

Create a Tool

import com.browserstack.aisdk.eval.model.CreateToolRequest;
import com.browserstack.aisdk.eval.model.ToolResponse;

import java.util.List;
import java.util.Map;

ToolResponse created = tools.create(
    CreateToolRequest.builder("search_web")
        .description("Search the web for up-to-date information on a topic.")
        .parameters(Map.of(
            "type", "object",
            "properties", Map.of(
                "query", Map.of("type", "string", "description", "The search query."),
                "maxResults", Map.of("type", "integer", "description", "Maximum results to return.", "default", 5)),
            "required", List.of("query")))
        .isRunnable(true)
        .labels(List.of("production"))
        .commitMessage("Initial version")
        .build());

System.out.println(created.getId());       // registry ID
System.out.println(created.getName());     // "search_web"
System.out.println(created.getVersion());  // 1

`CreateToolRequest.builder()` Options

Builder method	Type	Description
`builder(name)`	`String`	Tool name (alphanumeric, hyphens, underscores) — required
`.description(...)`	`String`	Human-readable description
`.parameters(...)`	`Map<String, Object>`	JSON Schema for tool parameters
`.strict(...)`	`boolean`	Enforce strict schema validation
`.sampleOutput(...)`	`Map<String, Object>`	Example output for documentation
`.isRunnable(...)`	`boolean`	Whether the tool can be executed
`.labels(...)`	`List<String>`	Labels (e.g. `List.of("production")`)
`.commitMessage(...)`	`String`	Version commit message

Get and Compile a Tool

The second argument selects the provider format ("openai" by default). Supported providers: "openai", "anthropic", "gemini".

import com.browserstack.aisdk.eval.model.ProviderToolResponse;

// Get latest version (OpenAI format, default)
ProviderToolResponse fetched = tools.get("search_web");

// Get in Anthropic format
ProviderToolResponse anthropic = tools.get("search_web", "anthropic");

// Get a specific version (version and label are mutually exclusive)
ProviderToolResponse pinned = tools.get("search_web", "openai", 2, null);

// Get by label
ProviderToolResponse labelled = tools.get("search_web", "openai", null, "production");

ProviderToolResponse.compile() strips registry metadata and returns a provider-ready map you can pass directly to an LLM SDK:

Map<String, Object> compiled = fetched.compile();
// Pass `compiled` into the tools array of your LLM provider call.

List Tools

import com.browserstack.aisdk.eval.model.ListToolsResponse;
import com.browserstack.aisdk.eval.model.ToolResponse;

ListToolsResponse listing = tools.list(
    20,             // limit
    null,           // cursor for pagination
    "production");  // optional label filter

for (ToolResponse tool : listing.getData()) {
    System.out.println(tool.getName() + " v" + tool.getVersion());
}

String nextCursor = listing.getMeta().getNextCursor();
if (nextCursor != null) {
    ListToolsResponse next = tools.list(20, nextCursor, null);
}

Update a Tool

Updating posts a new version. UpdateToolRequest.builder(name) returns the same builder as CreateToolRequest:

import com.browserstack.aisdk.eval.model.UpdateToolRequest;

ToolResponse updated = tools.update(
    UpdateToolRequest.builder("search_web")
        .description("Search the web for current information.")
        .parameters(Map.of(
            "type", "object",
            "properties", Map.of(
                "query", Map.of("type", "string"),
                "maxResults", Map.of("type", "integer", "default", 10),
                "language", Map.of("type", "string", "default", "en")),
            "required", List.of("query")))
        .labels(List.of("production"))
        .commitMessage("Add language parameter")
        .build());

System.out.println("New version: " + updated.getVersion());  // 2

Compile with Variable Substitution

compile(ToolCompileOptions) resolves {{variable}} placeholders inside the tool's parameters. string(...) substitutes into inline placeholders (Mustache, coerced to String); object(...) replaces a sole placeholder (where the entire value is exactly "{{key}}") with a typed value, preserving Map / List / Number / Boolean shapes.

import com.browserstack.aisdk.eval.model.ToolCompileOptions;

ProviderToolResponse tool = tools.get("search_web");

Map<String, Object> compiled = tool.compile(
    ToolCompileOptions.builder()
        .string("env", "production")
        .object("defaults", Map.of("maxResults", 10))
        .build());

ToolResponse.compile(ToolCompileOptions) performs the same substitution on a created/updated tool.

Attach a Tool to a Prompt

Pass a List<ToolRef> when creating a prompt. Each ToolRef pins a tool by name, optionally by version or label (mutually exclusive); when fetched back, prompt.getTools() returns a ToolList.

import com.browserstack.aisdk.eval.PromptsClient;
import com.browserstack.aisdk.eval.model.PromptResponse;
import com.browserstack.aisdk.eval.model.ToolList;
import com.browserstack.aisdk.eval.model.ToolRef;

PromptsClient prompts = sdk.prompts();

PromptResponse prompt = prompts.createText(
    "research-assistant",
    "You are a research assistant. Use the available tools to answer questions.",
    List.of("production"),
    List.of(
        ToolRef.of("search_web"),                  // latest version
        ToolRef.ofVersion("summarize", 2),         // pin to version 2
        ToolRef.ofLabel("translate", "production")  // pin to a label
    ));

// Fetch the prompt back and compile its attached tools
PromptResponse fetched = prompts.get("research-assistant");
ToolList toolList = fetched.getTools();
List<Map<String, Object>> compiledTools = toolList.compile(
    ToolCompileOptions.builder().string("env", "production").build());

Shutdown

sdk.shutdown();

Tools

On this page