BrowserStack AI Evals
EvaluationTools

Manage Tools

Create, get, list, and update LLM tools via SDK or REST API.

Manage Tools

The Tool class provides static methods that read credentials from env vars. You can also access the same methods via an AISDK client instance (client.tools.*) — both work identically.

Create a Tool

import { Tool } from '@browserstack/ai-sdk';

const toolInstance = await Tool.create({
  name: 'search_web',
  description: 'Search the web for up-to-date information on a topic.',
  parameters: {
    type: 'object',
    properties: {
      query: { type: 'string', description: 'The search query.' },
      maxResults: { type: 'integer', description: 'Maximum results to return.', default: 5 },
    },
    required: ['query'],
  },
  isRunnable: true,
  labels: ['production'],
});

console.log(toolInstance.id, toolInstance.version);

create() Parameters

ParameterTypeDefaultDescription
namestringrequiredTool name (alphanumeric, hyphens, underscores)
descriptionstringrequiredHuman-readable description
parametersobject{}JSON Schema for tool parameters
sampleOutputanynullExample output for documentation
isRunnablebooleanfalseWhether the tool can be executed
labelsstring[][]Labels (e.g., ["production"])
commitMessagestringnullVersion commit message

Get a Tool

The second argument selects the provider format ("openai" by default). Supported providers: "openai", "anthropic", "gemini".

// Get latest version (OpenAI format, default)
const result = await Tool.get('search_web');

// Get in Anthropic format
const result = await Tool.get('search_web', 'anthropic');

// Get in Gemini format
const result = await Tool.get('search_web', 'gemini');

// Get by version
const result = await Tool.get('search_web', undefined, { version: 2 });

// Get by label
const result = await Tool.get('search_web', undefined, { label: 'production' });

// always get fresh tool
const result = await Tool.get('search_web', undefined, { cacheTtlSeconds: 0 });

get() Parameters

ParameterTypeDefaultDescription
namestringrequiredTool name
provider"openai" | "anthropic" | "gemini""openai"Provider format for the returned tool
options.versionnumberPin to a specific version
options.labelstring"latest"Fetch by label (e.g. "production"). The highest version always carries the latest label.
options.cacheTtlSecondsnumberSDK defaultCache TTL in seconds. Set to 0 to bypass the cache for this call (still fetches fresh, does not disable caching globally).

Compile a Tool

result.compile() returns the tool in the provider-specific format requested during get(). By default that's OpenAI; if you passed "anthropic" or "gemini" to get(), the compiled output matches that provider's schema.

const result = await Tool.get('search_web');  // OpenAI format (default)
const compiled = result.compile();

const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'What happened in the news today?' }],
  tools: [compiled],
  tool_choice: 'auto',
});

See Variable Substitution for compiling with runtime variables.

List Tools

const listing = await Tool.list(
  20,             // limit
  undefined,      // cursor for pagination
  'production'    // optional label filter
);

for (const tool of listing.data) {
  console.log(tool.name, 'v' + tool.version);
}

if (listing.meta.nextCursor) {
  const next = await Tool.list(20, listing.meta.nextCursor);
}

list() Parameters

ParameterTypeDefaultDescription
limitnumber50Items per page (1–100)
cursorstringCursor for next page (from previous response)
labelstringFilter by label

Update a Tool

update() creates a new version of the tool with the given name. You can't rename a tool — passing a different name creates a new tool (not a rename of the existing one). The new version automatically receives the latest label.

const updated = await Tool.update({
  name: 'search_web',
  description: 'Search the web for current information.',
  parameters: {
    type: 'object',
    properties: {
      query: { type: 'string' },
      maxResults: { type: 'integer', default: 10 },
      language: { type: 'string', default: 'en' },
    },
    required: ['query'],
  },
  labels: ['production'],
  commitMessage: 'Add language parameter',
});

console.log('New version:', updated.version);

update() Parameters

ParameterTypeDefaultDescription
namestringrequiredMust match an existing tool name. Passing a new name creates a new tool.
descriptionstringrequiredHuman-readable description
parametersobject{}JSON Schema for tool parameters
sampleOutputanynullExample output for documentation
isRunnablebooleanfalseWhether the tool can be executed
labelsstring[][]Labels applied to the new version
commitMessagestringnullVersion commit message

Via AISDK Instance

All methods are also available on a client instance. Useful if you already have an AISDK client for tracing or other APIs:

import { AISDK } from '@browserstack/ai-sdk';

const client = new AISDK({
  publicKey: process.env.AISDK_PUBLIC_KEY,
  secretKey: process.env.AISDK_SECRET_KEY,
});

const result = await client.tools.get('search_web');
const compiled = result.compile();

Create a Tool

import os
from browserstack_ai_sdk import Tool

tool_instance = Tool.create(
    name="get_weather",
    description="Get the current weather for a given location.",
    parameters={
        "type": "object",
        "properties": {
            "location": {
                "type": "string",
                "description": "City and state, e.g. 'San Francisco, CA'",
            },
            "unit": {
                "type": "string",
                "enum": ["celsius", "fahrenheit"],
            },
        },
        "required": ["location"],
    },
    labels=["production"],
    commit_message="Initial version",
)

print(tool_instance.name)     # "get_weather"
print(tool_instance.id)       # registry ID
print(tool_instance.version)  # 1

create() Parameters

ParameterTypeDefaultDescription
namestrrequiredTool name (alphanumeric, hyphens, underscores)
descriptionstrrequiredHuman-readable description
parametersdict{}JSON Schema for tool parameters
sample_outputdictNoneExample output for documentation
is_runnableboolNoneWhether the tool can be executed
labelslist[str]NoneLabels (e.g., ["production"])
commit_messagestrNoneVersion commit message

Get a Tool

from browserstack_ai_sdk import Tool

# Get with default provider (OpenAI format)
tool = Tool.get("get_weather")

# Get for a specific provider
tool = Tool.get("get_weather", provider="openai")

# Get a specific version
tool = Tool.get("get_weather", version=2)

# Get by label
tool = Tool.get("get_weather", label="production")

# Always fetches fresh
tool = Tool.get("get_weather", cache_ttl_seconds=0)

get() Parameters

ParameterTypeDefaultDescription
namestrrequiredTool name
provider"openai" | "anthropic" | "gemini""openai"Provider format for the returned tool
versionintNonePin to a specific version
labelstr"latest"Fetch by label. The highest version always carries the latest label.
cache_ttl_secondsintSDK defaultCache TTL in seconds. Set to 0 to bypass the cache for this call (still fetches fresh, does not disable caching globally).

Compile a Tool

result.compile() returns the tool in the provider-specific format requested during get(). By default that's OpenAI; if you passed provider="anthropic" or provider="gemini" to get(), the compiled output matches that provider's schema.

from browserstack_ai_sdk import Tool
import openai

tool = Tool.get("search_products", provider="openai")  # OpenAI format
compiled = tool.compile()

client = openai.OpenAI()
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Find me a good laptop."}],
    tools=[compiled],
    tool_choice="auto",
)

See Variable Substitution for compiling with runtime variables.

Update a Tool

update() creates a new version of the tool with the given name. You can't rename a tool — passing a different name creates a new tool (not a rename of the existing one). The new version automatically receives the latest label.

from browserstack_ai_sdk import Tool

updated = Tool.update(
    name="get_weather",
    description="Get the current weather and 5-day forecast for a location.",
    parameters={
        "type": "object",
        "properties": {
            "location": {"type": "string"},
            "days": {"type": "integer", "minimum": 1, "maximum": 5},
        },
        "required": ["location"],
    },
    labels=["production"],
    commit_message="Add forecast support",
)
print(updated.version)  # 2

update() Parameters

ParameterTypeDefaultDescription
namestrrequiredMust match an existing tool name. Passing a new name creates a new tool.
descriptionstrrequiredHuman-readable description
parametersdict{}JSON Schema for tool parameters
sample_outputdictNoneExample output for documentation
is_runnableboolNoneWhether the tool can be executed
labelslist[str]NoneLabels applied to the new version
commit_messagestrNoneVersion commit message

List Tools

from browserstack_ai_sdk import Tool

response = Tool.list(limit=20)
for tool_data in response.get("data", []):
    print(tool_data["name"])

# Cursor-based pagination
response = Tool.list(limit=20, cursor=response.get("cursor"))

# Filter by label
response = Tool.list(label="production")

list() Parameters

ParameterTypeDefaultDescription
limitint50Items per page (1–100)
cursorstrNoneCursor for next page (from previous response)
labelstrNoneFilter by label

ToolList (from Prompts)

When you fetch a prompt that has an attached tool list, prompt.tools is a ToolList:

from browserstack_ai_sdk import Prompt
import openai

prompt = Prompt.get("my-prompt-with-tools", type="chat")

# Compile all tools at once
compiled_tools = prompt.tools.compile(strings={"env": "production"})

client = openai.OpenAI()
response = client.chat.completions.create(
    model="gpt-4o",
    messages=prompt.compile(user_message="What tools do you have?"),
    tools=compiled_tools,
    tool_choice="auto",
)

Via AISDK Instance

import os
from browserstack_ai_sdk import AISDK

client = AISDK(
    public_key=os.environ["AISDK_PUBLIC_KEY"],
    secret_key=os.environ["AISDK_SECRET_KEY"],
)

tool = client.tools.get("get_weather", provider="openai")
compiled = tool.compile()

Tool management via the Java SDK is coming soon. Track progress in our SDK changelog.


Dashboard

Use the BrowserStack AI Evals dashboard to create and manage tool definitions without writing code.

Create a Tool

Navigate to Tools in the left sidebar.

Click Create Tool to open the tool editor.

Fill in the Name and Description fields. The name must be alphanumeric with hyphens and underscores only (e.g. search_web).

Edit the Parameters JSON schema in the editor. This follows JSON Schema syntax — define property names, types, descriptions, and which fields are required.

Optionally set Labels (e.g. production, staging) and a Commit message to track what changed.

Click Save. Each save creates a new immutable version. The latest label always points to the most recent version.

Attach Tools to a Prompt

Go to Prompts in left side bar and open a prompt.

Scroll to the Tools button and click Create New Tool or choose an existing tool

Search for the tool by name, then select the version or label to attach.

Save the prompt. When the prompt is fetched via SDK, prompt.tools is a ToolList that can be compiled and passed directly to an LLM API call.