Shared Views

Secret links let you share an annotation queue with external reviewers — people outside your project — without requiring them to log in. Reviewers access a scoped, read-only view where they can only submit scores.

Creating a Secret Link

Open the queue settings

Go to the queue detail page and open Share / Secret Links.

Create a new link

Click + New Link and configure:

Option	Description
Expiry	How long the link is valid — 1 to 7 days from creation
Layouts	Which custom view layouts external reviewers can see
Default layout	Which layout loads by default
Run filter	Optionally restrict the link to items from specific runs

Copy the generated URL (format: https://your-domain.com/s/{token}) and share it with reviewers. The token is a 64-character hex string.

What External Reviewers Can Do

Action	Allowed
Browse queue items	Yes
View trace details	Yes
Submit scores for pending items	Yes
Update scores for completed items	Yes
View analytics and run progress	Yes
Create or delete items	No
Modify queue configuration	No

Secret links expire after the configured period (maximum 7 days). After expiry, the link returns a 401 error. Create a new link to continue sharing. Scores submitted via secret links are stored permanently even after the link expires.

Security Considerations

Scores submitted via secret links do not capture a user ID — they appear as anonymous annotations.
Anyone with the link URL can access the queue during the validity window. Treat the link as a shared secret.
Use run filters to restrict external reviewers to a specific batch of items rather than the entire queue.

Secret Link API Endpoints

All secret link endpoints use a token query parameter and do not require project API credentials.

Get Queue Info and Progress

curl "https://evals-api.browserstack.com/api/public/secret-links/annotation-queues?token={token}"

Response:

{
  "queue": {
    "id": "queue-uuid-1",
    "name": "Production Quality Review",
    "description": "...",
    "instructions": "Score each response on relevance and accuracy.",
    "projectId": "proj-uuid-1",
    "createdAt": "2026-04-07T10:00:00.000Z"
  },
  "runs": [
    {
      "id": "run-uuid-1",
      "name": "round-1",
      "createdAt": "2026-04-07T10:00:00.000Z",
      "totalItems": 50,
      "pendingItems": 12,
      "completedItems": 38
    }
  ],
  "stats": {
    "totalItems": 50,
    "pendingItems": 12,
    "completedItems": 38,
    "completionPercentage": 76
  },
  "scoreConfigs": [{ "id": "config-uuid-1", "name": "Relevance", "dataType": "NUMERIC" }],
  "config": { "defaultLayoutId": "layout-uuid-1", "runIds": ["run-uuid-1"] }
}

List Items

curl "https://evals-api.browserstack.com/api/public/secret-links/annotation-queues/items?token={token}&page=1&limit=50"

Submit a Score

curl -X POST "https://evals-api.browserstack.com/api/public/secret-links/annotation-queues/scores?token={token}" \
  -H "Content-Type: application/json" \
  -d '{
    "itemId": "item-uuid-1",
    "scoreConfigId": "config-uuid-1",
    "value": 0.9,
    "comment": "Response is highly relevant and accurate."
  }'

Field	Type	Required	Description
`itemId`	`string`	Yes	The queue item to score
`scoreConfigId`	`string`	Yes	Which score config to submit for
`value`	`number`	Numeric configs	Score value (must be within min/max)
`stringValue`	`string`	Categorical/text configs	Selected category label or free text
`comment`	`string`	No	Optional reviewer comment

Update an Existing Score

curl -X PUT "https://evals-api.browserstack.com/api/public/secret-links/annotation-queues/scores?token={token}" \
  -H "Content-Type: application/json" \
  -d '{
    "itemId": "item-uuid-1",
    "scoreConfigId": "config-uuid-1",
    "value": 0.75,
    "comment": "Revised after re-reading the reference."
  }'

Get Analytics

curl "https://evals-api.browserstack.com/api/public/secret-links/annotation-queues/analytics?token={token}"

Response:

{
  "runs": [
    {
      "runId": "run-uuid-1",
      "runName": "round-1",
      "createdAt": "2026-04-07T10:00:00.000Z",
      "pendingCount": 12,
      "completedCount": 38,
      "totalCount": 50,
      "scoreAverages": { "Relevance": 0.87 },
      "scoreDistributions": { "Quality": { "PASS": 30, "FAIL": 8 } },
      "scoreMetadata": [
        { "name": "Relevance", "dataType": "NUMERIC" },
        { "name": "Quality", "dataType": "CATEGORICAL" }
      ]
    }
  ],
  "scoreNames": {
    "numeric": ["Relevance"],
    "categorical": ["Quality"]
  }
}

Get View Layouts

curl "https://evals-api.browserstack.com/api/public/secret-links/annotation-queues/view-layouts?token={token}"

Dashboard Workflow

End-to-end walkthrough for setting up and completing a human review cycle.

Navigate to Annotation Queues

Open your project and click Annotation Queues in the left sidebar.

Create a queue

Click + New Queue. Enter a name and description. In the Instructions field, write clear guidance for reviewers (e.g. "Score each response on a scale of 0–1 for relevance to the user question."). Select the score configs to use.

Add items to the queue

Add traces to the queue manually from the trace detail page (Add to Queue button), or programmatically via the API. Items enter the queue in PENDING state.

Use Automation Rules to automatically route traces matching certain conditions into an annotation queue — no manual selection required.

Configure score rubrics

Go to Settings → Score Configurations and confirm the score configs attached to the queue are correctly defined. For categorical scores, ensure the labels and their numeric values accurately represent your rubric.

Review items

Open the queue and click Start Review. For each item:

Read the trace content in the detail panel.
Fill in the score fields (numeric values, category selections, or text).
Click Submit to mark the item as completed and move to the next.

Items are temporarily locked to the active reviewer (5-minute timeout) to prevent concurrent edits.

Track progress

Return to the queue overview to see the completion percentage per run. The Scores tab shows aggregate analytics: score averages, distributions, and item-level drill-down.

If external reviewers will participate, create a secret link via Share → New Link, configure the expiry and run filter, and send the link to reviewers. Their scores merge with internal annotations in the same queue.

Shared Views

On this page