Async Tracing

Overview

Async Tracing is used when the actual output or tool calls for a dataset run item should be sourced from traces, rather than being provided directly in the experiment run payload.

Flow

Experiment Run → Trigger API → AI SDK sends traces to platform → All dataset run item traces received → Evaluation triggered

How It Works

When registering an API (through swagger yaml), the platform checks for a specific configuration key in the response: AsyncTraceObservationName.

Example:

responses:
  '200':
    description: Weather information retrieved successfully
    content:
      application/json:
        schema:
          type: object
          properties:
            AsyncTraceObservationName:
              type: string
              description: Observation name for tracing
              example: "bedrock.generation"

If this key is present, the following HTTP headers are injected into the outgoing API call:

SDK Integration

Auto Tracing

If the AI SDK is configured for auto tracing, it will automatically capture the X-Reference-Id and X-Environment headers and trigger the downstream processing task.

Manual Tracing

If using manual tracing, pass the X-Reference-Id value as refId in the trace metadata:

client.trace({
  // ...
  environment: "<X-Environment>",
  metadata: {
    refId: "<X-Reference-Id>",
  },
  // ...
});

client.trace(
    # ...
    environment="<X-Environment>",
    metadata={
        "refId": "<X-Reference-Id>",
    },
    # ...
)