Ingest experiment spans with OpenTelemetry

An experiment runs your application against test data. Each input is an experiment item, optionally with an expected output. Each item corresponds to one trace that shows how your application produced the actual output from that input, with child spans for model calls, tool calls, and other work.

To make experiment data show correctly in Langfuse, model three levels of context in this order:

Configure a BaggageSpanProcessor as described in the Baggage propagation pattern. It copies Baggage entries to each newly created span. The following sections show how each level is represented.

1. Experiment context: baggage shared by all item traces

Create experiment Baggage once. Start every item execution with this Baggage and no active parent span: each execution must remain a separate trace. Do not create an enclosing "experiment" span.

experimentBaggage = baggageFromAttributes({
  "langfuse.experiment.id": experiment.id, // unique per experiment
  "langfuse.experiment.name": experiment.name, // unique per experiment
  "langfuse.experiment.dataset.id": experiment.datasetId, // recommended: Manage datasets in Langfuse
  "langfuse.experiment.description": experiment.description, // optional
  "langfuse.experiment.metadata.prompt_version": "v4", // optional
  "langfuse.environment": "experiment", // optional: separate environment for experiment traffic
})

2. Experiment item root: one trace per item

An experiment item has an input, an expected output, and the actual output from your application. Its trace captures the execution that produces the actual output from that input. Langfuse expects exactly one trace per experiment item.

Langfuse needs a clear root span to identify the item trace and its data. The root's langfuse.observation.input should be the experiment item input, and its langfuse.observation.output should be the actual output. Create the root with the experiment Baggage, then set the expected output and metadata directly on it. Its langfuse.experiment.item.root_observation_id must equal its own OTel spanId. Prefer this span to also be the OTel trace root, with no parent span or Langfuse parent observation.

for each item:
  root = tracer.startSpan(
    "experiment-item",
    withBaggage(ROOT_CONTEXT, experimentBaggage),
  )
  rootId = root.spanContext().spanId

  itemAttributes = {
    "langfuse.experiment.item.id": item.id, // DatasetItem ID when linking a Langfuse-managed dataset
    "langfuse.experiment.item.version": item.version, // optional: Langfuse-managed datasets only
    "langfuse.experiment.item.root_observation_id": rootId,
  }

  root.setAttributes({
    ...itemAttributes,
    "langfuse.experiment.item.expected_output": jsonEncode(item.expectedOutput), // optional
    "langfuse.experiment.item.metadata.country": item.metadata.country, // optional
  })

To link a Langfuse-managed dataset, set the experiment dataset ID to Dataset.id and langfuse.experiment.item.id to the DatasetItem.id returned by Langfuse. If you fetch a specific dataset version, set the item version to the same version timestamp.

When using local data, you can also set the experiment dataset ID and item IDs as local correlation identifiers. Use stable IDs to correlate the same local dataset across repeated experiment runs. Do not set an item version for local data.

3. Item context: baggage shared within one item trace

Before starting child spans, add the item identity and root ID to Baggage. This lets Langfuse link model calls, tool calls, and other work to the experiment item.

itemBaggage = mergeBaggage(
  experimentBaggage,
  baggageFromAttributes(itemAttributes),
)

withContext(withActiveSpan(withBaggage(ROOT_CONTEXT, itemBaggage), root)):
  runInstrumentedTask(item)

root.end()

Choose an environment for experiment runs

Experiment runs often deserve a separate environment such as experiment so they do not mix with production traffic. If the exporter only sends experiment runs, set langfuse.environment=experiment as a resource attribute. If the same exporter also sends production traffic, add langfuse.environment to the experiment Baggage. This applies it to every item trace and span in the experiment. See Environments for details.