Tracing

Beta. The Langfuse AI Gateway is in public beta and can still change.

The gateway records every model call it routes as a Langfuse generation. Gateway traces are regular Langfuse traces, so you can filter, evaluate, and chart them like traces from the SDKs.

Ingestion modes

There are different ingestion modes, which gateway managers set for the whole organization in Organization settings → AI Gateway → Setup:

Mode	Recorded	Use when
`FULL`	Everything `USAGE` records, plus the full input (system prompt, tool definitions, messages, tool calls and results) and the streamed output (text, reasoning or thinking, and tool calls with parsed arguments)	You want to debug, evaluate, or build datasets from real prompts and responses
`USAGE` (default)	The fields listed below, without input or output	You need cost, usage, and an audit trail, but prompts and responses must not be stored in Langfuse
`NONE` (self-hosted only)	Nothing. The gateway only routes requests.	You only want to hand out gateway keys instead of provider keys, without any traces

Full list of fields recorded in FULL and USAGE mode

the model (the dated response model when the provider returns one), matched to model prices, and scalar model parameters such as temperature (FULL also records structured options such as response_format)
token usage, including cached and reasoning tokens, plus cost, latency, and time to first token
level ERROR with the provider's status and message for failed calls, and WARNING for client cancellations
sessions, grouped automatically from each coding agent's own session ID
the trace user, which depends on the key type
the gateway key, its key metadata, and the provider credential in metadata, under langfuse.gateway.api_key.* and langfuse.gateway.connection.*

The benefit of having the FULL ingestion mode is that you can observe and improve the quality of the agent's responses. For example, you can run LLM-as-a-Judge evaluations on production traffic, review calls in annotation queues, or turn real prompts into datasets to improve your setup.

Add tags, metadata, an environment, a session, or a trace name with optional request headers. For how coding agent calls are grouped into traces and sessions, see the relevant coding agent guide.

Request headers

Any client can send these optional headers to enrich the traces the gateway records. They work in both FULL and USAGE ingestion modes. Each coding agent guide shows where to configure them.

Headers

Header	Format	Effect
`langfuse-user-id`	string	Trace user. Only used with service account keys: user keys always record the key owner. See trace user.
`langfuse-session-id`	string	Overrides the agent's own session.
`langfuse-environment`	string	Environment of the trace. Invalid values fall back to `default`.
`langfuse-tags`	comma-separated	Tags, trimmed and deduplicated.
`langfuse-metadata`	comma-separated `key:value`	Top-level metadata. Splits at the first colon, values stay strings. Percent-encode literal commas (`%2C`). For metadata clients can't change, use key metadata.
`langfuse-trace-name`	string	Overrides the default trace name (the agent name).

Example:

langfuse-user-id: jane@example.com
langfuse-environment: ci
langfuse-tags: claude-code,platform-team
langfuse-metadata: repo:langfuse-docs,branch:main

Rules

Do not set a static langfuse-session-id or langfuse-trace-name in a config file. A static session ID merges all your sessions into one, and a static trace name renames every trace. Set them per run from an environment variable, and only when you want that effect, for example in scripted or CI runs.
Explicit headers take precedence over values the gateway infers from the agent. Empty values are ignored.
langfuse-environment must match the environment format: lowercase letters, numbers, - and _, at most 40 characters, and not starting with langfuse. Any other value is recorded as default, and the request still succeeds.
Header values are percent-decoded. Encode a literal % as %25, and a comma inside a tag or metadata value as %2C.
Invalid values are dropped without failing the request.

The gateway ignores all context headers of a request, including traceparent and baggage, when they add up to more than 8 KiB. Each value is also capped at 1 KiB. Keep metadata and tags short.

Trace context and baggage

If your client already traces its work, for example with the Langfuse SDK or OpenTelemetry, the gateway can record its generation inside your trace instead of starting a new one:

traceparent (W3C Trace Context): the gateway uses its trace ID and nests the generation under the calling span. tracestate is kept. This is also how turn grouping works for Claude Code.
baggage (W3C Baggage): the gateway reads these entries, the same keys the Langfuse SDKs propagate:

Baggage key	Same as
`langfuse_user_id`	`langfuse-user-id`
`langfuse_session_id`	`langfuse-session-id`
`langfuse_trace_name`	`langfuse-trace-name`
`langfuse_environment`	`langfuse-environment`
`langfuse_tags`	`langfuse-tags`
`langfuse_metadata_<key>`	`langfuse-metadata`

When both are set, the langfuse-* header wins over the baggage entry. User keys ignore langfuse_user_id, like the header.

Don't also trace the same model call with a provider SDK integration, such as the Langfuse OpenAI integration. The call would be recorded twice: once by the integration and once by the gateway.

FAQ

What context does the gateway infer from coding agents?

For supported coding agents, the gateway also infers context from the agent's own headers and request body:

The session ID becomes the Langfuse session, prefixed with the agent name (for example claude-code:<session uuid>).
The original agent IDs are kept in metadata under agent.*: agent.name, agent.session_id, agent.id for subagents, agent.parent_session_id, and several Codex turn fields.
The provider's request ID is stored in the metadata key langfuse.gateway.upstream.request.id. Filter on it to find a single call.

See the relevant coding agent guide for how sessions and traces are grouped.

Which headers does the gateway add to its responses?

The gateway adds these headers to its responses:

Header	When	Value
`langfuse-request-id`	Every response, including authentication failures, gateway errors, and streams (sent with the first headers)	The gateway request ID, a UUIDv7
`langfuse-trace-id`	Responses to calls that produce a generation	The generation's trace ID, after `traceparent` and agent turn grouping
`langfuse-observation-id`	Responses to calls that produce a generation	The generation's observation ID
`langfuse-organization-id`	After the gateway key is resolved	Your Langfuse organization ID
`langfuse-project-id`	After the gateway key is resolved, when an ingestion project is set	The ingestion project ID
`langfuse-provider`	After the gateway called a provider	`openai` or `anthropic`
`langfuse-provider-connection-id`	After the gateway called a provider	The ID of the provider credential that served the call

Calls that produce a generation are POST /openai/v1/responses, /openai/v1/responses/compact, /openai/v1/chat/completions, and /anthropic/v1/messages. The trace and observation IDs are also set on 502 and 504 responses for provider failures, which still record a generation. Model listings, count_tokens, and requests rejected before they reach the provider (for example, with a 401) omit them.
Recording is best-effort, so in rare cases a returned trace ID has no stored trace.
Gateway error bodies also carry the request ID as a top-level request_id field, in the native error format of each namespace. Errors returned by the provider keep their original body.
The request ID is stored on the generation as the metadata key langfuse.gateway.request.id. Filter on it to find the call.
Your own x-request-id header never replaces the gateway request ID. It is stored separately as langfuse.gateway.client.request.id, and is not forwarded to the provider.
The provider's own request ID header (OpenAI x-request-id, Anthropic request-id) is still relayed unchanged.

To see the headers, add -i to a curl call:

curl -si "$LANGFUSE_GATEWAY_HOST/openai/v1/chat/completions" \
  -H "Authorization: Bearer $LANGFUSE_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"<model-id>","messages":[{"role":"user","content":"Say hello"}]}' \
  | grep -i '^langfuse-'

When you report a problem, include the langfuse-request-id. See reporting an issue.

Was this page helpful?

Tracing

Ingestion modes

Request headers

Headers

Rules

Trace context and baggage

FAQ

On this page