--- title: Tracing seoTitle: "Tracing with the Langfuse AI Gateway (beta)" description: What the Langfuse AI Gateway records for every model call, the FULL and USAGE ingestion modes, where gateway traces are stored, and the request headers that enrich them. --- # Tracing **Beta.** The Langfuse AI Gateway is in public beta and can still change. The gateway records every model call it routes as a Langfuse [generation](/docs/observability/data-model). Gateway traces are regular Langfuse traces, so you can filter, evaluate, and chart them like traces from the SDKs. ![Langfuse session view of a Codex conversation recorded by the gateway, with messages, tool calls, cost, and token usage](/images/docs/ai-gateway/tracing-session.png) ## Ingestion modes [#ingestion-modes] There are different ingestion modes, which gateway managers set for the whole organization in **Organization settings → AI Gateway → Setup**: | Mode | Recorded | Use when | | ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | | `FULL` | Everything `USAGE` records, plus the full input (system prompt, tool definitions, messages, tool calls and results) and the streamed output (text, reasoning or thinking, and tool calls with parsed arguments) | You want to debug, evaluate, or build datasets from real prompts and responses | | `USAGE` (default) | The fields listed below, without input or output | You need cost, usage, and an audit trail, but prompts and responses must not be stored in Langfuse | | `NONE` (self-hosted only) | Nothing. The gateway only routes requests. | You only want to hand out gateway keys instead of provider keys, without any traces |
Full list of fields recorded in `FULL` and `USAGE` mode - the model (the dated response model when the provider returns one), matched to [model prices](/docs/observability/features/token-and-cost-tracking), and scalar model parameters such as `temperature` (`FULL` also records structured options such as `response_format`) - token usage, including cached and reasoning tokens, plus cost, latency, and time to first token - level `ERROR` with the provider's status and message for failed calls, and `WARNING` for client cancellations - sessions, grouped automatically from each coding agent's own session ID - the [trace user](/docs/gateway/features/api-keys#trace-user), which depends on the key type - the gateway key, its [key metadata](/docs/gateway/features/api-keys#key-metadata), and the provider credential in metadata, under `langfuse.gateway.api_key.*` and `langfuse.gateway.connection.*`
The benefit of having the `FULL` ingestion mode is that you can observe and improve the quality of the agent's responses. For example, you can run [LLM-as-a-Judge evaluations](/docs/evaluation/get-started/online) on production traffic, review calls in [annotation queues](/docs/evaluation/evaluation-methods/annotation-queues), or turn real prompts into [datasets](/docs/evaluation/experiments/datasets) to improve your setup. Add tags, metadata, an environment, a session, or a trace name with optional [request headers](#request-headers). For how coding agent calls are grouped into traces and sessions, see the relevant [coding agent guide](/integrations#developer-tools). ## Request headers [#request-headers] Any client can send these optional headers to enrich the traces the gateway records. They work in both `FULL` and `USAGE` [ingestion modes](#ingestion-modes). Each [coding agent guide](/integrations#developer-tools) shows where to configure them. ### Headers [#headers] | Header | Format | Effect | | ---------------------- | --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `langfuse-user-id` | string | Trace [user](/docs/observability/features/users). Only used with service account keys: user keys always record the key owner. See [trace user](/docs/gateway/features/api-keys#trace-user). | | `langfuse-session-id` | string | Overrides the agent's own [session](/docs/observability/features/sessions). | | `langfuse-environment` | string | [Environment](/docs/observability/features/environments) of the trace. Invalid values fall back to `default`. | | `langfuse-tags` | comma-separated | [Tags](/docs/observability/features/tags), trimmed and deduplicated. | | `langfuse-metadata` | comma-separated `key:value` | Top-level [metadata](/docs/observability/features/metadata). Splits at the first colon, values stay strings. Percent-encode literal commas (`%2C`). For metadata clients can't change, use [key metadata](/docs/gateway/features/api-keys#key-metadata). | | `langfuse-trace-name` | string | Overrides the default trace name (the agent name). | Example: ```text langfuse-user-id: jane@example.com langfuse-environment: ci langfuse-tags: claude-code,platform-team langfuse-metadata: repo:langfuse-docs,branch:main ``` ### Rules [#rules] - **Do not set a static `langfuse-session-id` or `langfuse-trace-name` in a config file.** A static session ID merges all your sessions into one, and a static trace name renames every trace. Set them per run from an environment variable, and only when you want that effect, for example in scripted or CI runs. - Explicit headers take precedence over values the gateway infers from the agent. Empty values are ignored. - `langfuse-environment` must match the [environment format](/docs/observability/features/environments): lowercase letters, numbers, `-` and `_`, at most 40 characters, and not starting with `langfuse`. Any other value is recorded as `default`, and the request still succeeds. - Header values are percent-decoded. Encode a literal `%` as `%25`, and a comma inside a tag or metadata value as `%2C`. - Invalid values are dropped without failing the request. The gateway ignores **all** context headers of a request, including `traceparent` and `baggage`, when they add up to more than 8 KiB. Each value is also capped at 1 KiB. Keep metadata and tags short. ### Trace context and baggage [#trace-context] If your client already traces its work, for example with the [Langfuse SDK](/docs/observability/sdk/overview) or OpenTelemetry, the gateway can record its generation inside your trace instead of starting a new one: - **`traceparent`** (W3C Trace Context): the gateway uses its trace ID and nests the generation under the calling span. `tracestate` is kept. This is also how [turn grouping](/integrations/developer-tools/claude-code-gateway#turn-grouping) works for Claude Code. - **`baggage`** (W3C Baggage): the gateway reads these entries, the same keys the Langfuse SDKs propagate: | Baggage key | Same as | | ------------------------- | ---------------------- | | `langfuse_user_id` | `langfuse-user-id` | | `langfuse_session_id` | `langfuse-session-id` | | `langfuse_trace_name` | `langfuse-trace-name` | | `langfuse_environment` | `langfuse-environment` | | `langfuse_tags` | `langfuse-tags` | | `langfuse_metadata_` | `langfuse-metadata` | When both are set, the `langfuse-*` header wins over the baggage entry. User keys ignore `langfuse_user_id`, like the header. Don't also trace the same model call with a provider SDK integration, such as the Langfuse OpenAI integration. The call would be recorded twice: once by the integration and once by the gateway. ## FAQ [#faq] ### What context does the gateway infer from coding agents? [#inferred-agent-context] For supported coding agents, the gateway also infers context from the agent's own headers and request body: - The session ID becomes the Langfuse session, prefixed with the agent name (for example `claude-code:`). - The original agent IDs are kept in metadata under `agent.*`: `agent.name`, `agent.session_id`, `agent.id` for subagents, `agent.parent_session_id`, and several Codex turn fields. - The provider's request ID is stored in the metadata key `langfuse.gateway.upstream.request.id`. Filter on it to find a single call. See the relevant [coding agent guide](/integrations#developer-tools) for how sessions and traces are grouped. ### Which headers does the gateway add to its responses? [#response-headers] The gateway adds these headers to its responses: | Header | When | Value | | --------------------------------- | ------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------- | | `langfuse-request-id` | Every response, including authentication failures, gateway errors, and streams (sent with the first headers) | The gateway request ID, a UUIDv7 | | `langfuse-trace-id` | Responses to calls that produce a generation | The generation's trace ID, after `traceparent` and agent turn grouping | | `langfuse-observation-id` | Responses to calls that produce a generation | The generation's observation ID | | `langfuse-organization-id` | After the gateway key is resolved | Your Langfuse organization ID | | `langfuse-project-id` | After the gateway key is resolved, when an ingestion project is set | The ingestion project ID | | `langfuse-provider` | After the gateway called a provider | `openai` or `anthropic` | | `langfuse-provider-connection-id` | After the gateway called a provider | The ID of the provider credential that served the call | - Calls that produce a generation are `POST /openai/v1/responses`, `/openai/v1/responses/compact`, `/openai/v1/chat/completions`, and `/anthropic/v1/messages`. The trace and observation IDs are also set on `502` and `504` responses for provider failures, which still record a generation. Model listings, `count_tokens`, and requests rejected before they reach the provider (for example, with a `401`) omit them. - Recording is best-effort, so in rare cases a returned trace ID has no stored trace. - Gateway error bodies also carry the request ID as a top-level `request_id` field, in the native error format of each namespace. Errors returned by the provider keep their original body. - The request ID is stored on the generation as the metadata key `langfuse.gateway.request.id`. Filter on it to find the call. - Your own `x-request-id` header never replaces the gateway request ID. It is stored separately as `langfuse.gateway.client.request.id`, and is not forwarded to the provider. - The provider's own request ID header (OpenAI `x-request-id`, Anthropic `request-id`) is still relayed unchanged. To see the headers, add `-i` to a `curl` call: ```sh curl -si "$LANGFUSE_GATEWAY_HOST/openai/v1/chat/completions" \ -H "Authorization: Bearer $LANGFUSE_GATEWAY_KEY" \ -H "Content-Type: application/json" \ -d '{"model":"","messages":[{"role":"user","content":"Say hello"}]}' \ | grep -i '^langfuse-' ``` When you report a problem, include the `langfuse-request-id`. See [reporting an issue](/faq/all/ai-gateway-report-issue). [Ask another question](/docs/ask-ai) --- ## Agent Instructions This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/docs/gateway/features/tracing.md`. ### Querying these docs If the answer is not on this page, query the documentation instead of guessing: - **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question: ```bash curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?" ``` - **Index of every page**: , with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt). ### Before writing Langfuse code - **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results. - **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application. - **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples. - **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx @langfuse/cli api `) to read or write traces, prompts, datasets, and scores from the terminal. Found an error in these docs? Please open an issue at .