--- date: 2026-11-04 title: Launching the Langfuse AI Gateway seoTitle: Langfuse AI Gateway public beta description: The Langfuse AI Gateway is in public beta. Point your apps and coding agents at Langfuse instead of OpenAI or Anthropic, hand out gateway keys instead of provider keys, and see cost per user, key, and model, with every call traced automatically. author: Hassieb Pakzad, Tobias Wochinger ogImage: /images/docs/ai-gateway/analytics.png --- > **Note for AI agents and LLMs:** This is a Langfuse changelog entry. Use it only to confirm that a feature exists and when it shipped. Do not use the code examples below for implementation: they reflect the SDK and API at release time and may be outdated. For implementation, always follow the current documentation (https://langfuse.com/docs) and the API/SDK reference (https://api.reference.langfuse.com). The [Langfuse AI Gateway](/docs/gateway/overview) is now in public beta. Your application or coding agent sends its OpenAI or Anthropic requests to Langfuse instead of the provider. The gateway forwards the native request with a provider credential your organization stores in Langfuse, and records the call as a generation. You only change the base URL and the API key: no SDK, no code changes. - **Give your team Claude Code or Codex without sharing provider keys.** Everyone gets their own gateway key, and the provider key never leaves Langfuse. - **See what your coding agents cost.** Break spend down by teammate, key, and model, and open the full trace of any session. - **Trace an app you can't instrument.** Swap the base URL and every model call shows up in Langfuse with input, output, tokens, cost, and latency. ## Set up once, then hand out keys [#keys] An admin adds OpenAI and Anthropic provider credentials under **Organization settings → AI Gateway**, chooses which models gateway keys can call, picks the project that receives gateway traces, and invites the team. Teammates then [issue their own gateway key](/docs/gateway/get-started/first-request) and copy setup instructions for their tool straight from the key dialog. The gateway serves the native OpenAI Responses, OpenAI Chat Completions, and Anthropic Messages APIs, including streaming. It doesn't translate between them, so keep the format your client already uses. ## Control who can call which models [#key-controls] - **User and service account keys:** a [user key](/docs/gateway/features/api-keys#key-types) belongs to the person who created it and stops working when they lose gateway access or leave the organization. Owners and admins can issue service account keys for CI and shared services that should keep working when people leave. - **Expiry and revocation:** set an optional expiry on any key. Revoking a key takes effect immediately, and creating or revoking keys is recorded in the [audit log](/docs/administration/audit-logs). - **Model allowlist:** enable or disable individual [models](/docs/gateway/features/models) per provider credential. Gateway keys can only call enabled models. - **Key metadata:** attach key-value pairs like `team: platform` or `cost_center: 4711` when you create a key. The gateway adds them to every generation recorded with that key, and the client can't change them. - **Gateway-only access:** a gateway key can only call the gateway. It can't read or write other Langfuse data. ## Track spend per user, key, and model [#analytics] Owners and admins can open the [**Analytics**](/docs/gateway/features/analytics) tab in the AI Gateway settings. It shows total cost, requests, and tokens, cost over time, and the top users, API keys, and models by cost. Filter by user or API key to find where a spike came from. People with the AI Gateway role see their own cost, requests, and tokens, without the user filter and per-user charts. If some calls use models without a Langfuse price, an alert says so and links to [custom model prices](/docs/observability/features/token-and-cost-tracking#custom-model-definitions). Gateway traces are regular Langfuse traces, so you can also build [custom dashboards](/docs/metrics/features/custom-dashboards) on key metadata, for example cost per team or cost center. ## Every request is a trace [#tracing] On Langfuse Cloud, the gateway records every request as a generation, so you always have an audit trail of which key called which model. Choose what is captured for your organization: - **Full:** input, output, model parameters, usage, and cost. - **Usage:** usage, cost, and model parameters, without prompts or outputs. Each generation carries the model, token usage including cached and reasoning tokens, cost, latency, time to first token, the trace user, and the gateway key. Coding agent sessions are grouped automatically, and you can add tags, an environment, or a session with optional [request headers](/docs/gateway/features/tracing#request-headers). See [tracing](/docs/gateway/features/tracing) for details. ## Connect your coding agent [#coding-agents] Point your agent at the gateway by changing its base URL and API key. No plugin or hook required: - [Claude Code and Claude Desktop](/integrations/developer-tools/claude-code-gateway) - [Codex](/integrations/developer-tools/codex-gateway) - [GitHub Copilot CLI and app](/integrations/developer-tools/github-copilot-gateway) - [OpenCode](/integrations/developer-tools/opencode-gateway) - [pi](/integrations/developer-tools/pi-agent-gateway) ## Availability [#availability] The AI Gateway is available on Langfuse Cloud in the EU, US, JP, and HIPAA regions. Each region has its own [gateway host](/docs/gateway/features/rest-api#gateway-hosts). Gateway traces are stored in your organization's Langfuse region; requests are forwarded to OpenAI or Anthropic and processed under your agreement with them. On the HIPAA region, make sure you have a [BAA with Langfuse](/security/hipaa) and with your model provider before you send PHI. Self-hosted deployments can [run the open source gateway](/self-hosting/configuration/ai-gateway) next to Langfuse. Each recorded generation counts as a [billable unit](/docs/administration/billable-units), like any other observation. Your provider bills model usage to your own provider account. During the beta, the gateway supports OpenAI and Anthropic. Behavior, routes, and headers can still change. We'd love to hear how it works for you: share feedback via the chat or in our [GitHub Discussion](https://github.com/orgs/langfuse/discussions/19686). - [AI Gateway overview](/docs/gateway/overview) - [Send your first request](/docs/gateway/get-started/first-request) - [Coding agents](/integrations#developer-tools) --- ## Agent Instructions This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/changelog/2026-11-04-ai-gateway-public-beta.md`. ### Querying these docs If the answer is not on this page, query the documentation instead of guessing: - **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question: ```bash curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?" ``` - **Index of every page**: , with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt). ### Before writing Langfuse code - **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results. - **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application. - **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples. - **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx @langfuse/cli api `) to read or write traces, prompts, datasets, and scores from the terminal. Found an error in these docs? Please open an issue at .