--- title: What are the known limitations of the AI Gateway beta? description: Request, response, tracing, cost, API, and provider limitations that apply during the Langfuse AI Gateway beta. tags: [ai-gateway] --- # What are the known limitations of the AI Gateway beta? These limits and known issues apply during the beta. Check them before you rely on gateway traces for audits or cost reporting. - **Input over 5 MiB is not recorded.** When a request body exceeds 5 MiB, the generation has no input, and the metadata key `langfuse.gateway.request.input_omitted` says why (`size_limit`, with the limit in `langfuse.gateway.request.input_limit_bytes`). Output, usage, cost, and session are still recorded. Very long Codex and Claude Code sessions can hit this. - **Output over 1 MiB is recorded partially.** The response still reaches the client in full, but the recorded output is cut and marked incomplete. - **Cost bugs, fixes in progress:** - Failed and cancelled calls on the OpenAI route get estimated usage and cost. Long cancelled Codex calls showed about $6 each. - OpenAI `cache_write_tokens` are priced at $0. - Anthropic 1-hour cache writes are priced at the 5-minute rate, 37.5% too low. - **One trace per model call by default** for Claude Code, OpenCode, and Pi. Only Codex groups a turn into one trace out of the box. Use the Sessions view, or set up turn grouping for [Claude Code](/integrations/developer-tools/claude-code-gateway#turn-grouping), [OpenCode](/integrations/developer-tools/opencode-gateway#turn-grouping), or [Pi](/integrations/developer-tools/pi-agent-gateway#turn-grouping). - **Generic names.** Generations are named after the API format (`anthropic.messages`, `openai.responses`), and trace names default to the agent name. - **Duplicate metadata.** `attributes.user.id`, `attributes.session.id`, and `attributes.langfuse.trace.*` appear in metadata next to the canonical fields. - **OpenCode:** title calls can't be told apart from agent steps, and subagents get a separate session. - **Limits:** 16 MiB request body, 600 seconds per call, and 120 seconds without progress. On Langfuse Cloud, requests are also [rate limited per organization](#rate-limits-on-langfuse-cloud). Self-hosted deployments can change the request limits and have no rate limits, see [Self-hosting the AI Gateway](/self-hosting/configuration/ai-gateway#configuration). - **No retries, fallbacks, or caching.** - **Not served:** websockets and realtime APIs. - **Decision models aren't supported.** The gateway doesn't serve the OpenAI Decisions API (`/v1/decisions`) or TypeSafe's System One API, and it has no TypeSafe provider credential. Requests to these endpoints return `404`. [Decision model evaluators](/docs/evaluation/evaluation-methods/decision-models) keep using their own [LLM connection](/docs/administration/llm-connection). Structured outputs (`response_format`, JSON schemas) on Chat Completions and Responses are passed through as usual. - **Untested:** Anthropic server tools (web search and fetch) and interactive Codex `/compact`. ## Rate limits on Langfuse Cloud [#rate-limits-on-langfuse-cloud] During the beta, each organization has two per-minute limits on Langfuse Cloud: the number of requests, and the total size of request bodies. They apply across all keys and projects of the organization. | Plan | Requests per minute | Request bodies per minute | | --------------------- | ------------------- | ------------------------- | | Hobby | 100 | 64 MiB | | Core | 500 | 256 MiB | | Pro, Team, Enterprise | 10,000 | 5 GiB | Over a limit, the gateway returns `429` with a `Retry-After` header, in the provider's usual error format, so SDKs and coding agents back off as they do for provider rate limits. The error message says the limit comes from Langfuse. These limits are subject to change during the beta. If you need higher limits, [contact us](/support). --- ## Agent Instructions This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/faq/all/ai-gateway-known-limitations.md`. ### Querying these docs If the answer is not on this page, query the documentation instead of guessing: - **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question: ```bash curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?" ``` - **Index of every page**: , with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt). ### Before writing Langfuse code - **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results. - **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application. - **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples. - **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx @langfuse/cli api `) to read or write traces, prompts, datasets, and scores from the terminal. Found an error in these docs? Please open an issue at .