Set up Gateway as an admin
Beta. The Langfuse AI Gateway is in public beta and can still change.
This page walks through how to set up the AI Gateway for your organization. Read Core concepts first to understand how the gateway works. If you want to use the gateway with your coding agent/AI app as a user, go to Send your first request.
Self-hosting Langfuse? The gateway is a separate service that runs next to Langfuse. See Self-hosting the AI Gateway.
Setup steps
This guide helps you set up Langfuse Gateway as an admin.
HIPAA region: requests are forwarded to the provider you configure, using your credentials and on your instruction. Before you send protected health information (PHI) through the gateway, make sure you have a BAA with Langfuse and a BAA with that provider.
Configure the gateway
Go to Organization settings → AI Gateway → Setup and choose:
- the project, where the gateway records generations
- the ingestion mode:
FULL(input and output) orUSAGE(usage and cost only, the default). Self-hosted deployments also offerNONE(route without recording).
Add provider credentials
On the same page, select Add credential and add API keys for model providers you’d like to use. The credential's name becomes the prefix of its model IDs.
Provider credentials are separate from project LLM connections, which power the Playground and LLM-as-a-Judge.
Configure allowed models (optional)
By default, a credential allows every model the provider offers, including models the provider publishes later. To restrict a credential to selected models, turn models off on its models page. See Model management.
Give your team access
Use Notify your team to email teammates. People who are not in the organization yet are invited with the AI Gateway role: they can call the gateway, manage their own keys, and see analytics of their own usage, but can't see any projects. This role suits people who only need to call models, for example developers who use the gateway for their coding agent.
You can change anyone's role later in the organization's members settings, for example to let them see their traces in the ingestion project. See roles and permissions for what each role can do.
Not seeing what you expected?
Next steps
Last updated on