Model management
Beta. The Langfuse AI Gateway is in public beta and can still change.
As an admin, you can manage which models are allowed to be used. This page explains where the model list comes from, how admins choose models, and which model ID to send.
Available models
You can find a list of every model gateway keys can call under Organization settings → AI Gateway → Models. Everyone with gateway access can open this page, including Viewers and the AI Gateway role.
The list comes from each provider credential: the gateway pings the provider's own /models endpoint to check which models the credential can access.
If a provider rejects a credential during model discovery, for example because the key was rotated, the credential is marked as failing and the gateway stops routing to it. Update the key or select Retry in Organization settings → AI Gateway → Setup to restore it.
Manage models
Gateway managers (Owners and Admins) can configure models per provider credential:
- Go to Organization settings → AI Gateway → Setup and open a provider credential.
- Turn models on or off with the Enabled switch.
- Select Refresh provider models to fetch the provider's current list.
A credential with every model enabled also allows models that the provider publishes later. As soon as you turn one model off, the credential keeps an explicit list. Select Enable all to go back to allowing every model.
Model IDs
Model IDs have the form <credential>:<model>, for example openai-prod:<model>, so the gateway knows which credential to use. For coding agents that send the provider's own model names, IDs without a prefix also work if they start with gpt-, codex-, ft:, or claude-, and the gateway uses the oldest matching credential. Renaming a credential changes its prefix, so update clients that use the old IDs.
Troubleshooting and FAQ
Why can't I call a new model?
A model the provider just released is available right away only on credentials that have every model enabled. If an admin turned off any model on a credential, the credential keeps an explicit list, and new models stay off until an admin enables them under Organization settings → AI Gateway → Setup. Select Refresh provider models there if the model doesn't appear in the list yet.
Last updated on