---
title: "Vercel AI Gateway Integration"
sidebarTitle: Vercel AI Gateway
logo: /images/integrations/vercel_ai_gateway_icon.svg
logoAppearance: dark
description: "Learn how to integrate Langfuse with the Vercel AI Gateway using the OpenAI SDK."
---

# Vercel AI Gateway Integration

In this guide, we'll show you how to integrate [Langfuse](/) with the [Vercel AI Gateway](https://vercel.com/docs/ai-gateway).

The [Vercel AI Gateway](https://vercel.com/docs/ai-gateway) is a proxy service from Vercel that routes model requests to various AI providers. It offers a unified API to multiple providers and gives you the ability to set budgets, monitor usage, load-balance requests, and manage fallbacks, the core capabilities of an [LLM gateway](/resources/engineering/llm-gateway).

Since the Vercel AI Gateway uses the OpenAI API schema, we can utilize Langfuse's native integration with the OpenAI SDK, available in both [Python](/integrations/model-providers/openai-py) and [TypeScript](/integrations/model-providers/openai-js).

## Get started

<LangTabs items={["Python SDK", "JS/TS SDK"]}>

<Tab>

```bash
pip install langfuse openai
```

```python
import os

# Set your Langfuse API keys
LANGFUSE_SECRET_KEY="sk-lf-..."
LANGFUSE_PUBLIC_KEY="pk-lf-..."
# 🇪🇺 EU region
LANGFUSE_BASE_URL="https://cloud.langfuse.com"
# Other Langfuse data regions include 🇺🇸 US: https://us.cloud.langfuse.com, 🇯🇵 Japan: https://jp.cloud.langfuse.com and ⚕️ HIPAA: https://hipaa.cloud.langfuse.com
# Set your Vercel AI Gateway API key or OIDC token (Vercel AI Gateway uses the 'AI_GATEWAY_API_KEY' and 'VERCEL_OIDC_TOKEN' environment variables)
os.environ["OPENAI_API_KEY"] = "<YOUR_VERCEL_AI_GATEWAY_API_KEY_OR_OIDC_TOKEN>"
```

</Tab>

<Tab>

```bash
npm install @langfuse/openai @langfuse/otel @opentelemetry/sdk-node openai
```

```typescript
// Set your Langfuse API keys
process.env.LANGFUSE_SECRET_KEY = "sk-lf-...";
process.env.LANGFUSE_PUBLIC_KEY = "pk-lf-...";
// 🇪🇺 EU region
process.env.LANGFUSE_BASE_URL = "https://cloud.langfuse.com";
// Other Langfuse data regions: 🇺🇸 US https://us.cloud.langfuse.com, 🇯🇵 Japan https://jp.cloud.langfuse.com, ⚕️ HIPAA https://hipaa.cloud.langfuse.com

// Set your Vercel AI Gateway API key or OIDC token (Vercel AI Gateway uses the 'AI_GATEWAY_API_KEY' and 'VERCEL_OIDC_TOKEN' environment variables)
process.env.OPENAI_API_KEY = "<YOUR_VERCEL_AI_GATEWAY_API_KEY_OR_OIDC_TOKEN>";
```

Set up OpenTelemetry with the `LangfuseSpanProcessor`:

```typescript
import { NodeSDK } from "@opentelemetry/sdk-node";
import { LangfuseSpanProcessor } from "@langfuse/otel";

const sdk = new NodeSDK({
  spanProcessors: [new LangfuseSpanProcessor()],
});

sdk.start();
```

</Tab>

</LangTabs>

## Example 1: Simple LLM Call

Since the Vercel AI Gateway provides an OpenAI-compatible API, we can use the Langfuse OpenAI SDK wrapper to automatically log Vercel AI Gateway calls as generations in Langfuse.

- The `base_url` is set to the Vercel AI Gateway API endpoint.
- You can replace `"anthropic/claude-4-sonnet"` with any model available on the Vercel AI Gateway.
- The `default_headers` can include optional headers as per the Vercel AI Gateway documentation.

<LangTabs items={["Python SDK", "JS/TS SDK"]}>

<Tab>

```python
# Import the Langfuse OpenAI SDK wrapper
from langfuse.openai import openai

# Create an OpenAI client with the Vercel AI Gateway base URL
client = openai.OpenAI(
    base_url="https://ai-gateway.vercel.sh/v1",
    default_headers={
        "http-referer": "<YOUR_SITE_URL>",  # Optional: Your site URL
        "x-title": "<YOUR_SITE_NAME>",      # Optional: Your site name
    }
)

# Make a chat completion request
response = client.chat.completions.create(
    model="anthropic/claude-4-sonnet",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Tell me a fun fact about space."}
    ],
    name="fun-fact-request"  # Optional: Name of the generation in Langfuse
)

# Print the assistant's reply
print(response.choices[0].message.content)
```

</Tab>

<Tab>

```typescript
import { observeOpenAI } from "@langfuse/openai";
import OpenAI from "openai";

// Create an OpenAI client with the Vercel AI Gateway base URL
const openaiClient = new OpenAI({
    baseURL: "https://ai-gateway.vercel.sh/v1",
    defaultHeaders: {
        "http-referer": "<YOUR_SITE_URL>",  // Optional: Your site URL
        "x-title": "<YOUR_SITE_NAME>",      // Optional: Your site name
    }
});

// Create an observed client with Langfuse options
const client = observeOpenAI(openaiClient, {
    generationName: "fun-fact-request"  // Optional: Name of the generation in Langfuse
});

// Make a chat completion request
const response = await client.chat.completions.create({
    model: "anthropic/claude-4-sonnet",
    messages: [
        { role: "system", content: "You are a helpful assistant." },
        { role: "user", content: "Tell me a fun fact about space." }
    ]
});

// Print the assistant's reply
console.log(response.choices[0].message.content);
```

</Tab>

</LangTabs>

## Example 2: Nested LLM Calls

We can capture execution details of nested LLM calls, inputs, outputs, and execution times. This provides in-depth observability with minimal code changes.

- Nested functions create a hierarchy of traces.
- Each LLM call within the functions is logged, providing a detailed trace of the execution flow.

<LangTabs items={["Python SDK", "JS/TS SDK"]}>

<Tab>

By using the `@observe()` decorator, we can capture execution details of any Python function.

- The `@observe()` decorator captures inputs, outputs, and execution details of the functions.

```python
from langfuse import observe
from langfuse.openai import openai

# Create an OpenAI client with the Vercel AI Gateway base URL
client = openai.OpenAI(
    base_url="https://ai-gateway.vercel.sh/v1",
)

@observe()  # This decorator enables tracing of the function
def analyze_text(text: str):
    # First LLM call: Summarize the text
    summary_response = summarize_text(text)
    summary = summary_response.choices[0].message.content

    # Second LLM call: Analyze the sentiment of the summary
    sentiment_response = analyze_sentiment(summary)
    sentiment = sentiment_response.choices[0].message.content

    return {
        "summary": summary,
        "sentiment": sentiment
    }

@observe()  # Nested function to be traced
def summarize_text(text: str):
    return client.chat.completions.create(
        model="openai/gpt-3.5-turbo",
        messages=[
            {"role": "system", "content": "You summarize texts in a concise manner."},
            {"role": "user", "content": f"Summarize the following text:\n{text}"}
        ],
        name="summarize-text"
    )

@observe()  # Nested function to be traced
def analyze_sentiment(summary: str):
    return client.chat.completions.create(
        model="openai/gpt-3.5-turbo",
        messages=[
            {"role": "system", "content": "You analyze the sentiment of texts."},
            {"role": "user", "content": f"Analyze the sentiment of the following summary:\n{summary}"}
        ],
        name="analyze-sentiment"
    )

# Example usage
text_to_analyze = "OpenAI's GPT-4 model has significantly advanced the field of AI, setting new standards for language generation."
analyze_text(text_to_analyze)
```

</Tab>

<Tab>

By wrapping the calls in `startActiveObservation`, nested LLM calls made with the wrapped OpenAI client are automatically grouped under one trace.

```typescript
import { startActiveObservation } from "@langfuse/tracing";
import { observeOpenAI } from "@langfuse/openai";
import OpenAI from "openai";

// Create an OpenAI client with the Vercel AI Gateway base URL
const openaiClient = new OpenAI({
    baseURL: "https://ai-gateway.vercel.sh/v1",
});

async function analyzeText(text: string) {
    // Create a root observation for the entire analysis
    return await startActiveObservation("analyze-text", async (span) => {
        span.update({ input: { text } });

        // First LLM call: Summarize the text
        const summaryResponse = await summarizeText(text);
        const summary = summaryResponse.choices[0].message.content;

        // Second LLM call: Analyze the sentiment of the summary
        const sentimentResponse = await analyzeSentiment(summary!);
        const sentiment = sentimentResponse.choices[0].message.content;

        const result = {
            summary,
            sentiment
        };

        span.update({ output: result });

        return result;
    });
}

async function summarizeText(text: string) {
    const client = observeOpenAI(openaiClient, {
        generationName: "summarize-text"
    });

    return await client.chat.completions.create({
        model: "openai/gpt-3.5-turbo",
        messages: [
            { role: "system", content: "You summarize texts in a concise manner." },
            { role: "user", content: `Summarize the following text:\n${text}` }
        ]
    });
}

async function analyzeSentiment(summary: string) {
    const client = observeOpenAI(openaiClient, {
        generationName: "analyze-sentiment"
    });

    return await client.chat.completions.create({
        model: "openai/gpt-3.5-turbo",
        messages: [
            { role: "system", content: "You analyze the sentiment of texts." },
            { role: "user", content: `Analyze the sentiment of the following summary:\n${summary}` }
        ]
    });
}

// Example usage
const textToAnalyze = "OpenAI's GPT-4 model has significantly advanced the field of AI, setting new standards for language generation.";
analyzeText(textToAnalyze);
```

</Tab>

</LangTabs>

## Learn More

- **Vercel AI Gateway Docs**: [https://vercel.com/docs/ai-gateway](https://vercel.com/docs/ai-gateway)
- **Langfuse OpenAI Integration**: [https://langfuse.com/integrations/model-providers/openai-py](/integrations/model-providers/openai-py)
- **Langfuse `@observe()` Decorator**: [https://langfuse.com/docs/observability/sdk/instrumentation](/docs/observability/sdk/instrumentation)

<!-- agent-instructions -->

---

## Agent Instructions

This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/integrations/gateways/vercel-ai-gateway.md`.

### Querying these docs

If the answer is not on this page, query the documentation instead of guessing:

- **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question:

  ```bash
  curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?"
  ```

- **Index of every page**: <https://langfuse.com/llms.txt>, with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt).

### Before writing Langfuse code

- **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results.
- **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application.
- **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples.
- **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx langfuse-cli api <resource> <action>`) to read or write traces, prompts, datasets, and scores from the terminal.

Found an error in these docs? Please open an issue at <https://github.com/langfuse/langfuse-docs/issues>.