---
source: ⚠️ Jupyter Notebook
title: Tracing vLLM with Langfuse via OpenTelemetry
sidebarTitle: vLLM
logo: /images/integrations/vllm_icon.svg
description: Learn how to trace vLLM inference with Langfuse using OpenTelemetry for LLM observability.
category: Integrations
---

# vLLM Integration

This cookbook shows how to trace [vLLM](https://github.com/vllm-project/vllm) inference with [Langfuse](https://langfuse.com) using OpenTelemetry. vLLM has [built-in OpenTelemetry support](https://docs.vllm.ai/en/latest/features/observability.html) that can be configured to send traces to Langfuse's [OpenTelemetry endpoint](/docs/opentelemetry/get-started).

> **What is vLLM?** [vLLM](https://github.com/vllm-project/vllm) is a fast and easy-to-use library for LLM inference and serving. It features state-of-the-art throughput, efficient memory management with PagedAttention, continuous batching, and support for a wide range of open-source models.

> **What is Langfuse?** [Langfuse](https://langfuse.com) is an open-source AI engineering platform. It provides tracing, prompt management, and evaluation capabilities to help teams debug, analyze, and iterate on their LLM applications.

## Get Started

We'll walk through a simple example of using vLLM with Langfuse tracing via OpenTelemetry.

<Steps>
### Step 1: Install Dependencies

```python
%pip install vllm langfuse -q
```

### Step 2: Set Up Environment Variables

Get your Langfuse API keys by signing up for [Langfuse Cloud](https://cloud.langfuse.com) or [self-hosting Langfuse](https://langfuse.com/self-hosting).

```python
import os

# Get keys for your project from the project settings page: https://cloud.langfuse.com
os.environ.setdefault("LANGFUSE_PUBLIC_KEY", "pk-lf-...");
os.environ.setdefault("LANGFUSE_SECRET_KEY", "sk-lf-...");
os.environ.setdefault("LANGFUSE_BASE_URL", "https://cloud.langfuse.com"); # 🇪🇺 EU region
# Other Langfuse data regions include 🇺🇸 US: https://us.cloud.langfuse.com, 🇯🇵 Japan: https://jp.cloud.langfuse.com and ⚕️ HIPAA: https://hipaa.cloud.langfuse.com

# Configure OpenTelemetry endpoint & headers
os.environ.setdefault("OTEL_EXPORTER_OTLP_TRACES_PROTOCOL", "http/protobuf");
os.environ.setdefault("OTEL_SERVICE_NAME", "vllm");
```

### Step 3: Initialize OpenTelemetry Tracing

vLLM automatically exposes OpenTelemetry spans when configured. The Langfuse client set up in the next step captures these OTEL spans and sends them to Langfuse.

```python
from vllm import LLM, SamplingParams

langfuse_host = "https://cloud.langfuse.com"  # Other regions: https://us.cloud.langfuse.com (US), https://jp.cloud.langfuse.com (Japan), https://hipaa.cloud.langfuse.com (HIPAA)
otlp_traces_endpoint = f"{langfuse_host}/api/public/otel/v1/traces"

# --- vLLM ---
llm = LLM(
    model="facebook/opt-125m",
    otlp_traces_endpoint=otlp_traces_endpoint,
    disable_log_stats=False,
)
```

Now we initialize the Langfuse OTel client. `get_client()` initializes the Langfuse client using the credentials provided in the environment variables.

```python
from langfuse import get_client

langfuse = get_client()

# Verify connection
if langfuse.auth_check():
    print("Langfuse client is authenticated and ready!")
else:
    print("Authentication failed. Please check your credentials and host.")
```

### Step 4: Load the Model with vLLM

We load the model using vLLM's `LLM` class. In this example, we use a small model (`facebook/opt-125m`) for demonstration purposes. You can replace this with any model supported by vLLM.

```python
out = llm.generate(
    ["Write one sentence about Berlin."],
    SamplingParams(max_tokens=32),
)
print(out[0].outputs[0].text)
```

### Step 5: See traces in Langfuse

After running the model, you can see new spans in Langfuse.

_**Note:** vLLM currently only exports the token counts and latency metrics to Langfuse. The LLM input and output need to be manually captured in a separate trace using the Langfuse SDK. _

[Example trace in Langfuse](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/462b76c435b348aa31ab82351c8ae33b?observation=a95f1c8affd878e9&timestamp=2025-12-23T12:59:03.259Z)

</Steps>

<!-- agent-instructions -->

---

## Agent Instructions

This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/integrations/model-providers/vllm.md`.

### Querying these docs

If the answer is not on this page, query the documentation instead of guessing:

- **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question:

  ```bash
  curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?"
  ```

- **Index of every page**: <https://langfuse.com/llms.txt>, with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt).

### Before writing Langfuse code

- **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results.
- **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application.
- **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples.
- **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx langfuse-cli api <resource> <action>`) to read or write traces, prompts, datasets, and scores from the terminal.

Found an error in these docs? Please open an issue at <https://github.com/langfuse/langfuse-docs/issues>.