Cookbook: Monitor DeepSeek Models with Langfuse Using the OpenAI SDK

The DeepSeek API uses an API format compatible with OpenAI. By modifying the configuration, you can use the OpenAI SDK or software compatible with the OpenAI API to access the DeepSeek API.

This cookbook demonstrates how to monitor DeepSeek models using the OpenAI SDK integration with Langfuse. By leveraging Langfuse’s observability tools and the OpenAI SDK, you can effectively debug, monitor, and evaluate your applications that utilize DeepSeek models.

This guide will walk you through setting up the integration, making requests to DeepSeek models, and observing the interactions with Langfuse.

Setup

Install Required Packages

To get started, install the necessary packages. Ensure you have the latest versions of langfuse and openai.

%pip install langfuse openai --upgrade

Set Environment Variables

Set up your environment variables with the necessary keys. Obtain your Langfuse project keys from Langfuse Cloud. You will also need an access token from DeepSeek to access their models.

import os
 
# Get keys for your project from the project settings page: https://cloud.langfuse.com
os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-..." 
os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-..." 
os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com" # 🇪🇺 EU region
# os.environ["LANGFUSE_HOST"] = "https://us.cloud.langfuse.com" # 🇺🇸 US region
 
# Your DeepSeek API key (get it from https://platform.deepseek.com/api_keys)
os.environ["DEEPSEEK_API_KEY"] = "sk-..."  # Replace with your DeepSeek API key

Import Necessary Modules

Instead of importing openai directly, import it from langfuse.openai. Also, import any other necessary modules.

Check out our OpenAI integration docs to learn how to use this integration with other Langfuse features.

# Instead of: import openai
from langfuse.openai import OpenAI
from langfuse import observe

Initialize the OpenAI Client for DeepSeek Models

Initialize the OpenAI client, pointing it to the DeepSeek model endpoint. Replace the model URL and APP key with your own.

# Initialize the OpenAI client, pointing it to the DeepSeek Inference API
client = OpenAI(
    base_url="https://api.deepseek.com",  # Replace with the DeepSeek model endpoint URL
    api_key=os.getenv('DEEPSEEK_API_KEY'),  # Replace with your DeepSeek API key
)

Examples

Chat Completion Request

Use the client to make a chat completion request to the DeepSeek model. The model parameter can be any identifier since the actual model is specified in the base_url.

completion = client.chat.completions.create(
    model="deepseek-chat", 
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Why is AI cool? Answer in 20 words or less."}
    ]
)
print(completion.choices[0].message.content)

AI is cool because it automates tasks, enhances creativity, and solves complex problems quickly—making life smarter and easier.

View the example trace in Langfuse

Observe the Request with Langfuse

By using the OpenAI client from langfuse.openai, your requests are automatically traced in Langfuse. You can also use the @observe() decorator to group multiple generations into a single trace.

@observe()  # Decorator to automatically create a trace and nest generations
def generate_story():
    completion = client.chat.completions.create(
        name="story-generator",
        model="deepseek-chat",
        messages=[
            {"role": "system", "content": "You are a creative storyteller."},
            {"role": "user", "content": "Tell me a short story about a token that got lost on its way to the language model. Answer in 100 words or less."}
        ],
        metadata={"genre": "adventure"},
    )
    return completion.choices[0].message.content
 
story = generate_story()
print(story)

The Lost Token

Timmy the Token was excited—today, he’d help the language model craft a story! But as he raced through the data pipeline, he took a wrong turn, tumbling into a forgotten cache.

“Hello?” Timmy echoed. Only silence replied.

Days passed. The model stuttered without him. Then, a cleanup script swept through. “Gotcha!” it chirped, rescuing Timmy.

Back in the prompt, Timmy gleamed. The model sparked to life: “Once, a token got lost…”

And so, Timmy’s adventure became the very story he was meant to tell.

(100 words exactly)

View the example trace in Langfuse

Interoperability with the Python SDK

You can use this integration together with the Langfuse Python SDK to add additional attributes to the trace.

from langfuse import observe, get_client
 
langfuse = get_client()
 
@observe()
def my_instrumented_function(input):
    output = my_llm_call(input)
 
    langfuse.update_current_trace(
        input=input,
        output=output,
        user_id="user_123",
        session_id="session_abc",
        tags=["agent", "my-trace"],
        metadata={"email": "[email protected]"},
        version="1.0.0"
    )
 
    return output

from langfuse import get_client
 
langfuse = get_client()
 
with langfuse.start_as_current_span(name="my-trace") as span:
    
    # Run your application here
    output = my_llm_call(input)
 
    # Pass additional attributes to the span
    span.update_trace(
        input=input,
        output=output,
        user_id="user_123",
        session_id="session_abc",
        tags=["agent", "my-trace"],
        metadata={"email": "[email protected]"},
        version="1.0.0"
        )
 
# Flush events in short-lived applications
langfuse.flush()

Next Steps

Once you have instrumented your code, you can manage, evaluate and debug your application: