---
title: Observability for Sarvam AI with Langfuse
sidebarTitle: Sarvam AI
logo: /images/integrations/sarvam_icon.svg
logoAppearance: dark
description: Trace Sarvam AI chat, speech-to-text, and text-to-speech calls in Langfuse from Vercel AI SDK apps and LiveKit voice agents.
category: Integrations
---

# Observability for Sarvam AI with Langfuse

This guide shows how to trace **Sarvam AI** models with **Langfuse** from two places: a TypeScript app built with the **Vercel AI SDK**, and a real-time voice agent built with **LiveKit Agents**.

> **What is Sarvam AI?** [Sarvam AI](https://www.sarvam.ai) builds models for Indian languages: the `sarvam-30b` and `sarvam-105b` chat models, Saaras speech-to-text, Bulbul text-to-speech, and translation and transliteration APIs.

> **What is Langfuse?** [Langfuse](https://langfuse.com) is an open-source AI engineering platform. It helps teams trace LLM applications, debug issues, evaluate quality, and monitor cost and latency. Use [Langfuse Cloud](/cloud) (hosted by Langfuse, free tier, no infrastructure to run) or [self-host](/self-hosting) it.

This page shows two example integrations. Both use OpenTelemetry: the framework emits spans for each model call, and the Langfuse SDK exports them to your Langfuse project.

- [Vercel AI SDK](#vercel-ai-sdk): text generation, streaming, and tool calls with the `sarvam-ai-sdk` provider.
- [LiveKit Agents](#livekit-agents): voice agents that use Sarvam for speech-to-text, the LLM, and text-to-speech.

  These are examples, not the only way to trace Sarvam. You can trace Sarvam models with any Langfuse integration for the framework you already use. For example, use the [OpenAI SDK](/integrations/model-providers/openai-py) with Sarvam's OpenAI-compatible base URL `https://api.sarvam.ai/v1`, the Sarvam services in [Pipecat](/integrations/frameworks/pipecat), or a Sarvam model in [LangChain](/integrations/frameworks/langchain).

**Start with Langfuse** [Sign up](/cloud)

## Vercel AI SDK [#vercel-ai-sdk]

The community-maintained [`sarvam-ai-sdk`](https://github.com/sarvamai/sarvam-ai-sdk) package adds Sarvam as a Vercel AI SDK provider. Langfuse captures every AI SDK call that uses it through the [Langfuse Vercel AI SDK integration](/integrations/frameworks/vercel-ai-sdk).

<Steps>

### Install dependencies

_**Requires** Node.js 22 or later and AI SDK 7._

```bash
npm install ai@7 sarvam-ai-sdk @langfuse/vercel-ai-sdk @langfuse/otel @langfuse/tracing @opentelemetry/sdk-trace-node zod
```

  Each `sarvam-ai-sdk` minor version targets one AI SDK major version (`0.4.x` targets AI SDK 7). Keep `ai` on the major version your `sarvam-ai-sdk` release expects. See the [version table](https://github.com/sarvamai/sarvam-ai-sdk#version-compatibility).

### Configure credentials

Get Langfuse API keys from your project settings in [Langfuse Cloud](/cloud) or your [self-hosted](/self-hosting) instance, and a Sarvam API key from the [Sarvam dashboard](https://dashboard.sarvam.ai/).

```bash filename=".env"
LANGFUSE_SECRET_KEY = "sk-lf-..."
LANGFUSE_PUBLIC_KEY = "pk-lf-..."
LANGFUSE_BASE_URL = "https://cloud.langfuse.com" # 🇪🇺 EU region
# Other Langfuse data regions include 🇺🇸 US: https://us.cloud.langfuse.com, 🇯🇵 Japan: https://jp.cloud.langfuse.com and ⚕️ HIPAA: https://hipaa.cloud.langfuse.com

SARVAM_API_KEY = "..."
```

### Register the Langfuse integration

Create an `instrumentation.ts` file that sets up the `LangfuseSpanProcessor` and registers the Langfuse integration with the AI SDK once at startup.

```ts filename="instrumentation.ts"
import { registerTelemetry } from "ai";
import { LangfuseSpanProcessor } from "@langfuse/otel";
import { LangfuseVercelAiSdkIntegration } from "@langfuse/vercel-ai-sdk";
import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";

export const langfuseSpanProcessor = new LangfuseSpanProcessor();

const tracerProvider = new NodeTracerProvider({
  spanProcessors: [langfuseSpanProcessor],
});
tracerProvider.register();

registerTelemetry(new LangfuseVercelAiSdkIntegration());
```

### Call a Sarvam model

Import `instrumentation.ts` before any other module, then use `sarvam(...)` as the model in any AI SDK function. `propagateAttributes` sets the trace name, user, and session on every span the call creates.

```ts filename="app.ts"
import { langfuseSpanProcessor } from "./instrumentation";

import { generateText, stepCountIs, tool } from "ai";
import { propagateAttributes } from "@langfuse/tracing";
import { sarvam } from "sarvam-ai-sdk";
import { z } from "zod";

async function main() {
  const { text } = await propagateAttributes(
    {
      traceName: "sarvam-weather-assistant",
      userId: "user-123",
      sessionId: "session-456",
    },
    () =>
      generateText({
        model: sarvam("sarvam-105b"),
        prompt: "बेंगलुरु में आज मौसम कैसा है? हिंदी में जवाब दें।",
        tools: {
          getWeather: tool({
            description: "Get the current weather in a city",
            inputSchema: z.object({
              city: z.string().describe("The city to get the weather for"),
            }),
            execute: async ({ city }) => ({
              city,
              temperatureC: 27,
              condition: "partly cloudy",
            }),
          }),
        },
        stopWhen: stepCountIs(3),
        telemetry: { functionId: "sarvam-weather-assistant" },
      }),
  );

  console.log(text);
}

// Flush buffered spans before a short-lived script exits
main().finally(() => langfuseSpanProcessor.forceFlush());
```

Run it with `--env-file` so the keys in `.env` are loaded before `instrumentation.ts` runs:

```bash
npx tsx --env-file=.env app.ts
```

In a long-running server, the span processor exports spans in the background and you don't need `forceFlush()`. In serverless functions, such as a Next.js route handler on Vercel, call `after(() => langfuseSpanProcessor.forceFlush())` from `next/server` so spans are sent before the function stops.

### View the trace in Langfuse

Open your project in [Langfuse](https://cloud.langfuse.com/project/~/traces). The trace has an agent observation for the `generateText` call, a `sarvam-105b` generation with input, output, and token usage for each model step, and a tool observation for `getWeather`.

  ![Vercel AI SDK trace in Langfuse with two sarvam-105b generations and a getWeather tool call](/images/cookbook/integration-sarvam/sarvam-vercel-ai-sdk-trace.png)

[Example trace in Langfuse](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/fca085f66c4c0d0f8cbb16381662e9ed)

</Steps>

## LiveKit Agents [#livekit-agents]

The [`livekit-plugins-sarvam`](https://docs.livekit.io/agents/models/llm/plugins/sarvam/) plugin provides Sarvam speech-to-text, LLM, and text-to-speech for LiveKit Agents. LiveKit Agents has built-in OpenTelemetry support, so you pass it a `TracerProvider` that the Langfuse SDK exports from. The [LiveKit integration guide](/integrations/frameworks/livekit) covers the same setup for other providers and for Node.js.

<Steps>

### Install dependencies

_**Requires** Python 3.10 or later. Connecting the agent to a room also requires a [LiveKit](https://livekit.io) server, either LiveKit Cloud or self-hosted._

```bash
pip install "livekit-agents[sarvam,silero]~=1.8" langfuse opentelemetry-sdk python-dotenv
```

### Configure credentials

Add your Langfuse, Sarvam, and LiveKit credentials to a `.env` file next to your agent.

```bash filename=".env"
LANGFUSE_SECRET_KEY = "sk-lf-..."
LANGFUSE_PUBLIC_KEY = "pk-lf-..."
LANGFUSE_BASE_URL = "https://cloud.langfuse.com" # 🇪🇺 EU region
# Other Langfuse data regions include 🇺🇸 US: https://us.cloud.langfuse.com, 🇯🇵 Japan: https://jp.cloud.langfuse.com and ⚕️ HIPAA: https://hipaa.cloud.langfuse.com

SARVAM_API_KEY = "..."

LIVEKIT_URL = "wss://<your-project>.livekit.cloud"
LIVEKIT_API_KEY = "..."
LIVEKIT_API_SECRET = "..."
```

### Create a traced voice agent

`setup_langfuse()` creates a `TracerProvider`, passes it to LiveKit, and attaches the Langfuse exporter. `should_export_span=lambda span: True` exports LiveKit's speech-to-text, text-to-speech, and turn spans as well as the LLM calls. The room name becomes the Langfuse session ID, so all turns of one call are grouped together.

```python filename="agent.py"
from dotenv import load_dotenv
from langfuse import Langfuse
from opentelemetry.sdk.trace import TracerProvider

from livekit.agents import Agent, AgentServer, AgentSession, JobContext, cli, function_tool
from livekit.agents.telemetry import set_tracer_provider
from livekit.plugins import sarvam, silero

load_dotenv()


def setup_langfuse(ctx: JobContext) -> TracerProvider:
    tracer_provider = TracerProvider()
    # Group all turns of one call into a Langfuse session
    set_tracer_provider(tracer_provider, metadata={"langfuse.session.id": ctx.room.name})

    # Reads LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY and LANGFUSE_BASE_URL
    Langfuse(
        tracer_provider=tracer_provider,
        # Export LiveKit's STT, TTS and turn spans, not only LLM spans
        should_export_span=lambda span: True,
    )
    return tracer_provider


class Assistant(Agent):
    def __init__(self) -> None:
        super().__init__(
            instructions="You are a friendly voice assistant. Reply in the language the user speaks, in one or two short sentences.",
        )

    @function_tool
    async def get_weather(self, city: str) -> dict:
        """Get the current weather in a city."""
        return {"city": city, "temperature_c": 27, "condition": "partly cloudy"}


server = AgentServer()


@server.rtc_session()
async def entrypoint(ctx: JobContext):
    tracer_provider = setup_langfuse(ctx)

    async def flush_traces():
        tracer_provider.force_flush()

    ctx.add_shutdown_callback(flush_traces)

    session = AgentSession(
        stt=sarvam.STT(language="unknown"),  # auto-detect the spoken language
        # Use the /v1 endpoint: the plugin's default (/v2) can repeat tool calls
        llm=sarvam.LLM(model="sarvam-105b", base_url="https://api.sarvam.ai/v1"),
        tts=sarvam.TTS(target_language_code="hi-IN", model="bulbul:v3"),
        vad=silero.VAD.load(),
    )
    await session.start(agent=Assistant(), room=ctx.room)


if __name__ == "__main__":
    cli.run_app(server)
```

### Run the agent

Download the voice activity detection model files once, then start the agent and talk to it in your terminal:

```bash
python agent.py download-files
python agent.py console
```

To connect it to a LiveKit room instead, run `python agent.py dev` and join the room from the [Agents Playground](https://agents-playground.livekit.io).

### View the trace in Langfuse

Open your project in [Langfuse](https://cloud.langfuse.com/project/~/traces). Each conversation turn is an `agent_turn` observation with spans for speech-to-text and text-to-speech, `sarvam-105b` generations with token usage, and a tool observation for `get_weather`. Open **Sessions** to see all traces of one call together.

  ![LiveKit agent trace in Langfuse with sarvam-105b LLM requests and a get_weather tool call](/images/cookbook/integration-sarvam/sarvam-livekit-trace.png)

[Example trace in Langfuse](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/0928a8a8f3cda1a3d13fdfd9df2f762e)

</Steps>

## Track cost [#cost]

Langfuse records token usage for Sarvam models out of the box. To see cost, add a [custom model definition](/docs/observability/features/token-and-cost-tracking#custom-model-definitions) for each Sarvam model you use, with Sarvam's [per-token prices](https://www.sarvam.ai/api-pricing). Langfuse then calculates cost for every Sarvam generation, in both integrations.

## Interoperability with the Python SDK

You can use this integration together with the Langfuse [SDKs](/docs/observability/sdk/overview) to add additional attributes to the observation.

**Decorator**

The [`@observe()` decorator](/docs/observability/sdk/instrumentation#custom-instrumentation) provides a convenient way to automatically wrap your instrumented code and add additional attributes to the observation.

```python
from langfuse import observe, propagate_attributes, get_client

langfuse = get_client()

@observe()
def my_llm_pipeline(input):
    # Add additional attributes (user_id, session_id, metadata, version, tags) to all spans created within this execution scope
    with propagate_attributes(
        user_id="user_123",
        session_id="session_abc",
        tags=["agent", "my-observation"],
        metadata={"email": "user@langfuse.com"},
        version="1.0.0"
    ):

        # YOUR APPLICATION CODE HERE
        result = call_llm(input)

        return result

# Run the function
my_llm_pipeline("Hi")
```

Learn more about using the Decorator in the [Langfuse SDK instrumentation docs](/docs/observability/sdk/instrumentation#custom-instrumentation).

**Context Manager**

The [Context Manager](/docs/observability/sdk/instrumentation#custom-instrumentation) allows you to wrap your instrumented code using context managers (with `with` statements), which allows you to add additional attributes to the observation.

```python
from langfuse import get_client, propagate_attributes

langfuse = get_client()

with langfuse.start_as_current_observation(
    as_type="span",
    name="my-observation",
    trace_context={"trace_id": "abcdef1234567890abcdef1234567890"},  # Must be 32 hex chars
) as observation:

    # Add additional attributes (user_id, session_id, metadata, version, tags)
    # to all observations created within this execution scope
    with propagate_attributes(
        user_id="user_123",
        session_id="session_abc",
        metadata={"experiment": "variant_a", "env": "prod"},
        version="1.0",
    ):
        # YOUR APPLICATION CODE HERE
        result = call_llm("some input")

# Flush events in short-lived applications
langfuse.flush()
```

Learn more about using the Context Manager in the [Langfuse SDK instrumentation docs](/docs/observability/sdk/instrumentation#custom-instrumentation).

## Troubleshooting

<details>
<summary>No observations appearing</summary>

First, enable [debug mode](/docs/observability/sdk/advanced-features#logging--debugging) in the Python SDK:

```bash
export LANGFUSE_DEBUG="True"
```

Then run your application and check the debug logs:

- **OTel observations appear in the logs:** Your application is instrumented correctly but observations are not reaching Langfuse. To resolve this:
  1. Call [`langfuse.flush()`](/docs/observability/sdk/instrumentation#client-lifecycle--flushing) at the end of your application to ensure all observations are exported.
  2. Verify that you are using the correct API keys and base URL.
- **No OTel spans in the logs:** Your application is not instrumented correctly. Make sure the instrumentation runs before your application code.

</details>

<details>
<summary>Unwanted observations in Langfuse</summary>

The Langfuse SDK is based on OpenTelemetry. Other libraries in your application may emit OTel spans that are not relevant to you. These still count toward your [billable units](/docs/administration/billable-units), so you should filter them out. See [Unwanted spans in Langfuse](/faq/all/unwanted-http-database-spans) for details.

</details>

<details>
<summary>Missing attributes</summary>

Some attributes may be stored in the metadata object of the observation rather than being mapped to the Langfuse data model. If a mapping or integration does not work as expected, please [raise an issue on GitHub](/issues).

</details>

## Next Steps

Once you have instrumented your code, you can manage, evaluate and debug your application:

- [Manage prompts in Langfuse](/docs/prompt-management/get-started)
- [Add evaluation scores](/docs/evaluation/evaluation-methods/scores-via-sdk)
- [Run LLM-as-a-judge Evaluators](/docs/evaluation/evaluation-methods/llm-as-a-judge)
- [Create datasets](/docs/evaluation/experiments/datasets)
- [Create custom dashboards](/docs/metrics/features/custom-dashboards)
- [Test queries in the Playground](/docs/prompt-management/features/playground)

<!-- agent-instructions -->

---

## Agent Instructions

This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/integrations/model-providers/sarvam.md`.

### Querying these docs

If the answer is not on this page, query the documentation instead of guessing:

- **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question:

  ```bash
  curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?"
  ```

- **Index of every page**: <https://langfuse.com/llms.txt>, with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt).

### Before writing Langfuse code

- **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results.
- **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application.
- **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples.
- **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx @langfuse/cli api <resource> <action>`) to read or write traces, prompts, datasets, and scores from the terminal.

Found an error in these docs? Please open an issue at <https://github.com/langfuse/langfuse-docs/issues>.
