Pi with the Langfuse AI Gateway

Beta. The Langfuse AI Gateway is in public beta and can still change.

Requires Pi 1.5.0 or later and an ANTHROPIC and/or OPENAI provider credential.

For tracing Pi without the gateway, see the Pi Agent integration.

Setup

Create a gateway API key

Create a gateway API key under Organization settings → AI Gateway → Gateway keys. See API keys.

Export it in the shell you start Pi from:

export LANGFUSE_GATEWAY_KEY="<your-gateway-key>"

To keep the setup across sessions, add the export lines to ~/.zshrc.

Install Pi

Install Pi from the @earendil-works/pi-coding-agent package. The older @mariozechner/… package is deprecated.

npm install -g @earendil-works/pi-coding-agent

Add the gateway providers

Add gateway providers to ~/.pi/agent/models.json (or $PI_CODING_AGENT_DIR/models.json). $VAR interpolation works in apiKey and in headers.

{
  "providers": {
    "langfuse-anthropic": {
      "baseUrl": "https://gateway.cloud.langfuse.com/anthropic",
      "api": "anthropic-messages",
      "apiKey": "$LANGFUSE_GATEWAY_KEY",
      "compat": { "sendSessionAffinityHeaders": true },
      "headers": {
        "langfuse-tags": "pi,<team>",
        "langfuse-environment": "local"
      },
      "models": [
        {
          "id": "<anthropic-model-id>",
          "reasoning": true,
          "input": ["text", "image"],
          "contextWindow": 200000,
          "maxTokens": 64000
        }
      ]
    },
    "langfuse-openai": {
      "baseUrl": "https://gateway.cloud.langfuse.com/openai/v1",
      "api": "openai-responses",
      "apiKey": "$LANGFUSE_GATEWAY_KEY",
      "headers": {
        "langfuse-tags": "pi,<team>",
        "langfuse-environment": "local"
      },
      "models": [
        {
          "id": "<openai-model-id>",
          "reasoning": true,
          "input": ["text", "image"],
          "contextWindow": 400000,
          "maxTokens": 128000
        }
      ]
    }
  }
}

pi --provider langfuse-anthropic --model <anthropic-model-id>
pi -p --provider langfuse-openai --model <openai-model-id> "Summarize README.md"

Sessions on the Anthropic route require "compat": { "sendSessionAffinityHeaders": true }. Without it, Pi sends no session ID to Anthropic and the generations have no session. The OpenAI route always sends the session ID.
The Anthropic baseUrl has no /v1. The OpenAI baseUrl includes /v1.
Use "api": "openai-responses". openai-completions uses Chat Completions, which the gateway serves but which has not been verified with pi.
Replace the model IDs with IDs from /models, and set reasoning, contextWindow, and maxTokens to the values of that model.
List every model you use under models. Otherwise, Pi warns "Using custom model id" and guesses the context window and reasoning support.

Verify

Run a prompt:

pi -p --provider langfuse-anthropic --model <anthropic-model-id> "Reply with exactly: pong"

If Pi replies, it runs through the gateway.

Turn grouping (experimental, optional)

Experimental. This extension works with the current gateway but is not an official, published package yet. A dedicated turn header is planned.

Pi sends no turn ID, so every model call is its own trace. A small extension can send a W3C traceparent with a new trace ID per prompt, which the gateway adopts. Save it as ~/.pi/agent/extensions/langfuse-turn.ts (or .pi/extensions/ in a project), or load it for one run with pi -e ./langfuse-turn.ts:

import { randomBytes } from "node:crypto";

export default function (pi: any) {
  let traceId: string | undefined;

  // One trace per user prompt (agent run).
  pi.on("agent_start", () => {
    traceId = randomBytes(16).toString("hex");
  });

  pi.on("before_provider_headers", (event: any) => {
    if (!traceId) return;
    const spanId = randomBytes(8).toString("hex");
    event.headers["traceparent"] = `00-${traceId}-${spanId}-01`;
  });
}

All model calls of a prompt, including retries and follow-up messages within the run, land in one trace. The generations appear as top-level observations, because the parent span does not exist in Langfuse.
Calls after the run ends, such as auto-compaction, join the prompt that triggered them.

Troubleshooting and FAQ

Can headers differ per request?

No. Headers are static per provider. Vary them with $VAR, or define several providers.

Where are thinking tokens counted?

On Anthropic, thinking tokens are counted inside output tokens. OpenAI reasoning tokens are shown separately.

Why isn't Pi's prompt cached on small models?

Pi's default prompt (about 2,000 tokens) is cached on Sonnet but is below the minimum cacheable size of Haiku 4.5.

Was this page helpful?

On this page