Tracing AI SDK C++ with Langfuse

This guide shows how to use the built-in Langfuse tracing module in AI SDK C++ to capture traces for ai::Client::generate_text calls, including the LLM round-trip and every tool invocation.

What is AI SDK C++? AI SDK C++ is a modern C++20 toolkit, maintained by ClickHouse, for building AI-powered applications with providers like OpenAI and Anthropic. It exposes a unified ai::Client API for text generation, streaming, tools, and multi-step agent loops.

What is Langfuse? Langfuse is an open-source AI engineering platform that provides tracing, evaluation, prompt management, and metrics to debug and improve LLM applications. Use Langfuse Cloud (hosted by Langfuse, free tier, no infrastructure to run) or self-host it.

Compatibility note: the current upstream ai::langfuse transport predates Langfuse's OTLP-first v4 data model. For new Langfuse Cloud and self-hosted v4 deployments, prefer standard OpenTelemetry instrumentation exported to Langfuse's OTLP/HTTP endpoint. Use this page only when you explicitly need the current AI SDK C++ Langfuse module and have validated it against your target deployment.

Trace shape

The ai::langfuse component emits one trace per logical operation with the following structure:

1 trace – caller sets the input, output, metadata, and tags.
1 generation observation – model, model parameters, input messages, output text, aggregated usage, and finish reason.
1 span per tool call – parented to the generation, with tool arguments as input and the result (or error) as output.

Install AI SDK C++ with the Langfuse component

Add AI SDK C++ to your CMake project and link against the ai::langfuse target. The Langfuse component is built alongside the core, OpenAI, and Anthropic targets:

find_package(ai-sdk-cpp CONFIG REQUIRED)

target_link_libraries(my_app
  PRIVATE
    ai::sdk        # umbrella target (core + providers + langfuse)
    ai::langfuse   # or link this directly if you prefer fine-grained deps
)

The build defines AI_SDK_HAS_LANGFUSE=1 so you can guard tracing code behind a compile-time check if needed. See the AI SDK C++ README for full installation instructions.

Configure Langfuse credentials

Set the Langfuse credentials for the example. You can obtain these keys from your Langfuse Cloud project settings or from a self-hosted Langfuse instance.

# LLM provider (required by the example)
export OPENAI_API_KEY="sk-..."

# Langfuse credentials
export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."

# Langfuse host (optional, defaults to https://cloud.langfuse.com)
# 🇪🇺 EU cloud:  https://cloud.langfuse.com
# 🇺🇸 US cloud:  https://us.cloud.langfuse.com
# 🇯🇵 Japan:     https://jp.cloud.langfuse.com
# ⚕️ HIPAA:     https://hipaa.cloud.langfuse.com
# Self-host:    http://localhost:3000
export LANGFUSE_HOST="https://cloud.langfuse.com"

Instrument a generate_text call

Construct a Tracer once per process and call tracer.start_trace(...) for each logical operation. Pass the trace to ai::langfuse::generate_text, which wraps ai::Client::generate_text and records the LLM call and tool invocations automatically.

#include <ai/langfuse.h>
#include <ai/openai.h>
#include <ai/tools.h>

#include <cstdlib>
#include <iostream>

int main() {
  // 1. Create a Tracer once and reuse it for all traces.
  ai::langfuse::Tracer tracer({
      .host = std::getenv("LANGFUSE_HOST")
                ? std::getenv("LANGFUSE_HOST")
                : "https://cloud.langfuse.com",
      .public_key = std::getenv("LANGFUSE_PUBLIC_KEY"),
      .secret_key = std::getenv("LANGFUSE_SECRET_KEY"),
      .environment = "ai-sdk-cpp-example",
  });
  if (!tracer.is_valid()) {
    std::cerr << "Langfuse tracer not configured.\n";
    return 1;
  }

  // 2. Configure the LLM client and tools as usual.
  auto client = ai::openai::create_client();

  ai::GenerateOptions options;
  options.model = ai::openai::models::kGpt4oMini;
  options.system =
      "You are a concise assistant. Use the available tools when helpful.";
  options.prompt = "Look up alice and tell me the weather where she lives.";
  options.max_steps = 4;
  options.temperature = 0.0;
  // options.tools = ... ;  // register your ai::ToolSet here

  // 3. Start a trace and attach input/metadata.
  auto trace = tracer.start_trace("langfuse_tracing_example");
  trace->set_input(options.prompt);
  trace->set_metadata({{"example", "langfuse_tracing"}, {"sdk", "ai-sdk-cpp"}});

  // 4. Run generate_text via the Langfuse wrapper.
  auto result = ai::langfuse::generate_text(client, std::move(options), *trace);

  if (result) {
    std::cout << "Output: " << result.text << "\n";
    trace->set_output(result.text);
  } else {
    trace->set_output(ai::JsonValue{{"error", result.error_message()}});
  }

  // 5. Flush the batch synchronously to Langfuse.
  trace->end();
  return result ? 0 : 2;
}

The wrapper hooks into the existing on_tool_call_start / on_tool_call_finish callbacks on GenerateOptions and chains any callbacks you have already installed, so tracing composes cleanly with your own instrumentation.

Tracer configuration

ai::langfuse::Config accepts the following options:

Field	Description
`host`	Base URL of your Langfuse instance. Defaults to `https://cloud.langfuse.com`.
`public_key`	Public API key (`pk-lf-...`).
`secret_key`	Secret API key (`sk-lf-...`).
`release`	Optional release identifier attached to all traces (e.g. a commit SHA).
`environment`	Environment tag attached to all traces. Defaults to `default`.
`connection_timeout_sec`	HTTP connection timeout for the ingestion request. Defaults to 10 seconds.
`read_timeout_sec`	HTTP read timeout for the ingestion request. Defaults to 30 seconds.
`error_policy`	`kStrict` surfaces HTTP/JSON failures via `Trace::end()`; `kBestEffort` swallows them.

Trace API

A Trace accumulates events in memory and flushes them when end() is called.

auto trace = tracer.start_trace("sql-generation");

trace->set_input(user_prompt);
trace->set_user_id("user-123");
trace->set_session_id("session-abc");
trace->set_metadata({{"feature", "sql-assistant"}});
trace->add_tag("beta");

auto result = ai::langfuse::generate_text(client, std::move(options), *trace);

if (result) {
  trace->set_output(result.text);
} else {
  trace->set_output(ai::JsonValue{{"error", result.error_message()}});
}
trace->end();  // idempotent, synchronous flush

Trace methods are thread-safe, so callbacks fired from generate_text's worker threads can record into the same trace concurrently.

Notes and limitations

Trace::end() flushes synchronously; there is no background worker thread (yet).
Intermediate on_step_finish events are not emitted as separate generations — step text and usage roll up into the single parent generation observation.
The integration ships with AI SDK C++ as an optional CMake target. If you do not link ai::langfuse, your binary stays free of the additional HTTP and JSON code paths.
For greenfield Langfuse v4 integrations, prefer OpenTelemetry-based C++ instrumentation so you can export OTLP/HTTP directly to Langfuse's supported ingestion path.

Learn more

Was this page helpful?

On this page