Trace TanStack AI with Langfuse
TanStack AI is TanStack's framework for building AI-powered applications in TypeScript, with a provider-agnostic chat() API, streaming, and tool calling. By combining TanStack AI with Langfuse, you can trace, monitor, and analyze every agent step, tool call, and model response in development and production.
This notebook demonstrates how to use the openInferenceMiddleware from OpenInference to automatically instrument TanStack AI calls and send OpenTelemetry spans to Langfuse.
What is TanStack AI?
TanStack AI is a TypeScript framework for building AI applications. It provides a unified, provider-agnosticchat()API with built-in support for streaming, tool calling, and pluggable provider adapters (OpenAI, Anthropic, and more).
What is Langfuse?
Langfuse is an open source platform for LLM observability and monitoring. It helps you trace and monitor your AI applications by capturing metadata, prompt details, token usage, latency, and more.
Step 1: Install Dependencies
Install the necessary packages:
npm install @tanstack/ai @tanstack/ai-openai @arizeai/openinference-tanstack-ai @langfuse/otel @opentelemetry/sdk-nodeNote: This cookbook uses Deno.js for execution, which requires different syntax for importing packages and setting environment variables. For Node.js applications, the setup process is similar but uses standard
npmpackages andprocess.env.
Step 2: Configure Environment
Set up your Langfuse and OpenAI API keys. You can get Langfuse keys by signing up for a free Langfuse Cloud account or by self-hosting Langfuse. Get your OpenAI API key from the OpenAI Platform.
Deno.env.set("OPENAI_API_KEY", "sk-proj-...");
Deno.env.set("LANGFUSE_PUBLIC_KEY", "pk-lf-...");
Deno.env.set("LANGFUSE_SECRET_KEY", "sk-lf-...");
Deno.env.set("LANGFUSE_BASE_URL", "https://cloud.langfuse.com"); // 🇪🇺 EU region
// Other Langfuse data regions include 🇺🇸 US: https://us.cloud.langfuse.com, 🇯🇵 Japan: https://jp.cloud.langfuse.com and ⚕️ HIPAA: https://hipaa.cloud.langfuse.comStep 3: Initialize OpenTelemetry with Langfuse
Set up the OpenTelemetry SDK with the LangfuseSpanProcessor. The openInferenceMiddleware from OpenInference uses your application's existing OpenTelemetry tracer provider, so this setup must run before any TanStack AI calls.
We also provide a custom shouldExportSpan filter to include spans from the @arizeai/openinference-tanstack-ai instrumentation scope alongside the default Langfuse filter.
import { NodeSDK } from "npm:@opentelemetry/sdk-node";
import { LangfuseSpanProcessor, isDefaultExportSpan } from "npm:@langfuse/otel";
const sdk = new NodeSDK({
spanProcessors: [
new LangfuseSpanProcessor({
shouldExportSpan: ({ otelSpan }) =>
isDefaultExportSpan(otelSpan) ||
otelSpan.instrumentationScope.name ===
"@arizeai/openinference-tanstack-ai",
}),
],
});
sdk.start();Step 4: Run the Agent
Add openInferenceMiddleware() to the middleware option of TanStack AI's chat() call. All agent steps, tool calls, and model completions are automatically traced and sent to Langfuse. The middleware works for both streaming and non-streaming calls.
import { chat } from "npm:@tanstack/ai";
import { openaiText } from "npm:@tanstack/ai-openai";
import { openInferenceMiddleware } from "npm:@arizeai/openinference-tanstack-ai";
const text = await chat({
adapter: openaiText("gpt-4o-mini"),
stream: false,
systemPrompts: ["You are a concise technical explainer."],
messages: [
{ role: "user", content: "What is the capital of France? Answer in a single sentence." },
],
middleware: [openInferenceMiddleware()],
});
console.log(text);
await sdk.shutdown();Step 5: View Traces in Langfuse
After running the agent, navigate to your Langfuse Trace Table. You will find detailed traces of the agent's execution, providing insights into every agent step, tool call, input, output, and performance metric.
Interoperability with the JS/TS SDK
You can use this integration together with the Langfuse SDKs to add additional attributes or group observations into a single trace.
The Context Manager allows you to wrap your instrumented code using context managers (with with statements), which allows you to add additional attributes to the trace. Any observation created inside the callback will automatically be nested under the active observation, and the observation will be ended when the callback finishes.
import { startActiveObservation, propagateAttributes } from "npm:@langfuse/tracing";
await startActiveObservation("context-manager", async (span) => {
span.update({
input: { query: "What is the capital of France?" },
});
// Propagate userId to all child observations
await propagateAttributes(
{
userId: "user-123",
sessionId: "session-123",
metadata: {
source: "api",
region: "us-east-1",
},
tags: ["api", "user"],
version: "1.0.0",
},
async () => {
// YOUR CODE HERE
const { text } = await generateText({
model: openai("gpt-5"),
prompt: "What is the capital of France?",
experimental_telemetry: { isEnabled: true },
});
}
);
span.update({ output: "Paris" });
});Learn more about using the Context Manager in the Langfuse SDK instrumentation docs.
The observe wrapper is a powerful tool for tracing existing functions without modifying their internal logic. It acts as a decorator that automatically creates a span or generation around the function call. You can use the propagateAttributes function to add attributes to the observation from within the wrapped function.
import { observe, propagateAttributes } from "@langfuse/tracing";
import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";
// An existing function
const processUserRequest = observe(
async (userQuery: string) => {
// Propagate attributes to all child observations
return await propagateAttributes(
{
userId: "user-123",
sessionId: "session-123",
metadata: {
source: "api",
region: "us-east-1",
},
tags: ["api", "user"],
version: "1.0.0",
},
async () => {
// YOUR CODE HERE
const { text } = await generateText({
model: openai("gpt-5"),
prompt: userQuery,
experimental_telemetry: { isEnabled: true },
});
return text;
}
);
},
{ name: "process-user-request" }
);
const result = await processUserRequest("some query");Learn more about using the Decorator in the Langfuse SDK instrumentation docs.
Troubleshooting
No traces appearing
First, enable debug mode in the JS/TS SDK:
export LANGFUSE_LOG_LEVEL="DEBUG"Then run your application and check the debug logs:
- OTel spans appear in the logs: Your application is instrumented correctly but traces are not reaching Langfuse. To resolve this:
- Call
forceFlush()at the end of your application to ensure all traces are exported. This is especially important in short-lived environments like serverless functions. - Verify that you are using the correct API keys and base URL.
- Call
- No OTel spans in the logs: Your application is not instrumented correctly. Make sure the instrumentation runs before your application code.
Unwanted observations in Langfuse
The Langfuse SDK is based on OpenTelemetry. Other libraries in your application may emit OTel spans that are not relevant to you. These still count toward your billable units, so you should filter them out. See Unwanted spans in Langfuse for details.
Missing attributes
Some attributes may be stored in the metadata object of the observation rather than being mapped to the Langfuse data model. If a mapping or integration does not work as expected, please raise an issue on GitHub.
Next Steps
Once you have instrumented your code, you can manage, evaluate and debug your application:
Manage prompts in Langfuse
Add evaluation scores
Run LLM-as-a-judge Evaluators
Create datasets
Create custom dashboards
Test queries in the Playground
Last edited
