This is a Jupyter notebook

Example: Langfuse Prompt Management with Langchain (JS)

Langfuse Prompt Management helps to version control and manage prompts collaboratively in one place.

This example demonstrates how to use Langfuse Prompt Management together with Langchain JS.

Set Up Environment

Get your Langfuse API keys by signing up for Langfuse Cloud or self-hosting Langfuse. You’ll also need your OpenAI API key.

Note: This cookbook uses Deno.js for execution, which requires different syntax for importing packages and setting environment variables. For Node.js applications, the setup process is similar but uses standard npm packages and process.env.

// Langfuse authentication keys
Deno.env.set("LANGFUSE_PUBLIC_KEY", "pk-lf-***");
Deno.env.set("LANGFUSE_SECRET_KEY", "sk-lf-***");
 
// Langfuse host configuration
// For US data region, set this to "https://us.cloud.langfuse.com"
Deno.env.set("LANGFUSE_HOST", "https://cloud.langfuse.com")
 
// Set environment variables using Deno-specific syntax
Deno.env.set("OPENAI_API_KEY", "sk-proj-***");

With the environment variables set, we can now initialize the langfuseSpanProcessor which is passed to the main OpenTelemetry SDK that orchestrates tracing.

// Import required dependencies
import 'npm:dotenv/config';
import { NodeSDK } from "npm:@opentelemetry/sdk-node";
import { LangfuseSpanProcessor } from "npm:@langfuse/otel";
 
// Export the processor to be able to flush it later
// This is important for ensuring all spans are sent to Langfuse
export const langfuseSpanProcessor = new LangfuseSpanProcessor({
    publicKey: process.env.LANGFUSE_PUBLIC_KEY!,
    secretKey: process.env.LANGFUSE_SECRET_KEY!,
    baseUrl: process.env.LANGFUSE_HOST ?? 'https://cloud.langfuse.com', // Default to cloud if not specified
    environment: process.env.NODE_ENV ?? 'development', // Default to development if not specified
  });
 
// Initialize the OpenTelemetry SDK with our Langfuse processor
const sdk = new NodeSDK({
  spanProcessors: [langfuseSpanProcessor],
});
 
// Start the SDK to begin collecting telemetry
// The warning about crypto module is expected in Deno and doesn't affect basic tracing functionality. Media upload features will be disabled, but all core tracing works normally
sdk.start();

The LangfuseClient provides additional functionality beyond OpenTelemetry tracing, such as scoring, prompt management, and data retrieval. It automatically uses the same environment variables we set earlier.

import { LangfuseClient } from "npm:@langfuse/client";
 
const langfuse = new LangfuseClient();

Example 1: Text Prompt

Add new prompt

We add the prompt used in this example via the SDK. Alternatively, you can also edit and version the prompt in the Langfuse UI.

Name that identifies the prompt in Langfuse Prompt Management
Prompt with topic variable
Config including modelName, temperature
labels to include production to immediately use prompt as the default

For the sake of this notebook, we will add the prompt in Langfuse and use it right away. Usually, you’d update the prompt from time to time in Langfuse and your application fetches the current production version.

// Create a text prompt
await langfuse.prompt.create({
    name: "jokes",
    type: "text",
    prompt: "Tell me a joke about {{topic}}",
    labels: ["production"], // directly promote to production
    config: {
      model: "gpt-4o",
      temperature: 0.7,
      supported_languages: ["en", "fr"],
    }, // optionally, add configs (e.g. model parameters or model tools) or tags
  });

Prompt in Langfuse

Run example

Get current prompt version from Langfuse

// Get current `production` version
const prompt = await langfuse.prompt.get("jokes");

The prompt includes the prompt string

prompt.prompt

and the config object

prompt.config

Transform prompt into Langchain PromptTemplate

Use the utility method .getLangchainPrompt() to transform the Langfuse prompt into a string that can be used in Langchain.

Context: Langfuse declares input variables in prompt templates using double brackets ({{input variable}}). Langchain uses single brackets for declaring input variables in PromptTemplates ({input variable}). The utility method .getLangchainPrompt() replaces the double brackets with single brackets.

Also, pass the Langfuse prompt as metadata to the PromptTemplate to automatically link generations that use the prompt.

import { PromptTemplate } from "npm:@langchain/core/prompts"
 
const langfuseTextPrompt = await langfuse.prompt.get("jokes"); // Fetch a previously created text prompt
 
// Pass the langfuseTextPrompt to the PromptTemplate as metadata to link it to generations that use it
const langchainTextPrompt = PromptTemplate.fromTemplate(
  langfuseTextPrompt.getLangchainPrompt()
).withConfig({
  metadata: { langfusePrompt: langfuseTextPrompt },
});

Setup Langfuse Tracing for Langchain JS

We’ll use the native Langfuse Tracing for Langchain JS when executing this chain. This is fully optional and can be used independently from Prompt Management.

import { CallbackHandler } from "npm:@langfuse/langchain";
 
// 1. Initialize the Langfuse callback handler
const langfuseHandler = new CallbackHandler({
  sessionId: "user-session-123",
  userId: "user-abc",
  tags: ["langchain-test"],
});

Create chain

We use the modelName and temperature stored in prompt.config.

import { ChatOpenAI } from "npm:@langchain/openai"
import { RunnableSequence } from "npm:@langchain/core/runnables";
 
const model = new ChatOpenAI({
    modelName: prompt.config.model,
    temperature: prompt.config.temperature
});
const chain = RunnableSequence.from([promptTemplate, model]);

Invoke chain

const res = await chain.invoke(
    { topic: "developers" },
    { callbacks: [langfuseHandler] }
);

View trace in Langfuse

As we passed the langfuse callback handler, we can explore the execution trace in Langfuse.

Trace in Langfuse

Public trace in the Langfuse UI

Example 1: OpenAI functions and JsonOutputFunctionsParser

Add prompt to Langfuse

await langfuse.prompt.create({
    name: "extractor",
    prompt: "Extracts fields from the input.",
    config: {
      modelName: "gpt-4o",
      temperature: 0,
      schema: {
        type: "object",
        properties: {
          tone: {
            type: "string",
            enum: ["positive", "negative"],
            description: "The overall tone of the input",
          },
          word_count: {
            type: "number",
            description: "The number of words in the input",
          },
          chat_response: {
            type: "string",
            description: "A response to the human's input",
          },
        },
        required: ["tone", "word_count", "chat_response"],
      }
    }, // optionally, add configs (e.g. model parameters or model tools)
    labels: ["production"] // directly promote to production
});

Prompt in Langfuse

Fetch prompt

const extractorPrompt = await langfuse.prompt.get("extractor")

Transform into schema

const extractionFunctionSchema = {
    name: "extractor",
    description: extractorPrompt.prompt,
    parameters: extractorPrompt.config.schema,
}

Build chain

import { ChatOpenAI } from "npm:@langchain/openai";
import { JsonOutputFunctionsParser } from "npm:langchain/output_parsers";
 
// Instantiate the parser
const parser = new JsonOutputFunctionsParser();
 
// Instantiate the ChatOpenAI class
const model = new ChatOpenAI({ 
    modelName: extractorPrompt.config.modelName,
    temperature: extractorPrompt.config.temperature
});
 
// Create a new runnable, bind the function to the model, and pipe the output through the parser
const runnable = model
  .bind({
    functions: [extractionFunctionSchema],
    function_call: { name: "extractor" },
  })
  .pipe(parser);

Invoke chain

import { HumanMessage } from "npm:@langchain/core/messages";
 
// Invoke the runnable with an input
const result = await runnable.invoke(
    [new HumanMessage("What a beautiful day!")],
    { callbacks: [langfuseHandler] }
);

View trace in Langfuse

Trace in Langfuse

Public trace in the Langfuse UI

JS/TS SDK Example Langfuse SDK Performance Test

Was this page helpful?

Support