Langfuse Assistant

Configure the model

AI features support Amazon Bedrock's Converse API, the Anthropic Messages API, and OpenAI-compatible Chat Completions or Responses APIs.

Set LANGFUSE_AI_PROVIDER and LANGFUSE_AI_MODEL on both web and worker. LANGFUSE_AI_SMALL_MODEL is optional:

• LANGFUSE_AI_PROVIDER selects bedrock, anthropic, or openai.
• LANGFUSE_AI_MODEL sets the primary model used by the Assistant.
• LANGFUSE_AI_SMALL_MODEL sets an optional model for supplementary calls, such as conversation titles. If unset, these calls use LANGFUSE_AI_MODEL, which means paying the primary model's price for them. Langfuse Cloud pairs Claude Opus 5 with Claude Haiku 4.5.

Recommended model: Claude Opus 5. The Assistant's system prompt and tool loop are developed against it. Other models work, but tool calling and instruction following may vary.

An incomplete provider configuration counts as unconfigured. The UI then reports that the model is not configured.

# Required for Amazon Bedrock
LANGFUSE_AI_PROVIDER=bedrock
LANGFUSE_AI_MODEL=eu.anthropic.claude-opus-5

# Optional
LANGFUSE_AI_SMALL_MODEL=eu.anthropic.claude-haiku-4-5-20251001-v1:0
LANGFUSE_AI_AWS_BEDROCK_REGION=eu-west-1

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream"
      ],
      "Resource": "*"
    }
  ]
}

# Required for Anthropic
LANGFUSE_AI_PROVIDER=anthropic
LANGFUSE_AI_MODEL=claude-opus-5
LANGFUSE_AI_API_KEY=sk-ant-...

# Optional
LANGFUSE_AI_SMALL_MODEL=claude-haiku-4-5
# Defaults to https://api.anthropic.com/v1; include /v1 for custom endpoints
# LANGFUSE_AI_BASE_URL=https://gateway.internal/v1
# LANGFUSE_AI_EXTRA_HEADERS={"X-Gateway-Token":"..."}

# Required for OpenAI-compatible providers
LANGFUSE_AI_PROVIDER=openai
LANGFUSE_AI_MODEL=gpt-5.6-sol
LANGFUSE_AI_API_KEY=...

# Optional
LANGFUSE_AI_SMALL_MODEL=gpt-5.6-luna
# Defaults to https://api.openai.com/v1; required for other endpoints
# LANGFUSE_AI_BASE_URL=https://your-gateway.internal/v1
# LANGFUSE_AI_EXTRA_HEADERS={"X-Gateway-Token":"..."}
# Unset uses Responses for api.openai.com and Chat Completions elsewhere
# LANGFUSE_AI_USE_RESPONSES_API=true

Configure MCP

The Assistant reaches Langfuse data through this instance's native Langfuse MCP server. There is no separate MCP process to install.

The worker uses LANGFUSE_MCP_BASE_URL when set, otherwise NEXTAUTH_URL, and appends /api/public/mcp. Keep NEXTAUTH_URL set to the public Langfuse URL. Langfuse also uses it for links in emails, Slack notifications, and integration settings.

Set LANGFUSE_MCP_BASE_URL on the worker when it cannot reach the public URL. If that URL uses a different hostname than NEXTAUTH_URL, add the hostname to LANGFUSE_MCP_ALLOWED_HOSTS on web.

After applying the variables, confirm that the worker can resolve the hostname and reach /api/public/mcp on web. A 403 response with Invalid Host header means the hostname is missing from LANGFUSE_MCP_ALLOWED_HOSTS.

Configure a sandbox (optional)

We recommend completing this step for production deployments on AWS. The sandbox enables file and code execution for large-scale analysis while isolating that code from the Langfuse worker. It requires AWS resources in your account, an image built from the Langfuse release, and worker configuration.

Without a sandbox, the Assistant still works with Langfuse tools, but file and code-execution tools are unavailable.

Create the AWS resources

The Langfuse Terraform module for AWS (>=1.2.0) creates all of these and renders the worker configuration. Set enable_agent_sandbox_microvm = true, then continue with Build the MicroVM image.

If you do not use the module, create these resources before configuring the worker:

The build and execution roles are separate.

Build the MicroVM image

Langfuse does not publish a pullable MicroVM image. AWS builds it in your account from a zip artifact and an Amazon Linux 2023 base image. You need Lambda MicroVMs in the Region, an AWS CLI with lambda-microvms support (2.36.1 is known to work), Docker, pnpm, zip, git, and a checkout of the Langfuse repository.

The identity that runs the build script needs s3:PutObject on the artifact bucket, lambda:CreateMicrovmImage, ListMicrovmImages, GetMicrovmImage, UpdateMicrovmImage, and iam:PassRole on the build role.

Run build-microvm-image.sh from the Langfuse repository root:

export AWS_PROFILE=<profile>
export AWS_REGION=<region>
export S3_BUCKET=<artifact-bucket>
export MICROVM_IMAGE_NAME=langfuse-in-app-agent-sandbox
export LAMBDA_MICROVM_BUILD_ROLE_ARN=arn:aws:iam::<account-id>:role/<build-role>
export BASE_IMAGE_ARN=arn:aws:lambda:<region>:aws:microvm-image:al2023-1
export BASE_IMAGE_VERSION=0

bash packages/in-app-agent-sandbox-runtime/build-microvm-image.sh

If you created the resources with the Terraform module, terraform output agent_sandbox_build_env prints these values.

On success, the script prints IMAGE_ARN. Use it as LANGFUSE_IN_APP_AGENT_SANDBOX_AWS_LAMBDA_MICROVM_IMAGE_IDENTIFIER.

Configure IAM and network isolation

Grant the worker permission to run the image, pass the execution role, and pass the two connectors Langfuse uses. HTTP_INGRESS carries the worker's tool calls into the guest. ALL_INGRESS is a different named connector and is not used by Langfuse.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "lambda:RunMicrovm",
        "lambda:GetMicrovm",
        "lambda:CreateMicrovmAuthToken",
        "lambda:ResumeMicrovm",
        "lambda:SuspendMicrovm",
        "lambda:TerminateMicrovm"
      ],
      "Resource": "arn:aws:lambda:<region>:<account-id>:microvm-image:<image-name>"
    },
    {
      "Effect": "Allow",
      "Action": "iam:PassRole",
      "Resource": "arn:aws:iam::<account-id>:role/<execution-role>"
    },
    {
      "Effect": "Allow",
      "Action": "lambda:PassNetworkConnector",
      "Resource": [
        "arn:aws:lambda:<region>:aws:network-connector:aws-network-connector:HTTP_INGRESS",
        "arn:aws:lambda:<region>:<account-id>:network-connector:<egress-connector-name>"
      ]
    }
  ]
}

See Lambda MicroVM networking.

Configure the worker

Set the provider, image identifier, execution role, and Region on the worker. A partial configuration of these four variables fails at worker boot. Add the egress network connector for production isolation.

LANGFUSE_IN_APP_AGENT_SANDBOX_PROVIDER=lambda-microvm
LANGFUSE_IN_APP_AGENT_SANDBOX_AWS_LAMBDA_MICROVM_IMAGE_IDENTIFIER=arn:aws:lambda:<region>:<account-id>:microvm-image:<image-name>
LANGFUSE_IN_APP_AGENT_SANDBOX_AWS_LAMBDA_MICROVM_EXECUTION_ROLE_ARN=arn:aws:iam::<account-id>:role/<execution-role>
LANGFUSE_IN_APP_AGENT_SANDBOX_AWS_LAMBDA_MICROVM_REGION=<region>
# Strongly recommended for production isolation
LANGFUSE_IN_APP_AGENT_SANDBOX_AWS_LAMBDA_MICROVM_EGRESS_NETWORK_CONNECTOR_ARN=arn:aws:lambda:<region>:<account-id>:network-connector:<egress-connector-name>

Enable and verify the Assistant

After the model and MCP connection work, and after any optional sandbox setup, enable the Assistant on web and worker:

LANGFUSE_IN_APP_AGENT_ENABLED=true

Unset or false hides the Assistant even when AI Features is enabled for the organization. Workers with this variable consume the Assistant run queue. Leave it unset on ingestion-only workers.

Then:

1. Sign in as an organization owner or administrator.
2. Open organization Settings and enable AI Features.
3. Open the Langfuse Assistant in a project.
4. Ask it to summarize a recent trace and confirm the run moves from queued to running to complete.
5. If you configured a sandbox, ask it to write a file.