--- title: Halo (self-hosted) description: Configure the model, worker MCP access, and optional sandbox for the self-hosted Halo. label: "Version: v4" sidebarTitle: "Halo" --- # Halo This guide covers the infrastructure setup for the self-hosted Halo. For an introduction to the feature, see [Halo](/docs/halo). ## Availability and requirements [#requirements] Halo is in public beta on self-hosted deployments from version `>=v4.28.0`. It is not covered by SemVer, and its configuration may change between releases. [Subscribe to Langfuse OSS updates](/self-hosting/oss-newsletter) to hear when it leaves beta. To use Halo, provide access to a supported model provider: Amazon Bedrock, the Anthropic Messages API, an OpenAI-compatible endpoint, or Google Vertex AI (now Gemini Enterprise Agent Platform). Langfuse uses one instance-wide provider and model configuration, applied to both web and worker. Vertex support is available from `>=v4.39.0`. ## Choose your setup [#choose-your-setup] We recommend the AWS Lambda MicroVM sandbox. It enables file and code-execution tools, which let Halo process large sets of traces and observations with code instead of loading every item into the model context. | Setup | Platform | Available tools | Operational cost | | ------------------------------------ | ------------ | ----------------------------------------- | ---------------------------------------------------- | | AWS Lambda MicroVM (**recommended**) | AWS | Langfuse tools, files, and code execution | AWS resources, image builds, and upgrade maintenance | | Without a sandbox | Any platform | Langfuse tools | No additional infrastructure | Without a sandbox, file and code-execution tools are disabled. Halo can use only its built-in tools, including Langfuse tools exposed through MCP. Choose this setup for non-AWS deployments or when you do not want to operate the additional AWS infrastructure. ## Infrastructure overview [#what-runs-where] Halo runs are durable jobs. The web service admits and enqueues them, and the worker claims and executes them. Web also calls the model directly for [Ask AI](/docs/observability/features/filter-search-bar#ask-ai) and conversation titles. | Component | Status | Responsibility | | ------------------------------------------------------------ | ---------------------------- | ------------------------------------------------------------------------------------------ | | Langfuse Web | Existing | Enqueues runs, hosts the MCP server, and uses the model for Ask AI and conversation titles | | Langfuse Worker | Existing | Executes Halo runs and, when configured, starts the sandbox | | Langfuse AI model | New, required dependency | Serves model requests from web and worker | | Execution sandbox | New, optional infrastructure | Isolates file and code execution from the worker | ## Set up Halo [#setup-overview] The steps below use environment variables, which apply to every deployment. The Langfuse Helm chart and the AWS Terraform module expose the same settings as first-class values and place them on the right service for you. See [Deployment recipes](#deployment-recipes). ### Configure the model [#configure-the-model] [AI features](/security/ai-features) support Amazon Bedrock's Converse API, the Anthropic Messages API, OpenAI-compatible Chat Completions or Responses APIs, and Google Vertex AI. Set `LANGFUSE_AI_PROVIDER` and `LANGFUSE_AI_MODEL` on both web and worker. `LANGFUSE_AI_SMALL_MODEL` is optional: - `LANGFUSE_AI_PROVIDER` selects `bedrock`, `anthropic`, `openai`, or `vertex`. - `LANGFUSE_AI_MODEL` sets the primary model used by Halo. - `LANGFUSE_AI_SMALL_MODEL` sets an optional model for supplementary calls, such as conversation titles. If unset, these calls use `LANGFUSE_AI_MODEL`, which means paying the primary model's price for them. Langfuse Cloud pairs Claude Opus 5 with Claude Haiku 4.5. **Recommended model: Claude Opus 5.** Halo's system prompt and tool loop are developed against it. Other models work, but tool calling and instruction following may vary. An incomplete provider configuration counts as unconfigured. The UI then reports that the model is not configured. **Amazon Bedrock** ```bash # Required for Amazon Bedrock LANGFUSE_AI_PROVIDER=bedrock LANGFUSE_AI_MODEL=eu.anthropic.claude-opus-5 # Optional LANGFUSE_AI_SMALL_MODEL=eu.anthropic.claude-haiku-4-5-20251001-v1:0 LANGFUSE_AI_AWS_BEDROCK_REGION=eu-west-1 ``` Bedrock authenticates through the AWS credential chain rather than `LANGFUSE_AI_API_KEY`. Before configuring Langfuse, activate both models from the Bedrock model catalog. The first invocation of a third-party model starts an AWS Marketplace subscription, and Anthropic models also require the first-time-use form. Use an administrator identity to complete this once so the Langfuse runtime role can remain invoke-only. See [Request access to models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html), and confirm each model works in the Bedrock playground. IDs such as `eu.anthropic.claude-opus-5` are geo inference profiles rather than on-demand foundation-model IDs. Use the profile ID as the model value. The Region is optional when the workload already runs in the Bedrock Region. Langfuse resolves AWS credentials through the [default credential chain](https://docs.aws.amazon.com/sdkref/latest/guide/standardized-credentials.html). Prefer an ECS task role, EC2 instance role, or IRSA on Kubernetes so both services use temporary credentials. Grant invoke-only permissions: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream" ], "Resource": "*" } ] } ``` **Anthropic Messages** ```bash # Required for Anthropic LANGFUSE_AI_PROVIDER=anthropic LANGFUSE_AI_MODEL=claude-opus-5 LANGFUSE_AI_API_KEY=sk-ant-... # Optional LANGFUSE_AI_SMALL_MODEL=claude-haiku-4-5 # Defaults to https://api.anthropic.com/v1; include /v1 for custom endpoints # LANGFUSE_AI_BASE_URL=https://gateway.internal/v1 # LANGFUSE_AI_EXTRA_HEADERS={"X-Gateway-Token":"..."} ``` When `LANGFUSE_AI_BASE_URL` is unset, Langfuse calls `https://api.anthropic.com/v1`. For a custom Anthropic-compatible endpoint, set the full API base including `/v1`. `LANGFUSE_AI_EXTRA_HEADERS` must be a valid JSON object when set. Invalid JSON fails validation at boot. **OpenAI-compatible** ```bash # Required for OpenAI-compatible providers LANGFUSE_AI_PROVIDER=openai LANGFUSE_AI_MODEL=gpt-5.6-sol LANGFUSE_AI_API_KEY=... # Optional LANGFUSE_AI_SMALL_MODEL=gpt-5.6-luna # Defaults to https://api.openai.com/v1; required for other endpoints # LANGFUSE_AI_BASE_URL=https://your-gateway.internal/v1 # LANGFUSE_AI_EXTRA_HEADERS={"X-Gateway-Token":"..."} # Unset uses Responses for api.openai.com and Chat Completions elsewhere # LANGFUSE_AI_USE_RESPONSES_API=true ``` Use this provider for OpenAI and OpenAI-compatible gateways and proxies. When `LANGFUSE_AI_BASE_URL` is unset, Langfuse calls `https://api.openai.com/v1`. Set it, including `/v1`, for any other OpenAI-compatible endpoint. `LANGFUSE_AI_EXTRA_HEADERS` must be a valid JSON object when set. Invalid JSON fails validation at boot. The model must support OpenAI-style tool calling. When `LANGFUSE_AI_USE_RESPONSES_API` is unset, Langfuse uses Responses for `api.openai.com` and Chat Completions for other endpoints. Set it to `true` or `false` to override that choice. Responses requires the endpoint to serve `POST {base}/responses`. **Google Vertex AI** ```bash # Required for Google Vertex AI LANGFUSE_AI_PROVIDER=vertex LANGFUSE_AI_MODEL=claude-opus-5 # Optional LANGFUSE_AI_SMALL_MODEL=gemini-3.7-flash LANGFUSE_AI_VERTEX_LOCATION=global ``` Vertex (now [Gemini Enterprise Agent Platform](https://docs.cloud.google.com/gemini-enterprise-agent-platform/vertex-ai-name-changes)) authenticates through Google [application default credentials](https://cloud.google.com/docs/authentication/application-default-credentials) rather than `LANGFUSE_AI_API_KEY`. Langfuse reads the GCP project from those credentials, so there is no project-id env. Project-scoped LLM connections in the UI do not apply to Halo or Ask AI. Before configuring Langfuse, enable the Vertex AI API on the GCP project. For Claude, also enable the model in Model Garden. The first use of a partner model requires an administrator to accept the terms. See [Grant user access to partner models](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-partner-models#grant-user-access-to-partner-models), and confirm each model works. Both Claude and Gemini model IDs are accepted, for example `claude-opus-5` and `gemini-3.7-flash`. The location defaults to `global`. Set a Region such as `us-central1` when you need a regional or multi-region endpoint. See [Regional, global, and multi-region endpoints](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-partner-models#regional-global-and-multi-region-endpoints). Langfuse resolves credentials through application default credentials. Prefer a Cloud Run runtime service account or [GKE Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) so both services use temporary credentials. Grant `aiplatform.endpoints.predict`, which is included in the Agent Platform User role (`roles/aiplatform.user`). ### Configure MCP [#langfuse-mcp] Halo reaches Langfuse data through this instance's native [Langfuse MCP server](/docs/api-and-data-platform/features/mcp-server#set-up). There is no separate MCP process to install. The worker uses `LANGFUSE_MCP_BASE_URL` when set, otherwise `NEXTAUTH_URL`, and appends `/api/public/mcp`. Keep `NEXTAUTH_URL` set to the public Langfuse URL. Langfuse also uses it for links in emails, Slack notifications, and integration settings. Set `LANGFUSE_MCP_BASE_URL` on the worker when it cannot reach the public URL. If that URL uses a different hostname than `NEXTAUTH_URL`, add the hostname to `LANGFUSE_MCP_ALLOWED_HOSTS` on web. After applying the variables, confirm that the worker can resolve the hostname and reach `/api/public/mcp` on web. A `403` response with `Invalid Host header` means the hostname is missing from `LANGFUSE_MCP_ALLOWED_HOSTS`. ### Configure a sandbox (optional) [#sandbox] We recommend completing this step for production deployments on AWS. The sandbox enables file and code execution for large-scale analysis while isolating that code from the Langfuse worker. It requires AWS resources in your account, an image built from the Langfuse release, and worker configuration. Without a sandbox, Halo still works with Langfuse tools, but file and code-execution tools are unavailable. There is no non-AWS sandbox yet. Tell us what support you need in [GitHub Discussions](https://github.com/orgs/langfuse/discussions). #### Create the AWS resources [#create-the-aws-resources] The [Langfuse Terraform module for AWS](https://github.com/langfuse/langfuse-terraform-aws) (`>=1.2.0`) creates all of these and renders the worker configuration. Set `enable_agent_sandbox_microvm = true`, then continue with [Build the MicroVM image](#build-the-microvm-image). If you do not use the module, create these resources before configuring the worker: | Resource | Purpose | Notes | | ------------------------ | ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | | S3 bucket | Holds the zip artifact used during image builds | Keep it for image rebuilds during upgrades | | MicroVM build role | Lambda assumes it during `CreateMicrovmImage` | Trust `lambda.amazonaws.com` with `sts:AssumeRole` and `sts:TagSession`; allow artifact reads and build log writes | | MicroVM execution role | Identity used by the MicroVM guest | Trust `lambda.amazonaws.com` with `sts:AssumeRole` and `sts:TagSession`; attach no policies | | Security group | Denies guest traffic | Configure no ingress or egress rules | | Subnets | Carry the egress connector | Use private or unrouted subnets with no default route to the internet | | Connector operator role | Creates and operates the network connector | Trust `lambda.amazonaws.com` and `network-connectors.lambda.amazonaws.com` with `sts:AssumeRole` and `sts:TagSession` | | Egress network connector | Leaves the guest without network access | Use a VPC egress connector associated with MicroVM compute, the security group, and the subnets | The build and execution roles are separate. User-provided sandbox code runs as the execution role. Do not grant this role access to Langfuse databases, storage, or secrets. #### Build the MicroVM image [#build-the-microvm-image] Langfuse does not publish a pullable MicroVM image. AWS builds it in your account from a zip artifact and an Amazon Linux 2023 base image. You need Lambda MicroVMs in the Region, an AWS CLI with `lambda-microvms` support (2.36.1 is known to work), Docker, pnpm, zip, git, and a checkout of the Langfuse repository. The identity that runs the build script needs `s3:PutObject` on the artifact bucket, `lambda:CreateMicrovmImage`, `ListMicrovmImages`, `GetMicrovmImage`, `UpdateMicrovmImage`, and `iam:PassRole` on the build role. Run [`build-microvm-image.sh`](https://github.com/langfuse/langfuse/blob/main/packages/in-app-agent-sandbox-runtime/build-microvm-image.sh) from the Langfuse repository root: ```bash export AWS_PROFILE= export AWS_REGION= export S3_BUCKET= export MICROVM_IMAGE_NAME=langfuse-in-app-agent-sandbox export LAMBDA_MICROVM_BUILD_ROLE_ARN=arn:aws:iam:::role/ export BASE_IMAGE_ARN=arn:aws:lambda::aws:microvm-image:al2023-1 export BASE_IMAGE_VERSION=0 bash packages/in-app-agent-sandbox-runtime/build-microvm-image.sh ``` If you created the resources with the Terraform module, `terraform output agent_sandbox_build_env` prints these values. On success, the script prints `IMAGE_ARN`. Use it as `LANGFUSE_IN_APP_AGENT_SANDBOX_AWS_LAMBDA_MICROVM_IMAGE_IDENTIFIER`. #### Configure IAM and network isolation [#iam-and-network-isolation] Grant the worker permission to run the image, pass the execution role, and pass the two connectors Langfuse uses. `HTTP_INGRESS` carries the worker's tool calls into the guest. `ALL_INGRESS` is a different named connector and is not used by Langfuse. ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "lambda:RunMicrovm", "lambda:GetMicrovm", "lambda:CreateMicrovmAuthToken", "lambda:ResumeMicrovm", "lambda:SuspendMicrovm", "lambda:TerminateMicrovm" ], "Resource": "arn:aws:lambda:::microvm-image:" }, { "Effect": "Allow", "Action": "iam:PassRole", "Resource": "arn:aws:iam:::role/" }, { "Effect": "Allow", "Action": "lambda:PassNetworkConnector", "Resource": [ "arn:aws:lambda::aws:network-connector:aws-network-connector:HTTP_INGRESS", "arn:aws:lambda:::network-connector:" ] } ] } ``` See [Lambda MicroVM networking](https://docs.aws.amazon.com/lambda/latest/dg/microvms-networking.html). #### Configure the worker Set the provider, image identifier, execution role, and Region on the worker. A partial configuration of these four variables fails at worker boot. Add the egress network connector for production isolation. ```bash LANGFUSE_IN_APP_AGENT_SANDBOX_PROVIDER=lambda-microvm LANGFUSE_IN_APP_AGENT_SANDBOX_AWS_LAMBDA_MICROVM_IMAGE_IDENTIFIER=arn:aws:lambda:::microvm-image: LANGFUSE_IN_APP_AGENT_SANDBOX_AWS_LAMBDA_MICROVM_EXECUTION_ROLE_ARN=arn:aws:iam:::role/ LANGFUSE_IN_APP_AGENT_SANDBOX_AWS_LAMBDA_MICROVM_REGION= # Strongly recommended for production isolation LANGFUSE_IN_APP_AGENT_SANDBOX_AWS_LAMBDA_MICROVM_EGRESS_NETWORK_CONNECTOR_ARN=arn:aws:lambda:::network-connector: ``` Production isolation requires the egress network connector. Without it, AWS attaches its default `INTERNET_EGRESS` connector and sandboxed code can reach the public internet. An invalid connector ARN fails the MicroVM launch. ### Enable and verify Halo [#turn-on-the-assistant] After the model and MCP connection work, and after any optional sandbox setup, enable Halo on web and worker: ```bash LANGFUSE_IN_APP_AGENT_ENABLED=true ``` Unset or `false` hides Halo even when **AI Features** is enabled for the organization. Workers with this variable consume Halo's run queue. Leave it unset on ingestion-only workers. Then: 1. Sign in as an organization owner or administrator. 2. Open organization **Settings** and enable **AI Features**. 3. Open Halo in a project. 4. Ask it to summarize a recent trace and confirm the run moves from queued to running to complete. 5. If you configured a sandbox, ask it to write a file. Self-hosted deployment telemetry reports `assistantRuns`, the number of Halo runs started in the reporting window. It does not include Halo conversations, prompts, or project data. Langfuse OSS deployments can opt out by setting `TELEMETRY_ENABLED=false` on all application containers. Enterprise self-hosted telemetry is required for license compliance. See [Self-hosted telemetry](/self-hosting/security/telemetry) for the complete field list. ## Deployment recipes [#deployment-recipes] ### Docker Compose Add the model, activation, and MCP variables to `.env`. The shipped Compose file forwards each value to the correct service. ```bash filename=".env" LANGFUSE_AI_PROVIDER= LANGFUSE_AI_MODEL= # Optional, a cheaper model for supplementary calls such as conversation titles LANGFUSE_AI_SMALL_MODEL= # Required for Anthropic and OpenAI-compatible providers LANGFUSE_AI_API_KEY= LANGFUSE_MCP_BASE_URL=http://langfuse-web:3000 LANGFUSE_MCP_ALLOWED_HOSTS=langfuse-web LANGFUSE_IN_APP_AGENT_ENABLED=true ``` Recreate both services after changing `.env`. A restart does not re-interpolate environment variables. ### Kubernetes The [Langfuse Helm chart](/self-hosting/deployment/kubernetes-helm) exposes Halo under `langfuse.aiFeatures` from chart `v2.1.0`. The chart renders each variable on the services that read it, so you do not split them between web and worker yourself. For Bedrock, attach the workload role with `langfuse.serviceAccount.annotations`. For Vertex on GKE, attach a Workload Identity service account the same way. ```yaml filename="values.yaml" langfuse: aiFeatures: provider: bedrock model: eu.anthropic.claude-opus-5 # Optional, a cheaper model for supplementary calls such as conversation titles smallModel: eu.anthropic.claude-haiku-4-5-20251001-v1:0 bedrockRegion: eu-west-1 inAppAgent: enabled: true mcp: useInternalWebUrl: true # Optional, requires the AWS resources from the sandbox step sandbox: provider: lambda-microvm imageIdentifier: arn:aws:lambda:eu-west-1::microvm-image:langfuse-in-app-agent-sandbox executionRoleArn: arn:aws:iam:::role/ region: eu-west-1 egressNetworkConnectorArn: arn:aws:lambda:eu-west-1::network-connector: serviceAccount: annotations: eks.amazonaws.com/role-arn: arn:aws:iam:::role/ ``` `inAppAgent.mcp.useInternalWebUrl` points the worker's MCP calls at the in-cluster web Service and allows that hostname on web. It sets `LANGFUSE_MCP_BASE_URL` and leaves `NEXTAUTH_URL` alone, which the worker also uses for links in emails and Slack notifications. The chart gives web and worker one shared ServiceAccount, so both services assume the same IAM role. Scope it to Bedrock invoke and, if used, the MicroVM permissions above. For Vertex, grant `aiplatform.endpoints.predict` on that identity. For settings the chart does not expose, use `langfuse.additionalEnv`, which applies to web and worker. Pass `LANGFUSE_AI_VERTEX_LOCATION` this way when you need a location other than `global`. ### AWS (Terraform) The [Langfuse Terraform module for AWS](https://github.com/langfuse/langfuse-terraform-aws) supports Halo from `v1.2.0`. It configures the model, grants Bedrock invoke on the Langfuse IAM role, and optionally creates the sandbox. The module deploys the Helm chart, so it fills in the values above for you. ```hcl module "langfuse" { # ... app_version = "4.28.0" # or newer enable_ai_features = true ai_features_provider = "bedrock" ai_features_model = "eu.anthropic.claude-opus-5" enable_in_app_agent = true # Optional, a cheaper model for supplementary calls such as conversation titles ai_features_small_model = "eu.anthropic.claude-haiku-4-5-20251001-v1:0" # Optional, isolated file and code execution enable_agent_sandbox_microvm = true } ``` The sandbox runs in an isolated VPC with no route to the Langfuse VPC or the internet, shared with the [code evaluator](/self-hosting/configuration/code-evaluators) runners. Each workload gets its own deny-all security group. The module does not build the MicroVM image, so run the [build script](#build-the-microvm-image) after `terraform apply`. ### Generic environment variables Apply the shared model configuration and activation variable to both services: ```bash LANGFUSE_AI_PROVIDER= LANGFUSE_AI_MODEL= # Optional, a cheaper model for supplementary calls such as conversation titles LANGFUSE_AI_SMALL_MODEL= LANGFUSE_IN_APP_AGENT_ENABLED=true ``` Add provider-specific `LANGFUSE_AI_*` variables to both services. Set `LANGFUSE_MCP_BASE_URL` only on worker and `LANGFUSE_MCP_ALLOWED_HOSTS` only on web. Add `LANGFUSE_IN_APP_AGENT_SANDBOX_*` variables only to workers that run the sandbox. ## Operations and reference [#operations-and-reference] ### Upgrade the sandbox image [Rebuild and republish](#build-the-microvm-image) the MicroVM image whenever you upgrade Langfuse. Build it from the same Langfuse release so its tools stay aligned with the worker. Keep the build artifact and AWS resources as part of the deployment. ### Trace Halo in your own instance [#trace-the-assistant-in-your-own-instance] To record Halo runs as traces, set `LANGFUSE_AI_FEATURES_PROJECT_ID` on web and worker to the ID of a project on the same Langfuse deployment: ```bash LANGFUSE_AI_FEATURES_PROJECT_ID= ``` Halo turns and Ask AI completions then appear as traces in that project, tagged with the source project. Langfuse writes them through its internal ingestion path, so no API keys or outbound requests are required. Leave the variable unset to record nothing. Use a dedicated project. Halo traces contain prompts, tool calls, model output, and data read from other projects. Anyone with access to the target project can read that data. ### Full configuration reference See [Halo environment variables](/self-hosting/configuration#langfuse-assistant) for defaults, limits, and optional settings. For common setup failures, see [Troubleshoot the self-hosted Halo](/faq/all/troubleshoot-self-hosted-halo). --- If you experience any issues when self-hosting Langfuse, please: 1. Check out [Troubleshooting & FAQ](/self-hosting/troubleshooting-and-faq) page. 2. Use [Ask AI](/docs/ask-ai) to get instant answers to your questions. 3. Ask the maintainers on [GitHub Discussions](/gh-support). 4. Create a bug report or feature request on [GitHub](/issues). Enterprise-grade support is available when self-hosting Langfuse. Learn more on our [pricing page](/pricing-self-host). --- ## Agent Instructions This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/self-hosting/configuration/halo.md`. ### Querying these docs If the answer is not on this page, query the documentation instead of guessing: - **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question: ```bash curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?" ``` - **Index of every page**: , with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt). ### Before writing Langfuse code - **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results. - **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application. - **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples. - **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx @langfuse/cli api `) to read or write traces, prompts, datasets, and scores from the terminal. Found an error in these docs? Please open an issue at .