Self-host Langfuse

Langfuse is open source and can be self-hosted using Docker. This section contains guides for different deployment scenarios.

Deployment Options

The following options are available:

• Langfuse Cloud: A fully managed version of Langfuse that is hosted and maintained by the Langfuse team.
• Self-host Langfuse: Run Langfuse on your own infrastructure.
  • Local: Run Langfuse on your own machine in 5 minutes using Docker Compose.
  • VM: Run Langfuse on a single VM using Docker Compose.
  • Docker
  • Kubernetes (Helm): Run Langfuse on a Kubernetes cluster using Helm.
  • Planned: Cloud-specific deployment guides, please upvote and comment on the following threads: AWS, Google Cloud, Azure.

Architecture

Langfuse only depends on open source components and can be deployed locally, on cloud infrastructure, or on-premises.

Langfuse consists of two application containers, storage components, and an optional LLM API/Gateway.

• Application Containers
  • Langfuse Web: The main web application serving the Langfuse UI and APIs.
  • Langfuse Worker: A worker that asynchronously processes events.
• Storage Components:
  • Postgres: The main database for transactional workloads.
  • Clickhouse: High-performance OLAP database which stores traces, observations, and scores.
  • Redis/Valkey cache: A fast in-memory data structure store. Used for queue and cache operations.
  • S3/Blob Store: Object storage to persist all incoming events, multi-modal inputs, and large exports.
• LLM API / Gateway: Some features depend on an external LLM API or gateway.

Langfuse can be deployed within a VPC or on-premises in high-security environments. Internet access is optional. See networking documentation for more details.

Optimized for performance, reliability, and uptime

Langfuse self-hosted is optimized for production environments. It is the exact same codebase as Langfuse Cloud, just deployed on your own infrastructure. The Langfuse teams serves thousands of teams with Langfuse Cloud with high availability (status page) and performance.

Some of the optimizations include:

• Queued trace ingestion: All traces are received in batches by the Langfuse Web container and immediately written to S3. Only a reference is persisted in Redis for queueing. Afterwards, the Langfuse Worker will pick up the traces from S3 and ingest them into Clickhouse. This ensures that high spikes in request load do not lead to timeouts or errors constrained by the database.
• Caching of API keys: API keys are cached in-memory in Redis. Thereby, the database is not hit on every API call and unauthorized requests can be rejected with very low resource usage.
• Caching of prompts (SDKs and API): Even though prompts are cached client-side by the Langfuse SDKs and only revalidated in the background (docs), they need to be fetched from the Langfuse on first use. Thus, API response times are very important. Prompts are cached in a read-through cache in Redis. Thereby, hot prompts can be fetched from Langfuse without hitting a database.
• OLAP database: All read-heavy analytical operations are offloaded to an OLAP database (Clickhouse) for fast query performance.
• Multi-modal traces in S3: Multi-modal traces can include large videos or arbitrary files. To enable support for these, they are directly uploaded to S3/Blob Storage from the client SDKs. Learn more here.
• Recoverability of events: All incoming tracing and evaluation events are persisted in S3/Blob Storage first. Only after successful processing, the events are written to the database. This ensures that even if the database is temporarily unavailable, the events are not lost and can be processed later.
• Background migrations: Long-running migrations that are required by an upgrade but not blocking for regular operations are offloaded to a background job. This massively reduces the downtime during an upgrade. Learn more here.

If you have any feedback or questions regarding the architecture, please reach out to us.

Features

Langfuse supports many configuration options and self-hosted features. For more details, please refer to the configuration guide.

Subscribe to updates

Release notes are published on GitHub. Langfuse uses tagged semver releases (versioning policy).

You can subscribe to our mailing list to get notified about new releases and new major versions.

You can also watch the GitHub releases to get notified about new releases:

Support

If you experience any issues, please join us on Discord or contact the maintainers at [email protected].

For support with production deployments, the Langfuse team provides dedicated enterprise support. To learn more, reach out to [email protected] or schedule a demo.

Alternatively, you may consider using Langfuse Cloud, which is a fully managed version of Langfuse. You can find information about its security and privacy here.

FAQ

• Are prebuilt ARM images available?
• Are older versions of the SDK compatible with newer versions of Langfuse?
• I cannot connect to my docker deployment, what should I do?
• I have forgotten my password
• How can I restrict access on my self-hosted instance to internal users?
• How to manage different environments in Langfuse?
• Can I deploy multiple instances of Langfuse behind a load balancer?
• What kind of telemetry does Langfuse collect?

GitHub Discussions