Self-host Langfuse

Langfuse consists of two application containers, storage components, and an optional LLM API/Gateway.

• Application Containers
  • Langfuse Web: The main web application serving the Langfuse UI and APIs.
  • Langfuse Worker: A worker that asynchronously processes events.
• Storage Components:
  • Postgres: The main database for transactional workloads.
  • Clickhouse: High-performance OLAP database which stores traces, observations, and scores.
  • Redis/Valkey cache: A fast in-memory data structure store. Used for queue and cache operations.
  • S3/Blob Store: Object storage to persist all incoming events, multi-modal inputs, and large exports.
• LLM API / Gateway: Some features depend on an external LLM API or gateway.

Langfuse can be deployed within a VPC or on-premises in high-security environments. Internet access is optional. See networking documentation for more details.

Some of the optimizations include:

• Queued trace ingestion: All traces are received in batches by the Langfuse Web container and immediately written to S3. Only a reference is persisted in Redis for queueing. Afterwards, the Langfuse Worker will pick up the traces from S3 and ingest them into Clickhouse. This ensures that high spikes in request load do not lead to timeouts or errors constrained by the database.
• Caching of API keys: API keys are cached in-memory in Redis. Thereby, the database is not hit on every API call and unauthorized requests can be rejected with very low resource usage.
• Caching of prompts (SDKs and API): Even though prompts are cached client-side by the Langfuse SDKs and only revalidated in the background (docs), they need to be fetched from the Langfuse on first use. Thus, API response times are very important. Prompts are cached in a read-through cache in Redis. Thereby, hot prompts can be fetched from Langfuse without hitting a database.
• Hyper-optimized ClickHouse schema: Read-heavy analytical operations use a wide, mostly immutable observations table that avoids read-time joins and deduplication while taking advantage of ClickHouse's most performant access patterns. Read how we simplified Langfuse for scale.
• Multi-modal traces in S3: Multi-modal traces can include large videos or arbitrary files. To enable support for these, they are directly uploaded to S3/Blob Storage from the client SDKs. Learn more here.
• Recoverability of events: All incoming tracing and evaluation events are persisted in S3/Blob Storage first. Only after successful processing, the events are written to the database. This ensures that even if the database is temporarily unavailable, the events are not lost and can be processed later.
• Background migrations: Long-running migrations that are required by an upgrade but not blocking for regular operations are offloaded to a background job. This massively reduces the downtime during an upgrade. Learn more here.