--- title: Scaling Langfuse Deployments description: Learn how to scale your self-hosted Langfuse deployment to handle more traffic and data. label: "Version: v4" sidebarTitle: "Sizing & Scaling" --- # Scaling This guide covers how you can operate your Langfuse deployment at scale and includes best practices and tweaks to get the best performance. ## Minimum Infrastructure Requirements [#minimum-infrastructure-requirements] | Service | Minimum Requirements | | ------------------------------------------------------------------------------- | ------------------------------------------------------------ | | [Langfuse Web Container](/self-hosting/deployment/infrastructure/containers) | 2 CPU, 4 GiB Memory | | [Langfuse Worker Container](/self-hosting/deployment/infrastructure/containers) | 2 CPU, 4 GiB Memory | | [PostgreSQL Database](/self-hosting/deployment/infrastructure/postgres) | 2 CPU, 4 GiB Memory | | [Redis/Valkey Instance](/self-hosting/deployment/infrastructure/cache) | 1 CPU, 1.5 GiB Memory | | [ClickHouse](/self-hosting/deployment/infrastructure/clickhouse) | 2 CPU, 8 GiB Memory | | [Blob Storage](/self-hosting/deployment/infrastructure/blobstorage) | Serverless (S3 or compatible) or MinIO (2 CPU, 4 GiB Memory) | ## Ingestion Throughput Langfuse is designed to handle a large amount and volume of ingested data. On very high loads, it may become necessary to apply additional settings that influence the throughput. ### Scaling the worker containers For most environments, we recommend to scale the worker containers by their CPU load as this is a straightforward metric to measure. A load above 50% for a 2 CPU container is an indicator that the instance is saturated and that the throughput should increase by adding more containers. In addition, the Langfuse worker publishes tagged queue depth metrics via StatsD that can be used to scale the worker containers. `langfuse.queue.ingestion.depth` with the tag `type:waiting` is the main metric that we use to make scaling decisions. The queue metrics can also be published to AWS CloudWatch by setting `ENABLE_AWS_CLOUDWATCH_METRIC_PUBLISHING=true` to configure auto-scalers based on AWS metrics. Starting with Langfuse [3.170.0](https://github.com/langfuse/langfuse/releases/tag/v3.170.0), tagged queue metrics ending in `.depth`, `.rate`, or `.dlq_oldest_age` are published in the `Langfuse` CloudWatch namespace with every tag except `unit` flattened into the metric name as a `.tag_value` segment, sorted by tag name, instead of publishing the tags as dimensions. Therefore, the StatsD metric and tag above are published to CloudWatch as `langfuse.queue.ingestion.depth.type_waiting`.
Queue metric compatibility for Langfuse 3.169.0 and earlier Langfuse versions before [3.169.0](https://github.com/langfuse/langfuse/releases/tag/v3.169.0) do not emit the tagged queue depth metric. Use the legacy `langfuse.queue.ingestion.length` metric when configuring autoscaling for these versions. Langfuse 3.169.0 emits the tagged depth metric to StatsD, but its CloudWatch publisher does not flatten the tags and therefore does not provide a distinct waiting-depth series. Upgrade to 3.170.0 or later before using the queue depth metric for CloudWatch autoscaling.
### Reducing ClickHouse reads within the ingestion processing Per default, the Langfuse worker reads the existing event from ClickHouse and merges it with any incoming data. This increases the load on ClickHouse and may limit the total throughput. For projects that were not migrated from a previous version of Langfuse, this is optional as the full event history is available in S3. You can set `LANGFUSE_SKIP_INGESTION_CLICKHOUSE_READ_MIN_PROJECT_CREATE_DATE` to a date in the past before your first project was created, e.g. `2025-01-01`. Please note that any S3/blob storage deletion lifecycle rules in combination with late updates to events may cause duplicates in the event history. If you use the default integration methods with the Langfuse SDKs or OpenTelemetry this should not affect you. ### Separating ingestion and user interface When the ingestion load is high, the Langfuse web interface and API calls may become slow or unresponsive. In this case, splitting the langfuse-web deployment into one ingestion handling and one user interface handling deployment can help to keep the user interface responsive. You can create a new identical replica of the langfuse-web deployment and route all traffic to `/api/public/ingestion*`, `/api/public/media*`, and `/api/public/otel*` to the new deployment. ### Increasing S3 (Blobstorage) Write Concurrency The blob storage backend is used to store raw events, multi-modal inputs, batch exports, and other files. In very high throughput scenarios the number of allowed sockets from the S3 client library may be exhausted and requests are being throttled. If this happens, we usually observe an increase in memory usage on the web container that processes ingestion and media workloads. The corresponding log message looks like this: `@smithy/node-http-handler:WARN - socket usage at capacity=150 and 387 additional requests are enqueued.`. In this situation, we recommend to increase the number of concurrent writes by setting `LANGFUSE_S3_CONCURRENT_WRITES` to a value larger than 50 (the default). Each additional write socket comes with a small memory overhead, so we recommend to increase the value gradually and observe the behaviour of your service. ## Slow UI queries or API calls If you notice long-loading screens within the UI or slow API calls, it is usually related to insufficient resources on the ClickHouse database or missing time filters. The tracing data is indexed by projectId and time, i.e. adding filter conditions on those should significantly improve performance. If all filters are in place, a larger ClickHouse instance may increase the observed performance. ClickHouse is designed to scale vertically, i.e. adding more memory to the instance should yield faster response times. You can check the [ClickHouse Docs](https://clickhouse.com/docs/operations/tips#using-less-than-16gb-of-ram) on which memory size to choose for your workloads. In general, we recommend at least 16 GiB of memory for larger deployments. ### Routing reads to a separate ClickHouse compute group [#clickhouse-read-only-url] On deployments with [compute-compute separation](https://clickhouse.com/docs/cloud/reference/warehouses) — primarily [ClickHouse Cloud](https://clickhouse.com/cloud) and [BYOC](https://clickhouse.com/cloud/bring-your-own-cloud) — you can isolate heavy analytical reads from the write path by pointing Langfuse at a dedicated read-only compute group. This keeps dashboard and public-API read traffic from contending with ingestion inserts and background merges on the primary compute. Set `CLICKHOUSE_READ_ONLY_URL` and Langfuse will route UI and public-API read queries to the given endpoint while writes, migrations, and ingestion continue to use `CLICKHOUSE_URL`: Credentials (`CLICKHOUSE_USER`, `CLICKHOUSE_PASSWORD`) and the database name are reused from the primary configuration, so the read-only compute group must accept the same user. Both variables are optional and unset by default; on a single-node ClickHouse or a non-compute-separated cluster they provide no benefit because the endpoint would be the same as the primary. ### Skipping the FINAL modifier for OTel projects [#skip-final-otel-projects] Langfuse stores observations in a ClickHouse `ReplacingMergeTree` and, by default, appends the `FINAL` modifier to public-API observations queries so that the latest version of each row wins at read time. This is required when ingestion produces multiple versions of the same observation (e.g. separate start, end, and update events from the legacy SDKs), but it adds query-time merge work and slows reads down. Projects that ingest **exclusively via OpenTelemetry** (the OTel SDKs or the `/api/public/otel` endpoint) write each observation as a single immutable row, so `FINAL` is unnecessary for them. Setting `LANGFUSE_SKIP_FINAL_FOR_OTEL_PROJECTS=true` enables per-project tracking: when a project ingests through the OTel endpoint, Langfuse marks it in Redis with a 24-hour TTL, and subsequent observations reads for that project (in `GET /api/public/observations` and the observations CTE in `GET /api/public/traces`) skip the `FINAL` modifier. The TTL refreshes on every OTel ingestion call, and the marker expires automatically if a project stops sending OTel traffic, so the behavior reverts safely if a project switches back to legacy ingestion. This is the recommended way to benefit from the optimization on mixed deployments where only some projects are on OTel. If you are certain that **every** project on the instance ingests via OTel only, you can instead set `LANGFUSE_API_CLICKHOUSE_DISABLE_OBSERVATIONS_FINAL=true` to drop the `FINAL` modifier globally, without the Redis lookup. This flag takes precedence over the per-project logic. Do not enable it if any project still uses the legacy ingestion path, as observations reads may otherwise return stale or duplicate rows. ## Increasing Disk Usage LLM tracing data may contain large payloads due to inputs and outputs being tracked. In addition, ClickHouse stores observability data within its system tables. If you notice that your disk space is increasing significantly on S3/Blob Storage or ClickHouse, we can recommend the following. In general, the most effective way to free disk space is to configure a [data retention](/docs/data-retention) policy. If this is not available in your plan, consider the options below. For larger deployments where ClickHouse disk growth becomes an operational bottleneck, consider [ClickHouse Cloud or BYOC](/self-hosting/deployment/infrastructure/clickhouse#cloudbyoc). These deployments separate storage from compute, so storage can grow independently of the compute that serves Langfuse queries. This does not replace retention policies because retained tracing data still consumes storage, but it removes much of the manual disk planning, volume expansion, and single-shard capacity management required in self-managed OSS ClickHouse setups. ### S3 / Blob Storage Disk Usage You can implement lifecycle rules to automatically remove old files from your blob storage. We recommend to keep events for as long as you want to access them within the UI or you want to update them. For most customers, a default of 30 days is a good choice. However, this does not apply to the media bucket used for storing uploaded media files. Setting a retention policy on this bucket is not recommended because: 1. Referenced media files in traces would break 2. Future uploads of the same file would fail since file upload status is tracked by hash in Postgres Instead, we recommend using the [Langfuse data-retention feature](/docs/data-retention) to manage media files properly and avoid broken references across the product. ### ClickHouse Disk Usage To automatically remove data within ClickHouse, you can use the [TTL](https://clickhouse.com/docs/guides/developer/ttl) feature. See the ClickHouse documentation for more details on how to configure it. This is applicable for the `traces`, `observations`, `scores`, and `event_log` table within ClickHouse. The following query helps to identify the largest tables in Clickhouse: ```sql SELECT table, formatReadableSize(size) as size, rows FROM ( SELECT table, database, sum(bytes) AS size, sum(rows) AS rows FROM system.parts WHERE active GROUP BY table, database ORDER BY size DESC ) ``` #### Slow or timing-out deletions [#clickhouse-deletion-timeouts] If data retention jobs or project/trace deletions fail with timeout errors, the client-side ClickHouse HTTP timeout for delete operations is likely exhausted. Increase `LANGFUSE_CLICKHOUSE_DELETION_TIMEOUT_MS` (default `600000`, i.e. 10 minutes) to give long-running deletes more headroom. This applies to both scheduled retention jobs and user-triggered deletes across traces, observations, scores, dataset run items, and events. On ClickHouse 25.7 and above, you can further reduce mutation pressure by opting into lightweight deletes and updates. Set `CLICKHOUSE_LIGHTWEIGHT_DELETE_MODE` from its default `alter_update` to `lightweight_update` (or `lightweight_update_force`) to resolve `DELETE` statements via lightweight deletes instead of `ALTER TABLE ... DELETE` mutations. Additionally, set `CLICKHOUSE_USE_LIGHTWEIGHT_UPDATE=true` to route tracing-table updates through native `UPDATE` statements instead of `ALTER TABLE ... UPDATE` mutations. Both reduce the amount of background merge work ClickHouse performs on deletion-heavy workloads. Native `UPDATE` statements require the `enable_lightweight_update` setting on the ClickHouse side. Enable it for the Langfuse user before setting `CLICKHOUSE_USE_LIGHTWEIGHT_UPDATE=true`, otherwise the updates are rejected: ```sql CREATE SETTINGS PROFILE langfuse_lightweight_update SETTINGS enable_lightweight_update = 1 TO 'user'; ``` See [user permissions](/self-hosting/deployment/infrastructure/clickhouse#user-permissions) for the full set of ClickHouse grants Langfuse requires. #### ClickHouse system log tables [#clickhouse-system-log-tables] On default ClickHouse configurations, the system log tables (`trace_log`, `text_log`, `opentelemetry_span_log`, `asynchronous_metric_log`, `metric_log`, `latency_log`) can dominate disk usage. They have no TTL by default, and the query profiler writes to `system.trace_log` continuously. Langfuse does not read from these tables, so you can safely reduce them. Two options: **Option 1 — Disable unused system log tables.** Mount a file into `/etc/clickhouse-server/config.d/` that opts out of the tables Langfuse never reads. This is the approach taken by default in the [Langfuse Terraform AWS module](https://github.com/langfuse/langfuse-terraform-aws/pull/26): ```xml ``` On the [Langfuse Helm chart](/self-hosting/deployment/kubernetes-helm) (`v2.x`), you can ship the same block via `clickhouse.cluster.settings`. Keep `query_log`, `part_log`, and `error_log` enabled — they are useful for debugging and remain small. **Option 2 — Aggressive TTLs.** If you want to keep the tables around for debugging, attach short TTLs and disable the query profiler instead: ```xml 0 0 ENGINE = MergeTree PARTITION BY toYYYYMM(event_date) ORDER BY (event_date, event_time) TTL event_date + INTERVAL 7 DAY ENGINE = MergeTree PARTITION BY toYYYYMM(finish_date) ORDER BY (finish_date, finish_time_us) TTL finish_date + INTERVAL 7 DAY ENGINE = MergeTree PARTITION BY toYYYYMM(event_date) ORDER BY (event_date, event_time) TTL event_date + INTERVAL 30 DAY ``` TTL directives only apply when the system table is first created. On an existing install, after deploying the config you also need to retrofit the tables — for example, for `system.trace_log`: ```sql SET max_table_size_to_drop = 0; TRUNCATE TABLE system.trace_log; ALTER TABLE system.trace_log MODIFY TTL event_date + INTERVAL 7 DAY; ``` Repeat for each table you want to cap. See the upstream discussion in [langfuse/langfuse#13123](https://github.com/langfuse/langfuse/issues/13123). ## High Redis CPU Load If you observe high Redis Engine CPU utilization (above 90%), we recommend to check the following: - Use an instance with at least 4 CPUs. This will allow Redis to schedule networking and background tasks on separate CPUs. - Ensure that you have [Redis Cluster mode](/self-hosting/deployment/infrastructure/cache#redis-cluster-mode) enabled. If the high CPU utilization persists, it is possible to shard the queues that Langfuse uses across multiple nodes. Set `LANGFUSE_INGESTION_QUEUE_SHARD_COUNT` and `LANGFUSE_TRACE_UPSERT_QUEUE_SHARD_COUNT` to a value greater than 1 to enable sharding. We recommend a value that is approximately 2-3 times the number of shards you have within your Redis cluster to ensure an equal distribution among the nodes, as each queue-shard will be allocated to a random slot in Redis (see [Redis Cluster](https://redis.io/docs/latest/operate/oss_and_stack/management/scaling/) docs for more details). Sharding the queues is an advanced feature and should only be used if you have a high Redis CPU load and have followed the above recommendations. Once you have sharded your queue, do _not_ reduce the number of Shards. Make sure to scale `LANGFUSE_INGESTION_QUEUE_PROCESSING_CONCURRENCY` and `LANGFUSE_TRACE_UPSERT_WORKER_CONCURRENCY` accordingly as it counts _per shard_. Per default, we target a concurrency of 20 per worker, i.e. set it to 2 if you have 10 queue-shards. ## FAQ ## GitHub Discussions --- If you experience any issues when self-hosting Langfuse, please: 1. Check out [Troubleshooting & FAQ](/self-hosting/troubleshooting-and-faq) page. 2. Use [Ask AI](/ask-ai) to get instant answers to your questions. 3. Ask the maintainers on [GitHub Discussions](/gh-support). 4. Create a bug report or feature request on [GitHub](/issues). Enterprise-grade support is available when self-hosting Langfuse. Learn more on our [pricing page](/pricing-self-host). --- ## Agent Instructions This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/self-hosting/configuration/scaling.md`. ### Querying these docs If the answer is not on this page, query the documentation instead of guessing: - **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question: ```bash curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?" ``` - **Index of every page**: , with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt). ### Before writing Langfuse code - **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results. - **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application. - **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples. - **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx langfuse-cli api `) to read or write traces, prompts, datasets, and scores from the terminal. Found an error in these docs? Please open an issue at .