--- title: AWS ECS Fargate description: How we use AWS ECS Fargate for application containers. --- # AWS ECS Fargate We are using [AWS ECS Fargate](https://docs.aws.amazon.com/ecs/) as the runtime for our web and worker application containers. To separate traffic and load patterns we run multiple deployments of each service with different configurations with specific routing or queue worker configurations. In addition, all of the containers are autoscaled dynamically based on application load and metrics. ## Terminology - **Cluster**: A logical group of services and tasks. - **Service**: A collection of tasks that share the same configuration. Services are long-lived and are a collection of tasks. - **Task**: A specific, ephemeral group of containers. Comparable with a `Pod` in Kubernetes. Tasks are short-lived and can be scaled horizontally. - **Task Definition**: A specific configuration for a task. It contains environment variables, the docker image, and other configuration. - **Sidecar**: Additional containers that run alongside the main container in a task. ## Common Concepts ### Components Split We run the `web` and `worker` services in multiple different configurations: - **web**: The standard web application that serves most of the user-facing traffic. - **web-iso**: An additional web container that specific routes are sent to. Currently, this affects all trace-related public API routes, e.g. `/api/public/traces` and `/api/public/observations` (among others). As those workloads are CPU heavy they would have a noisy-neighbor impact on more latency sensitive routes if they would be kept in the main container group. Traffic separation happens on the LoadBalancer level using path-based rules. - **web-ingestion**: A container group that handles media APIs, `/api/public/ingestion`, and `/api/public/otel/v1/traces` requests. This allows for more fine-grained autoscaling and protects the stability of ingestion routes from the remaining implementation. - **worker**: The standard worker container that performs most of the queue processing. ingestion-queue and otel-ingestion-queue processing is disabled. - **worker-cpu**: A worker deployment that handles CPU-heavy processing, mainly the ingestion-queue and the otel-ingestion-queue. This increases throughput and reduces error rates due to CPU stalls on other worker jobs. ### Sidecars Each of our tasks consists of three containers. The core application container that we ship for Langfuse-OSS, and two sidecar containers for monitoring. One sidecar container handles logging using Fluent Bit and forwards all application logs directly to DataDog. The second sidecar container is the Datadog Agent who tracks metrics and also acts as an OpenTelemetry Collector for traces. ## Deployment ### Configuration Changes The container configuration, e.g. environment variables, can be changed via Terraform. This updates the underlying Task Definition and triggers a new deployment for the service. Manual scaling overwrites will be reverted on the next Terraform apply. ### Version Changes The `Deploy to ECS` action on GitHub will build new versions of the application containers and deploy them into the cluster. This will create a new Task Definition version and also trigger a new deployment for the service. ### Common Issues - Sometimes the container deployment gets stuck and remains in a "Deploying" state for a long time. This can surface as a deployment timeout within GitHub when running deploys. Often this is the case when multiple deployments happen concurrently and AWS ECS cannot decide which containers to stop and has no capacity to start additional containers. Our scaling settings force it to never exceed 100% scaling capacity and not drop below the desired capacity. This can be resolved by increasing the desired capacity temporarily. - Overwriting deployed versions with stale plans. Terraform takes the current container version at the `plan` time as the baseline for subsequent writes. If a deployment happened between the plan and the apply steps, it might be the case that the apply reverts the deploy. Therefore, we recommend to keep the time between plans and applies as short as possible. ## Performance Notes ### Logging large bodies Performing large computations within logging calls has a strong impact on the application performance, even if the corresponding logs are disabled. E.g. `logger.debug(JSON.stringify(bigObject))` will first stringify the object and then make a logging decision. You can use the Datadog profiling view and flamegraph to identify those. ## Scaling - For the `web` containers we use request-based autoscaling. We have a target number of requests and add or remove capacity if we deviate from the target. - For the `worker` containers we use queue-length-based autoscaling. We define a target backlog per instance and add capacity if the backlog exceeds the target. --- ## Agent Instructions This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/handbook/product-engineering/infrastructure/ecs.md`. ### Querying these docs If the answer is not on this page, query the documentation instead of guessing: - **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question: ```bash curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?" ``` - **Index of every page**: , with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt). ### Before writing Langfuse code - **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results. - **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application. - **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples. - **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx langfuse-cli api `) to read or write traces, prompts, datasets, and scores from the terminal. Found an error in these docs? Please open an issue at .