Langfuse launches AI Gateway Learn more

Air-gapped LLM observability for government and regulated environments

Government agencies, defense contractors, and some banks run AI systems in networks that can't reach the internet. Their observability tooling has to work under the same rules: no outbound connections, no vendor-operated services, and cryptography that passes a security assessment. Langfuse is open source and self-hostable, and it does not require internet access, so it can run inside such networks.

TL;DR: Self-host Langfuse inside the isolated network, with its dependencies (Postgres, ClickHouse, Redis, S3-compatible storage) running there too. Langfuse does not require internet access. The Langfuse Government license adds FIPS mode, which runs Langfuse's cryptography on FIPS 140-3 validated modules in approved mode, images built on Iron Bank's hardened UBI 9, and no telemetry. Features that call a model, such as LLM-as-a-Judge and the Playground, need a model endpoint reachable inside the network.

Deployment architecture
UI, API, and SDK clients connect to Langfuse Web. Web connects to PostgreSQL, Redis or Valkey, ClickHouse, and S3 or blob storage. Redis queues jobs for Langfuse Worker, which connects to PostgreSQL, ClickHouse, and S3. Both containers can optionally connect to LLM providers: Web for the playground, Worker for evaluations. The optional Langfuse AI gateway receives model requests from applications and coding agents, resolves gateway keys and uploads traces through Langfuse Web, and relays the requests to the LLM providers. The application containers, gateway, and storage run in your VPC or on-premises. The LLM providers can also run in the same VPC or a peered VPC.VPC / on-premisesUI, API & SDKsApps & agentsModel requestsResolve + tracesRequestsPlaygroundEvaluationsRelayAI GatewayOptional (beta)langfuse/ai-gatewayLangfuse WebUI & API serverlangfuse/langfuseLangfuse WorkerAsync event processinglangfuse/workerPostgreSQLTransactional dataRedis / ValkeyCache & job queueClickHouseObservability dataS3 / Blob storageEvents & attachmentsLLM providersOptional (bring your own)
Web connectionsWorker connectionsOptional

The optional AI gateway relays model requests from your apps and coding agents to your LLM providers and records them in Langfuse. LLM providers can also run in the same VPC or a peered VPC.

What runs where

Every component of a Langfuse deployment can run in your network:

ComponentIn an isolated network
Langfuse Web and WorkerContainer images mirrored to your internal registry
Postgres, Redis or ValkeySelf-managed or a managed service available in your environment
ClickHouseSelf-managed, for example with the ClickHouse Kubernetes Operator
Blob storageAny S3-compatible store inside the network
SDKs and OpenTelemetryYour applications send traces to the internal Langfuse URL
LLM API for evaluationsOptional; a model endpoint inside the network or a peered VPC

Langfuse checks a cached GitHub API for new releases. Without internet access, that check fails gracefully. See networking and the list of hostnames Langfuse contacts, which you can use to configure an egress firewall.

Langfuse Government

The Government license, a key starting with lf_gov_, includes everything in Enterprise and adds:

  • FIPS mode. Langfuse runs its cryptography on FIPS 140-3 validated modules operated in approved mode, and refuses to start when it can't. It is available from Langfuse v4.71.0. See FIPS mode.
  • Hardened images. From v4.55.0, the web and worker images are built on Iron Bank's hardened Red Hat UBI 9, run as a non-root user, and ship no package manager, curl, or Python. See hardened images.
  • No telemetry. Deployments with a Government license send no telemetry to Langfuse, whatever TELEMETRY_ENABLED is set to.

Langfuse holds no FIPS 140-3 certificate of its own. The Cryptographic Module Validation Program validates modules, not products, and Langfuse uses validated modules in approved mode. The host must boot in FIPS mode, and you are responsible for running Postgres, ClickHouse, Redis, blob storage, and TLS termination with validated modules. End-to-end documentation for a complete air-gapped stack is still in progress, as of November 2026. Talk to us about your requirements.

What needs a model endpoint

Tracing, prompt management, datasets, dashboards, annotation queues, and scores from your own code work without any model access. These features call a model and need an LLM connection that is reachable inside the network:

LLM connections accept a custom base URL, so an internally hosted model behind an OpenAI-compatible API can power them. Langfuse blocks private IP ranges for these URLs by default, so allow the internal host with LANGFUSE_LLM_CONNECTION_WHITELISTED_HOST or a related variable. See blocked IP addresses. Code evaluators score traces without calling a model.

The self-hosted Langfuse AI Gateway ignores egress proxies, so it can't reach public provider APIs from an isolated network. It can route to an internal OpenAI- or Anthropic-compatible endpoint through a custom base URL, once its host is allowed on web and the gateway. See self-host an open source AI gateway.

Hardening checklist

  • Mirror the Langfuse images to your internal registry, or build them yourself.
  • Disable open sign-up and enforce SSO with your identity provider. See hardening.
  • Use scram-sha-256 authentication on Postgres when running in FIPS mode.
  • Keep databases on private networks, and expose only the web container. See network exposure.
  • Turn on audit logs and data retention, both included in Enterprise and Government.

For the full go-live list, see the production checklist for self-hosted Langfuse.

FAQ

Does Langfuse need internet access?

No. Langfuse does not require internet access. The release check fails gracefully, and OSS telemetry can be turned off with TELEMETRY_ENABLED=false. Government deployments send no telemetry.

Is Langfuse FedRAMP authorized?

FedRAMP authorizes cloud services. Langfuse Government is self-hosted software that runs in your own authorized environment, so it becomes part of your system's authorization boundary. For the certifications of Langfuse Cloud, see the security page.

Are the hardened images available for arm64?

Yes, the images are published for linux/amd64 and linux/arm64. The FIPS certificate's tested environments list Red Hat Enterprise Linux 9 on x86_64 and IBM platforms, so confirm other hosts and arm64 with your assessor.


Was this page helpful?