--- title: Data Masking (self-hosted) description: Configure data masking in your self-hosted Langfuse deployment to redact sensitive information from tracing events. label: "Version: v4" sidebarTitle: "Data Masking" --- # Data Masking Masking sensitive data is crucial for **compliance** (GDPR, HIPAA, PCI DSS) and **user privacy** when using LLM observability. Langfuse provides two complementary approaches to data masking: | Approach | Description | Best For | | ------------------------------------------------------------------ | -------------------------------------------- | ------------------------------------------------------- | | [Client-Side Masking](#client-side-masking) | Mask data in the SDK before transmission | Preventing sensitive data from leaving your application | | [Server-Side Ingestion Masking](#server-side-ingestion-masking-ee) | Mask data via HTTP callback during ingestion | Centralized policy enforcement across all clients (EE) | For maximum security, consider using both approaches together. ## Client-Side Masking Client-side masking allows you to redact sensitive information directly in your application before data is sent to Langfuse. This ensures sensitive data never leaves your application. ```mermaid sequenceDiagram participant App as Your Application participant SDK as Langfuse SDK participant LF as Langfuse App->>SDK: Generate trace event Note over SDK: Apply masking function SDK->>LF: Send masked event Note over LF: Process masked event ``` **Key benefits:** - Data is masked before transmission—sensitive information never reaches Langfuse - Configured per SDK instance by application developers - No additional infrastructure required For comprehensive documentation including code examples, advanced patterns, and integration guides, see the [Client-Side Masking documentation](/docs/observability/features/masking). --- ## Server-Side Ingestion Masking (EE) [#server-side-ingestion-masking-ee] This feature requires an Enterprise license. Please add your [license key](/self-hosting/license-key) to activate it. Server-side ingestion masking allows self-hosted Langfuse administrators to define custom callback logic for masking or redacting sensitive data from tracing events as they are ingested. This provides centralized data masking across all clients. Server-side masking is a centralized safety net, not a replacement for client-side masking when sensitive data must never leave the application boundary. The callback masks data before it is persisted in ClickHouse and downstream Langfuse views. **Key benefits:** - Single point of configuration for all tracing data - Platform administrator control - Safety net for data that bypasses client-side masking ### How It Works 1. When a tracing event is processed, Langfuse checks if a masking callback URL is configured. 2. If configured, Langfuse sends the OpenTelemetry trace object to your callback endpoint via HTTP POST. 3. Your callback service processes the data and returns the masked object. 4. Langfuse persists the masked data in ClickHouse. ```mermaid sequenceDiagram participant SDK as Langfuse SDK participant LF as Langfuse participant Callback as Masking Callback participant CH as ClickHouse SDK->>LF: Send trace event LF->>Callback: POST (OpenTelemetry object) Note over Callback: Apply masking logic Callback->>LF: Return masked object LF->>CH: Persist masked event ``` Scrubbing runs asynchronously in a worker. Events land in [blob storage](/self-hosting/deployment/infrastructure/blobstorage) in the interim before they are masked and written to ClickHouse. See the [self-hosting architecture](/self-hosting#architecture) for how web, worker, and storage fit together. ### Configuration Masking configuration is split across the Langfuse Web and Worker containers. The Web container extracts propagated headers from the incoming request; the Worker container makes the outbound callback request. Set each variable on the container listed below. Setting all variables on both containers is also safe. Configure the following environment variables on the **Langfuse Worker** container: | Variable | Required / Default | Description | | ------------------------------------------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `LANGFUSE_INGESTION_MASKING_CALLBACK_URL` | Required to enable | The HTTP(S) URL of your masking callback endpoint. When set, all ingestion events will be sent to this endpoint for masking before processing. | | `LANGFUSE_INGESTION_MASKING_CALLBACK_TIMEOUT_MS` | `500` | Timeout in milliseconds for the callback request. If the callback does not respond within this time, the behavior is determined by the fail mode setting. | | `LANGFUSE_INGESTION_MASKING_CALLBACK_FAIL_CLOSED` | `false` | When set to `true`, events are dropped if the callback fails or times out, and a warning is logged. When `false` (default, fail open), events are processed without masking if the callback fails. | | `LANGFUSE_INGESTION_MASKING_MAX_RETRIES` | `1` | Maximum number of retries for failed callback requests. If the callback fails after this many attempts, the behavior is determined by the fail mode setting. | Configure the following environment variable on the **Langfuse Web** container: | Variable | Required / Default | Description | | ----------------------------------------------- | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `LANGFUSE_INGESTION_MASKING_PROPAGATED_HEADERS` | `''` | Comma-separated list of header names to extract from the incoming OpenTelemetry ingestion request on the Web container and forward to the masking callback on the Worker container. Header names are case-insensitive. | ### Callback Interface #### Request Langfuse sends a `POST` request to your callback URL with: **Headers:** | Header | Description | | ----------------------- | ---------------------------------------------------- | | `Content-Type` | `application/json` | | `X-Langfuse-Org-Id` | The organization ID associated with the trace event. | | `X-Langfuse-Project-Id` | The project ID associated with the trace event. | **Body:** The request body contains the OpenTelemetry trace object in JSON format. This is the raw tracing data that would be stored in Langfuse. The event body follows the [OpenTelemetry Trace Request Proto](https://github.com/open-telemetry/opentelemetry-proto/blob/main/opentelemetry/proto/trace/v1/trace.proto). #### Response Your callback must return: - **HTTP Status**: `200 OK` for successful masking - **Body**: The masked OpenTelemetry object in the **exact same schema** as the input The response object must maintain the same structure as the input. Only modify the values you want to mask—do not add, remove, or rename fields. Langfuse parses the callback response as JSON and expects the OpenTelemetry shape during downstream processing, but it does not run a separate structural validation at the callback boundary. #### Error Handling Error handling behavior is configured via `LANGFUSE_INGESTION_MASKING_CALLBACK_FAIL_CLOSED`: | Scenario | Fail open (default, `false`) | Fail closed (`true`) | | --------------------- | ---------------------------------------- | ----------------------------- | | Callback timeout | Event processed unmasked, warning logged | Event dropped, warning logged | | HTTP error (4xx, 5xx) | Event processed unmasked, warning logged | Event dropped, warning logged | | Invalid JSON response | Event processed unmasked, warning logged | Event dropped, warning logged | | Network error | Event processed unmasked, warning logged | Event dropped, warning logged | ### Limitations Server-side ingestion masking only applies to events ingested via the [OpenTelemetry endpoint](/integrations/native/opentelemetry) (`/api/public/otel`). This includes: - **Python SDK v3+** and **TypeScript SDK v4+** (OTEL-native) - Third-party OpenTelemetry instrumentation libraries (OpenLLMetry, OpenLIT, etc.) Events ingested via the legacy `/api/public/ingestion` endpoint are not processed through the masking callback. For project or organization-specific masking requirements, we recommend to use the `X-Langfuse-Org-Id` and `X-Langfuse-Project-Id` headers or `LANGFUSE_INGESTION_MASKING_PROPAGATED_HEADERS` to customize your masking decisions. ### Example Implementation Here's an example masking callback service in Python using FastAPI: ```python from fastapi import FastAPI, Request, Header from typing import Optional import re app = FastAPI() def mask_pii(data): """Recursively mask PII in the data structure.""" if isinstance(data, str): # Mask email addresses data = re.sub(r'\b[\w.-]+?@\w+?\.\w+?\b', '[REDACTED_EMAIL]', data) # Mask phone numbers data = re.sub(r'\b\d{3}[-.]?\d{3}[-.]?\d{4}\b', '[REDACTED_PHONE]', data) # Mask credit card numbers data = re.sub(r'\b\d{4}[-\s]?\d{4}[-\s]?\d{4}[-\s]?\d{4}\b', '[REDACTED_CC]', data) return data elif isinstance(data, dict): return {k: mask_pii(v) for k, v in data.items()} elif isinstance(data, list): return [mask_pii(item) for item in data] return data @app.post("/mask") async def mask_trace( request: Request, x_langfuse_org_id: Optional[str] = Header(None), x_langfuse_project_id: Optional[str] = Header(None) ): """ Masking callback endpoint for Langfuse ingestion. Receives OpenTelemetry trace objects and returns masked versions. """ body = await request.json() # Apply masking logic masked_body = mask_pii(body) # Optionally, apply different rules based on org/project # if x_langfuse_project_id == "specific-project-id": # masked_body = apply_special_masking(masked_body) return masked_body ``` Deploy this service and configure Langfuse to use it: ```bash LANGFUSE_INGESTION_MASKING_CALLBACK_URL=https://your-masking-service.internal/mask LANGFUSE_INGESTION_MASKING_CALLBACK_TIMEOUT_MS=500 LANGFUSE_INGESTION_MASKING_CALLBACK_FAIL_CLOSED=true ``` ### Performance Considerations - **Latency**: The masking callback adds latency to the ingestion path. Keep your callback service fast (ideally < 100ms). - **Timeout**: The default 500ms timeout is designed to balance reliability with performance. Adjust based on your masking complexity. - **Availability**: Your masking service should be highly available, especially with fail-closed mode enabled. - **Colocation**: Deploy your masking service close to your Langfuse deployment to minimize network latency. Sidecar containers are recommended. ### Troubleshooting - Events are being dropped unexpectedly 1. Check that your masking service is responding within the configured timeout. 2. Verify the response schema matches the input schema exactly. 3. Review Langfuse Worker container logs for warning messages. 4. Temporarily set `LANGFUSE_INGESTION_MASKING_CALLBACK_FAIL_CLOSED=false` to diagnose issues. - High latency on trace ingestion 1. Monitor your masking service response times. 2. Consider increasing `LANGFUSE_INGESTION_MASKING_CALLBACK_TIMEOUT_MS` if your masking logic requires more time. 3. Optimize your masking logic or add caching where appropriate. 4. Ensure network latency between Langfuse and your masking service is minimal. - Masking not being applied 1. Verify `LANGFUSE_INGESTION_MASKING_CALLBACK_URL` is correctly set on the Langfuse Worker container. 2. Check that your masking service is reachable from the Langfuse Worker container. 3. Ensure your masking logic is correctly modifying the data and returning it. --- If you experience any issues when self-hosting Langfuse, please: 1. Check out [Troubleshooting & FAQ](/self-hosting/troubleshooting-and-faq) page. 2. Use [Ask AI](/ask-ai) to get instant answers to your questions. 3. Ask the maintainers on [GitHub Discussions](/gh-support). 4. Create a bug report or feature request on [GitHub](/issues). Enterprise-grade support is available when self-hosting Langfuse. Learn more on our [pricing page](/pricing-self-host). --- ## Agent Instructions This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/self-hosting/security/data-masking.md`. ### Querying these docs If the answer is not on this page, query the documentation instead of guessing: - **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question: ```bash curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?" ``` - **Index of every page**: , with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt). ### Before writing Langfuse code - **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results. - **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application. - **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples. - **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx langfuse-cli api `) to read or write traces, prompts, datasets, and scores from the terminal. Found an error in these docs? Please open an issue at .