--- title: Advanced Features description: Configure masking, logging, sampling, multi-project routing, evaluations, and environment-specific behaviors for Python and JS/TS. category: SDKs --- # Advanced features Use these methods to harden your Langfuse instrumentation, protect sensitive data, and adapt the SDKs to your specific environment. ## Filtering by Instrumentation Scope [#filtering-by-instrumentation-scope] Langfuse now applies a **default span filter** in both SDKs to keep exports LLM-focused without extra configuration. By default, a span is exported if **any** of these are true: - It was created by the Langfuse SDK (`instrumentation_scope.name == "langfuse-sdk"`) - It has at least one `gen_ai.*` attribute - It comes from a known LLM instrumentation scope (for example `openinference.*`, `langsmith`, `haystack`, `litellm`, `agent_framework`, `strands-agents`, `vllm`, `opentelemetry.instrumentation.anthropic`) If you want another integration added to the default instrumentation scope allowlist, open an issue in [langfuse/langfuse](https://github.com/langfuse/langfuse/issues) with the scope name and a sample span. You can inspect a span's instrumentation scope in Langfuse under `metadata.scope.name`. Filtered-out spans do not appear in the UI. To identify filtered scopes: 1. Enable debug logging (`Langfuse(debug=True)` or `LANGFUSE_DEBUG="True"` on Python, `LANGFUSE_DEBUG="true"` or `LANGFUSE_LOG_LEVEL="DEBUG"` on JS/TS). 2. Run your application and check logs for dropped-span messages and instrumentation scope names. 3. Add those scopes to your allowlist logic by composing with `is_default_export_span` / `isDefaultExportSpan`. 4. Optional: temporarily use `should_export_span=lambda span: True` or `shouldExportSpan: () => true` to inspect all spans, then restore filtering. Earlier SDK versions exported all non-blocked spans by default. To restore that behavior, provide an always-true custom filter callback. Filtering spans may break the parent-child relationships in your traces. For example, if you filter out a parent span but keep its children, you may see "orphaned" observations in the Langfuse UI. Please use the debugging flow above to re-add filtered out spans. Default behavior (recommended): ```python from langfuse import Langfuse # Smart default filter (Langfuse + GenAI/LLM spans) langfuse = Langfuse() ``` Export everything: ```python from langfuse import Langfuse langfuse = Langfuse(should_export_span=lambda span: True) ``` Passing `should_export_span` replaces the default filter. To keep default behavior and extend it, compose with `is_default_export_span`. Compose custom logic with built-in predicates: ```python from langfuse import Langfuse from langfuse.span_filter import is_default_export_span langfuse = Langfuse( should_export_span=lambda span: ( is_default_export_span(span) or ( span.instrumentation_scope is not None and span.instrumentation_scope.name.startswith("my_framework") ) ) ) ``` Only export spans created by the Langfuse SDK: ```python from langfuse import Langfuse from langfuse.span_filter import is_langfuse_span langfuse = Langfuse(should_export_span=is_langfuse_span) ``` Available Python helpers: `is_default_export_span`, `is_langfuse_span`, `is_genai_span`, `is_known_llm_instrumentor`, `KNOWN_LLM_INSTRUMENTATION_SCOPE_PREFIXES`. `blocked_instrumentation_scopes` still works for backward compatibility, but is deprecated and planned for removal in a future version. Prefer expressing deny rules in `should_export_span`. Deprecated compatibility example: ```python from langfuse import Langfuse langfuse = Langfuse( should_export_span=lambda span: True, blocked_instrumentation_scopes=["sqlalchemy", "psycopg"], ) ``` Default behavior (recommended): ```ts filename="instrumentation.ts" import { NodeSDK } from "@opentelemetry/sdk-node"; import { LangfuseSpanProcessor } from "@langfuse/otel"; const sdk = new NodeSDK({ // Smart default filter (Langfuse + GenAI/LLM spans) spanProcessors: [new LangfuseSpanProcessor()], }); sdk.start(); ``` Custom filtering: ```ts filename="instrumentation.ts" /shouldExportSpan/ import { NodeSDK } from "@opentelemetry/sdk-node"; import { LangfuseSpanProcessor, ShouldExportSpan } from "@langfuse/otel"; const shouldExportSpan: ShouldExportSpan = ({ otelSpan }) => otelSpan.instrumentationScope.name !== "express"; const sdk = new NodeSDK({ spanProcessors: [new LangfuseSpanProcessor({ shouldExportSpan })], }); sdk.start(); ``` Passing `shouldExportSpan` replaces the default filter, so include default conditions in your callback if you want to extend rather than replace the default behavior. Compose with the default filter: ```ts filename="instrumentation.ts" import { isDefaultExportSpan, type ShouldExportSpan } from "@langfuse/otel"; const shouldExportSpan: ShouldExportSpan = ({ otelSpan }) => isDefaultExportSpan(otelSpan) || otelSpan.instrumentationScope.name.startsWith("my-framework"); ``` Available JS/TS helpers from `@langfuse/otel`: `isDefaultExportSpan`, `isLangfuseSpan`, `isGenAISpan`, `isKnownLLMInstrumentor`, `KNOWN_LLM_INSTRUMENTATION_SCOPE_PREFIXES`. Export everything: ```ts filename="instrumentation.ts" new LangfuseSpanProcessor({ shouldExportSpan: () => true }); ``` You can read more about using Langfuse with an existing OpenTelemetry setup [here](/faq/all/existing-otel-setup). ## Mask sensitive data If your trace data might contain sensitive information such as PII or secrets, configure a masking hook before sending spans to Langfuse. For Python SDK applications, prefer `mask_otel_spans` because it runs at export stage on raw OpenTelemetry span attributes, including spans created by third-party instrumentations. Use `mask_otel_spans` to return sparse patches for the OpenTelemetry spans that should change before export. ```python import re from typing import Optional from langfuse import Langfuse from langfuse.types import ( MaskOtelSpansParams, MaskOtelSpansResult, OtelSpanPatch, ) email_pattern = re.compile(r"\b[\w.-]+?@[\w.-]+?\.\w+?\b") def mask_otel_spans( *, params: MaskOtelSpansParams ) -> Optional[MaskOtelSpansResult]: patches = {} for identifier, span in params.spans.items(): replacements = {} for key, value in span.attributes.items(): if isinstance(value, str): masked_value = email_pattern.sub("[EMAIL_REDACTED]", value) if masked_value != value: replacements[key] = masked_value if replacements: patches[identifier] = OtelSpanPatch(set_attributes=replacements) return MaskOtelSpansResult(span_patches=patches) langfuse = Langfuse(mask_otel_spans=mask_otel_spans) ``` `mask_otel_spans` is synchronous and usually runs on the OpenTelemetry batch span processor worker thread; during `flush()` and shutdown it may run on the caller thread. Keep the function fast to avoid backing up the export queue. See [masking](/docs/observability/features/masking) for the full behavior, error handling, and legacy `mask` comparison. You can provide a `mask` function to the [`LangfuseSpanProcessor`](https://langfuse-js-git-main-langfuse.vercel.app/classes/_langfuse_otel.LangfuseSpanProcessor.html). This function will be applied to the input, output, and metadata of every observation. The function receives an object `{ data }`, where `data` is the stringified JSON of the attribute's value. It should return the masked data. ```ts filename="instrumentation.ts" /mask: / import { NodeSDK } from "@opentelemetry/sdk-node"; import { LangfuseSpanProcessor } from "@langfuse/otel"; const spanProcessor = new LangfuseSpanProcessor({ mask: ({ data }) => data.replace(/\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b/g, "***MASKED_CREDIT_CARD***"), }); const sdk = new NodeSDK({ spanProcessors: [spanProcessor] }); sdk.start(); ``` ## Logging & debugging The Langfuse SDK can expose detailed logging and debugging information to help you troubleshoot issues with your application. **Via environment variable:** You can set the log level using the `LANGFUSE_DEBUG` environment variable to enable the debug mode. ```bash export LANGFUSE_DEBUG="True" ``` **In code:** The Langfuse SDK uses Python's standard `logging` module. The main logger is named `"langfuse"`. To enable detailed debug logging, you can either: 1. Set the `debug=True` parameter when initializing the `Langfuse` client. 2. Configure the `"langfuse"` logger manually: ```python import logging langfuse_logger = logging.getLogger("langfuse") langfuse_logger.setLevel(logging.DEBUG) ``` The default log level for the `langfuse` logger is `logging.WARNING`. You can configure the global SDK logger to control the verbosity of log output. This is useful for debugging. **Via environment variable:** You can set the log level using the `LANGFUSE_LOG_LEVEL` environment variable to enable the debug mode. ```bash export LANGFUSE_LOG_LEVEL="DEBUG" ``` **In code:** ```typescript /configureGlobalLogger/ import { configureGlobalLogger, LogLevel } from "@langfuse/core"; // Set the log level to DEBUG to see all log messages configureGlobalLogger({ level: LogLevel.DEBUG }); ``` Available log levels are `DEBUG`, `INFO`, `WARN`, and `ERROR`. ## Sampling Sampling lets send only a subset of traces to Langfuse. This is useful to reduce costs and noise in high-volume applications. **In code:** You can configure the SDK to sample traces by setting the `sample_rate` parameter during client initialization. This value should be a float between `0.0` (sample 0% of traces) and `1.0` (sample 100% of traces). If a trace is not sampled, none of its observations (spans, generations) or associated scores will be sent to Langfuse. ```python from langfuse import Langfuse # Sample approximately 20% of traces langfuse_sampled = Langfuse(sample_rate=0.2) ``` **Via environment variable:** You can also set the sample rate using the `LANGFUSE_SAMPLE_RATE` environment variable. ```bash export LANGFUSE_SAMPLE_RATE="0.2" ``` **In code:** Langfuse respects OpenTelemetry's sampling decisions. Configure a sampler on your OTEL `NodeSDK` to control which traces reach Langfuse and reduce noise/costs in high-volume workloads. ```ts filename="instrumentation.ts" /TraceIdRatioBasedSampler/ import { NodeSDK } from "@opentelemetry/sdk-node"; import { LangfuseSpanProcessor } from "@langfuse/otel"; import { TraceIdRatioBasedSampler } from "@opentelemetry/sdk-trace-base"; const sdk = new NodeSDK({ sampler: new TraceIdRatioBasedSampler(0.2), spanProcessors: [new LangfuseSpanProcessor()], }); sdk.start(); ``` **Via environment variable:** You can also set the sample rate using the `LANGFUSE_SAMPLE_RATE` environment variable. ```bash export LANGFUSE_SAMPLE_RATE="0.2" ``` ## Isolated TracerProvider [#isolated-tracer-provider] You can configure a separate OpenTelemetry TracerProvider for use with Langfuse. This creates isolation between Langfuse tracing and your other observability systems. Benefits of isolation: - Langfuse spans won't be sent to your other observability backends (e.g., Datadog, Jaeger, Zipkin) - Third-party library spans won't be sent to Langfuse - Independent configuration and sampling rates While TracerProviders are isolated, they share the same OpenTelemetry context for tracking active spans. This can cause span relationship issues where: - A parent span from one TracerProvider might have children from another TracerProvider - Some spans may appear "orphaned" if their parent spans belong to a different TracerProvider - Trace hierarchies may be incomplete or confusing Plan your instrumentation carefully to avoid confusing trace structures. ```python {4, 5} from opentelemetry.sdk.trace import TracerProvider from langfuse import Langfuse langfuse_tracer_provider = TracerProvider() # do not set to global tracer provider to keep isolation langfuse = Langfuse(tracer_provider=langfuse_tracer_provider) langfuse.start_observation(name="myspan").end() # Span will be isolated from remaining OTEL instrumentation ``` Isolate Langfuse spans with a custom provider and avoid sending them to other exporters. ```ts /setLangfuseTracerProvider/ import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node"; import { setLangfuseTracerProvider } from "@langfuse/tracing"; // Create a new TracerProvider and register the LangfuseSpanProcessor // do not set this TracerProvider as the global TracerProvider const langfuseTracerProvider = new NodeTracerProvider({ spanProcessors: [new LangfuseSpanProcessor()], }) // Register the isolated TracerProvider setLangfuseTracerProvider(langfuseTracerProvider) ``` You can read more about using Langfuse with an existing OpenTelemetry setup [here](/faq/all/existing-otel-setup). ## Multi-project setups [#multi-project-setup-experimental] Multi-project setups are **experimental** in the Python SDK and have important limitations regarding third-party OpenTelemetry integrations. The Langfuse Python SDK supports routing traces to different projects within the same application by using multiple public keys. This works because the Langfuse SDK adds a specific span attribute containing the public key to all spans it generates. **How it works:** 1. **Span Attributes**: The Langfuse SDK adds a specific span attribute containing the public key to spans it creates 2. **Multiple Processors**: Multiple span processors are registered onto the global tracer provider, each with their respective exporters bound to a specific public key 3. **Filtering**: Within each span processor, spans are filtered based on the presence and value of the public key attribute **Important Limitation with Third-Party Libraries:** Third-party libraries that emit OpenTelemetry spans automatically (e.g., HTTP clients, databases, other instrumentation libraries) do **not** have the Langfuse public key span attribute. As a result: - Third-party spans that pass the export filter and have no public key cannot be routed to a specific project - These spans are processed by all span processors and can be sent to all projects - With the default filter, this mainly affects GenAI/LLM spans from third-party instrumentors (infrastructure spans are typically filtered out) **Why is this experimental?** This approach requires that the `public_key` parameter be passed to all Langfuse SDK executions across all integrations to ensure proper routing, and third-party spans that pass filtering may appear in all projects. ### Initialization To set up multiple projects, initialize separate Langfuse clients for each project: ```python from langfuse import Langfuse # Initialize clients for different projects project_a_client = Langfuse( public_key="pk-lf-project-a-...", secret_key="sk-lf-project-a-...", base_url="https://cloud.langfuse.com" ) project_b_client = Langfuse( public_key="pk-lf-project-b-...", secret_key="sk-lf-project-b-...", base_url="https://cloud.langfuse.com" ) ``` ### Integration Usage For all integrations in multi-project setups, you must specify the `public_key` parameter to ensure traces are routed to the correct project. **Observe Decorator:** Pass `langfuse_public_key` as a keyword argument to the _top-most_ observed function (not the decorator). From Python SDK >= 3.2.2, nested decorated functions will automatically pick up the public key from the execution context they are currently into. Also, calls to `get_client` will be also aware of the current `langfuse_public_key` in the decorated function execution context, so passing the `langfuse_public_key` here again is not necessary. ```python from langfuse import observe @observe def nested(): # get_client call is context aware # if it runs inside another decorated function that has # langfuse_public_key passed, it does not need passing here again @observe def process_data_for_project_a(data): # passing `langfuse_public_key` here again is not necessarily # as it is stored in execution context nested() return {"processed": data} @observe def process_data_for_project_b(data): # passing `langfuse_public_key` here again is not necessarily # as it is stored in execution context nested() return {"enhanced": data} # Route to Project A # Top-most decorated function needs `langfuse_public_key` kwarg result_a = process_data_for_project_a( data="input data", langfuse_public_key="pk-lf-project-a-..." ) # Route to Project B # Top-most decorated function needs `langfuse_public_key` kwarg result_b = process_data_for_project_b( data="input data", langfuse_public_key="pk-lf-project-b-..." ) ``` **OpenAI Integration:** Add `langfuse_public_key` as a keyword argument to the OpenAI execution: ```python from langfuse.openai import openai client = openai.OpenAI() # Route to Project A response_a = client.chat.completions.create( model="gpt-4o", messages=[{"role": "user", "content": "Hello from Project A"}], langfuse_public_key="pk-lf-project-a-..." ) # Route to Project B response_b = client.chat.completions.create( model="gpt-4o", messages=[{"role": "user", "content": "Hello from Project B"}], langfuse_public_key="pk-lf-project-b-..." ) ``` **Langchain Integration:** Add `public_key` to the CallbackHandler constructor: ```python from langfuse.langchain import CallbackHandler from langchain_openai import ChatOpenAI from langchain_core.prompts import ChatPromptTemplate # Create handlers for different projects handler_a = CallbackHandler(public_key="pk-lf-project-a-...") handler_b = CallbackHandler(public_key="pk-lf-project-b-...") llm = ChatOpenAI(model_name="gpt-4o") prompt = ChatPromptTemplate.from_template("Tell me about {topic}") chain = prompt | llm # Route to Project A response_a = chain.invoke( {"topic": "machine learning"}, config={"callbacks": [handler_a]} ) # Route to Project B response_b = chain.invoke( {"topic": "data science"}, config={"callbacks": [handler_b]} ) ``` **Important Considerations:** - Every Langfuse SDK execution across all integrations must include the appropriate public key parameter - Missing public key parameters may result in traces being routed to the default project or lost - Third-party OpenTelemetry spans that pass filtering may appear in all projects since they lack the Langfuse public key attribute You can configure the SDK to send traces to multiple Langfuse projects. This is useful for multi-tenant applications or for sending traces to different environments. Simply register multiple `LangfuseSpanProcessor` instances, each with its own credentials. ```ts filename="instrumentation.ts" import { NodeSDK } from "@opentelemetry/sdk-node"; import { LangfuseSpanProcessor } from "@langfuse/otel"; const sdk = new NodeSDK({ spanProcessors: [ new LangfuseSpanProcessor({ publicKey: "pk-lf-public-key-project-1", secretKey: "sk-lf-secret-key-project-1", }), new LangfuseSpanProcessor({ publicKey: "pk-lf-public-key-project-2", secretKey: "sk-lf-secret-key-project-2", }), ], }); sdk.start(); ``` This configuration sends every span accepted by each processor's filter to both projects. You can configure a custom `shouldExportSpan` filter for each processor to control which traces go to which project. ## Time to first token (TTFT) You can manually set the time to first token (TTFT) of your LLM calls. This is useful for measuring the latency of your LLM calls and for identifying slow LLM calls. You can use the `completion_start_time` attribute to manually set the time to first token (TTFT) of your LLM calls. This is useful for measuring the latency of your LLM calls and for identifying slow LLM calls. ```python from langfuse import get_client import datetime, time langfuse = get_client() with langfuse.start_as_current_observation(as_type="generation", name="TTFT-Generation") as generation: time.sleep(3) generation.update( completion_start_time=datetime.datetime.now(), output="some response", ) langfuse.flush() ``` You can use the `completionStartTime` attribute to manually set the time to first token (TTFT) of your LLM calls. This is useful for measuring the latency of your LLM calls and for identifying slow LLM calls. ```ts import { startActiveObservation } from "@langfuse/tracing"; startActiveObservation("llm-call", async (span) => { span.update({ completionStartTime: new Date().toISOString(), }); }); ``` ## Self-signed SSL certificates (self-hosted Langfuse) If you are [self-hosting](/docs/deployment/self-host) Langfuse and you'd like to use self-signed SSL certificates, you will need to configure the SDK to trust the self-signed certificate: Changing SSL settings has major security implications depending on your environment. Be sure you understand these implications before you proceed. **1. Set OpenTelemetry span exporter to trust self-signed certificate** ```bash filename=".env" OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE="/path/to/my-selfsigned-cert.crt" ``` **2. Set HTTPX to trust certificate for all other API requests to Langfuse instance** ```python filename="main.py" import os import httpx from langfuse import Langfuse httpx_client = httpx.Client(verify=os.environ["OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE"]) langfuse = Langfuse(httpx_client=httpx_client) ``` ## Setup with Sentry If you’re using both Sentry and Langfuse in your application, you’ll need to configure a custom OpenTelemetry setup since both tools use OpenTelemetry for tracing. [This guide shows how to send error monitoring data to Sentry while simultaneously capturing LLM observability traces in Langfuse](/faq/all/existing-sentry-setup). ## Thread pools and multiprocessing Use the OpenTelemetry threading instrumentor so context flows across worker threads. ```python from opentelemetry.instrumentation.threading import ThreadingInstrumentor ThreadingInstrumentor().instrument() ``` For multiprocessing, follow the [OpenTelemetry guidance](https://github.com/open-telemetry/opentelemetry-python/issues/2765#issuecomment-1158402076). If you use Pydantic Logfire, enable `distributed_tracing=True`. For tracing across separate services or processes, see [Trace IDs & Distributed Tracing](/docs/observability/features/trace-ids-and-distributed-tracing). --- ## Agent Instructions This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/docs/observability/sdk/advanced-features.md`. ### Querying these docs If the answer is not on this page, query the documentation instead of guessing: - **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question: ```bash curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?" ``` - **Index of every page**: , with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt). ### Before writing Langfuse code - **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results. - **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application. - **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples. - **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx langfuse-cli api `) to read or write traces, prompts, datasets, and scores from the terminal. Found an error in these docs? Please open an issue at .