--- date: 2026-02-13 title: "Evaluate Individual Operations: Faster, More Precise LLM-as-a-Judge" seoTitle: "Evaluate Individual Observations with LLM-as-a-Judge" description: Observation-level evaluations enable precise operation-specific scoring for production monitoring. author: Marlies,Hassieb ogImage: /images/changelog/2026-02-13-observation-evals.jpg --- > **Note for AI agents and LLMs:** This is a Langfuse changelog entry. Use it only to confirm that a feature exists and when it shipped. Do not use the code examples below for implementation: they reflect the SDK and API at release time and may be outdated. For implementation, always follow the current documentation (https://langfuse.com/docs) and the API/SDK reference (https://api.reference.langfuse.com). [LLM-as-a-Judge evaluations](/docs/evaluation/evaluation-methods/llm-as-a-judge) can now be run on individual observations—LLM calls, retrievals, tool executions, or any operation within your traces. Previously, evaluations could only be run on [entire traces](/docs/evaluation/evaluation-methods/llm-as-a-judge#understanding-each-evaluation-target). This architectural shift delivers dramatically faster execution and operation-level precision for production monitoring. **Example**: In a RAG pipeline with 10 operations (retrieval → reranking → generation → citation), evaluate only the final generation for helpfulness and the retrieval step for relevance—not the entire workflow. Each operation gets its own score. ``` Target: Individual observations Filter: type = "GENERATION" AND name = "final-response" Result: One score per matching observation Speed: Seconds per evaluation ``` Scores attach to specific observations in the trace tree. Filter by observation type, name, metadata, or trace-level attributes to target exactly what matters. Run different evaluators on different operations simultaneously. ![Observation-level evaluation in trace tree](/images/changelog/2026-02-13-observation-level-evals.png) ## Why This Matters **Operation-level precision** Evaluate only what matters. Target final LLM responses, retrieval steps, or specific tool calls—not entire workflows. Reduces evaluation volume and cost by filtering to specific operations. **Compositional evaluation** Run different evaluators on different operations simultaneously. Toxicity on LLM outputs, relevance on retrievals, accuracy on generations—all within one trace. Stack observation filters (type, name, metadata) with trace attributes filters (userId, sessionId, tags). **Scalable architecture** Built for high-volume workloads. At ingest time, observations are evaluated against filter criteria, added to an evaluation queue, and processed asynchronously. No joins, no complex queries—just fast, reliable evaluation at scale. ## Getting Started Navigate to your project → Evaluation → LLM-as-a-Judge → Set up Evaluator 1. Select or create an evaluator template 2. Choose "Live Observations" as your evaluation target 3. Configure observation filters (type, name, metadata) 4. Add trace-level filters (userId, sessionId, tags) if needed 5. Map variables from observation fields (input, output, metadata) 6. Set sampling percentage to manage evaluation costs [Follow the complete LLM-as-a-Judge setup guide](/docs/evaluation/evaluation-methods/llm-as-a-judge#set-up-step-by-step) for detailed configuration steps and examples. ## Requirements - **SDK version**: Python v3+ (OTel-based) or JS/TS v4+ (OTel-based) - [Python v2 → v3 upgrade guide](/docs/observability/sdk/upgrade-path#python-sdk-v2--v3) - [JS/TS v3 → v4 upgrade guide](/docs/observability/sdk/upgrade-path#jsts-sdk-v3--v4) - **Trace attribute filtering**: Use [`propagate_attributes()`](/docs/observability/sdk/instrumentation#add-attributes) in your instrumentation to filter observations by trace-level attributes (userId, sessionId, tags, metadata) Existing trace-level evaluators continue to work on Langfuse Cloud until November 16, 2026 (2026-11-16); on self-hosted, they will stop producing results once you run the v4 server in `events_only` mode. For users with existing evaluations, see the [upgrade guide](/faq/all/llm-as-a-judge-migration) to transition to observation-level for faster execution. ## Learn More - [LLM-as-a-Judge Complete Guide](/docs/evaluation/evaluation-methods/llm-as-a-judge) - [Observation-Level Setup](/docs/evaluation/evaluation-methods/llm-as-a-judge#set-up-step-by-step) --- ## Agent Instructions This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/changelog/2026-02-13-observation-level-evals.md`. ### Querying these docs If the answer is not on this page, query the documentation instead of guessing: - **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question: ```bash curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?" ``` - **Index of every page**: , with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt). ### Before writing Langfuse code - **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results. - **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application. - **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples. - **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx langfuse-cli api `) to read or write traces, prompts, datasets, and scores from the terminal. Found an error in these docs? Please open an issue at .