---
title: "How to evaluate sessions and conversations"
tags: [guide]
description: "How to evaluate entire sessions in Langfuse rather than individual traces: session scores, LLM-as-a-judge on conversations, annotation, and SDK patterns."
---

# How to evaluate sessions/conversations?

This guide explains how to evaluate entire [sessions](/docs/observability/features/sessions) (such as conversations, threads, etc.), rather than just individual traces.
The information here applies to all [evaluation methods](/docs/evaluation/overview) supported by Langfuse.

Scores in Langfuse can be assigned to traces, observations, or sessions (see the [data model](/docs/evaluation/scores/data-model#scores)).

## Session-level scores

1. You can add a [custom score](/docs/evaluation/evaluation-methods/custom-scores) that references the session ID.
2. You can annotate the session using [annotation queues](/docs/evaluation/evaluation-methods/annotation#annotation-queues).

## LLM-as-a-Judge evaluators [#llm-as-a-judge]

[LLM-as-a-Judge evaluators](/docs/evaluation/evaluation-methods/llm-as-a-judge) can be applied to observations (recommended) or traces (legacy).
They cannot be applied directly to sessions, as Langfuse does not inherently know when a session has concluded.

It is recommended to evaluate the conversation using an observation-level LLM-as-a-Judge evaluator. Target an observation that contains the complete conversation history, typically the root observation or the LLM call that receives the conversation history as input (i.e., conversation history or short-term memory). The evaluator only sees data on the observation it matches, so make sure your application writes the full history onto that observation.

You can either:

1. Evaluate the conversation on every trace of the session.
2. Or apply a special [tag](/docs/observability/features/tags) to the final trace of the session (e.g., `conversation_end`) and configure the evaluator to only run on observations with this tag. Observation-level evaluators check filter attributes like tags on the observation itself, so propagate trace-level attributes to observations with `propagate_attributes()` (Python) or `propagateAttributes()` (JS/TS); see [this FAQ](/faq/all/observation-eval-not-executing) for details. This approach can help reduce evaluation costs and make metrics more stable.

In both cases, the score will be assigned to the matched observation.

Trace-level LLM-as-a-Judge evaluators are legacy: on Langfuse Cloud they stop producing results at the v4 cutover on November 16, 2026 (2026-11-16). On self-hosted, they will stop producing results once you run the v4 server in `events_only` mode. See the [upgrade guide](/faq/all/llm-as-a-judge-migration) for migrating them to observation-level evaluators.

<!-- agent-instructions -->

---

## Agent Instructions

This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/resources/engineering/evaluating-sessions-conversations.md`.

### Querying these docs

If the answer is not on this page, query the documentation instead of guessing:

- **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question:

  ```bash
  curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?"
  ```

- **Index of every page**: <https://langfuse.com/llms.txt>, with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt).

### Before writing Langfuse code

- **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results.
- **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application.
- **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples.
- **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx langfuse-cli api <resource> <action>`) to read or write traces, prompts, datasets, and scores from the terminal.

Found an error in these docs? Please open an issue at <https://github.com/langfuse/langfuse-docs/issues>.