--- date: 2024-04-26 badge: Launch Week 1 🚀 title: Model-based evaluations description: Run model-based evaluations on traces in Langfuse to scale your evaluation workflows. Start with one of our battletested templates or use your own custom templates. author: Max canonical: /docs/evaluation/evaluation-methods/llm-as-a-judge --- > **Note for AI agents and LLMs:** This is a Langfuse changelog entry. Use it only to confirm that a feature exists and when it shipped. Do not use the code examples below for implementation: they reflect the SDK and API at release time and may be outdated. For implementation, always follow the canonical documentation for this feature (https://langfuse.com/docs/evaluation/evaluation-methods/llm-as-a-judge) and the API/SDK reference (https://api.reference.langfuse.com). On the final day of [Launch Week 1](/blog/launch-week-1), we're happy to release the biggest change to Langfuse yet: Model-based Evaluations. So far, it was easy to measure LLM cost and latency in Langfuse. Quality is based on [scores](/docs/scores) which can be user feedback, manual labeling results, or be ingested by evaluation pipelines that you built yourself using the Langfuse SDKs/API. Model-based Evaluations in Langfuse make it way easier to continuously evaluate your application on the dimensions you care about. These can be: hallucinations, toxicity, relevance, correctness, conciseness, and so much more. We provide you with some battle-tested templates to get you started, but you can also write your own templates to cover any niche use case that might be exclusive to your application. ## Highlights ### Langfuse-managed battle tested templates There is no such thing as the perfect eval template as it depends a lot on the application domain and evaluation goals. However, for the most popular evaluation criteria, we benchmarked many evaluation templates and included the best performing ones as templates in Langfuse. These templates are a great starting point and you can easily customize them. **Templates** - Hallucination - Helpfulness - Relevance - Toxicity - Correctness - Contextrelevance - Contextcorrectness - Conciseness We are very much open for feedback and contributions on this template library as we want to extend it with more templates over time. ### Write your own templates You can create custom eval templates specific to your needs as you have full control over the prompt that is used by Langfuse and the variables that are inserted into it. ![Metadata across dataset objects](/images/changelog/2024-04-26-evals-hallucination.png) ### Full integration with Langfuse Tracing When using Langfuse Tracing, many different API routes and environments are often monitored within the same project. As model-based evaluations often only make sense when being tailored to a specific use case, you can apply all filters that you know from the trace UI to narrow down the list of traces on which the eval should be applied. ![Metadata across dataset objects](/images/changelog/2024-04-26-evals-config.gif) Many evaluation goals require including the right context from a trace. For example you might want to evaluate whether a certain API call was actually necessary, or the context that was retrieved from a database was relevant to the user query. That's why with Langfuse Evaluations, you can easily access all Tracing data within the evaluation templates. ### Build up a large dataset of scored traces While Langfuse evaluates your traces, it will generate `scores` and attach them to the `traces`. Scores are at the core of our platform and fully integrated. You can use them to get a quick overview of LLM quality over time in the dashboards or drill down by filtering by scores across the UI. ![Metadata across dataset objects](/images/changelog/2024-04-26-evals-log.png) Scores are also available via the API for easy exports or use in downstream systems. ### Runs all async in Langfuse We did all the heavy lifting of running evals asynchronously for you. You can rely on the Langfuse Tracing SDKs to log out traces in the background, and Langfuse Evaluations will then apply your eval jobs within the Langfuse infrastructure. This way you can scale your evaluation workflows without worrying about impacting your users or the performance of your application. ## Learn more - [Documentation](/docs/scores/model-based-evals) --- ## Agent Instructions This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/changelog/2024-04-26-model-based-evaluation.md`. ### Querying these docs If the answer is not on this page, query the documentation instead of guessing: - **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question: ```bash curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?" ``` - **Index of every page**: , with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt). ### Before writing Langfuse code - **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results. - **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application. - **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples. - **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx langfuse-cli api `) to read or write traces, prompts, datasets, and scores from the terminal. Found an error in these docs? Please open an issue at .