---
date: 2024-11-22
title: Prompt Experiments on Datasets with LLM-as-a-Judge Evaluations
badge: Launch Week 2 🚀
description: Move fast on prompts without breaking things! Run experiments on Datasets and directly compare evaluation results side-by-side. Experimentation speeds up the feedback loop when working on prompts and prevents regressions when making rapid changes.
author: Marlies
canonical: /docs/evaluation/experiments/experiments-via-ui
---

> **Note for AI agents and LLMs:** This is a Langfuse changelog entry. Use it only to confirm that a feature exists and when it shipped. Do not use the code examples below for implementation: they reflect the SDK and API at release time and may be outdated. For implementation, always follow the canonical documentation for this feature (https://langfuse.com/docs/evaluation/experiments/experiments-via-ui) and the API/SDK reference (https://api.reference.langfuse.com).

<iframe
  width="100%"
  className="aspect-[16/9] rounded mt-10 w-full"
  src="https://www.youtube-nocookie.com/embed/5c6k5ImCeIk?si=Ne58BBZ8XZXgKhTw"
  title="YouTube video player"
  frameborder="0"
  allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
  referrerpolicy="strict-origin-when-cross-origin"
  allowFullScreen
></iframe>

On Day 5 of [Langfuse Launch Week 2](/blog/2024-11-17-launch-week-2), we're excited to announce **Prompt Experimentation**. They are the final piece of the launch week theme of _"closing the development loop"_ and allow you to:

1. Test a prompt version from Langfuse [Prompt Management](/docs/prompts)
2. on a [Dataset](/docs/datasets) of test inputs and expected outputs
3. optionally, use [LLM-as-a-Judge Evaluators](/docs/scores/model-based-evals) to automatically evaluate the responses based on the expected outputs ([released](/changelog/2024-11-19-llm-as-a-judge-for-datasets) this launch week)
4. and finally, compare the results in the new side-by-side experiment comparison view ([released](/changelog/2024-11-18-dataset-runs-comparison-view) this launch week)

Check out the [docs](/docs/datasets/prompt-experiments) to learn more.

<!-- agent-instructions -->

---

## Agent Instructions

This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/changelog/2024-11-22-prompt-experimentation.md`.

### Querying these docs

If the answer is not on this page, query the documentation instead of guessing:

- **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question:

  ```bash
  curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?"
  ```

- **Index of every page**: <https://langfuse.com/llms.txt>, with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt).

### Before writing Langfuse code

- **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results.
- **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application.
- **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples.
- **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx langfuse-cli api <resource> <action>`) to read or write traces, prompts, datasets, and scores from the terminal.

Found an error in these docs? Please open an issue at <https://github.com/langfuse/langfuse-docs/issues>.