---
title: "06 Experiments"
description: "Learner guide: 06 Experiments"
---
# 06 Experiments

Workshop material is maintained in the public [`langfuse/langfuse-workshop`](https://github.com/langfuse/langfuse-workshop) repository. Use the repository for the runnable app, checkpoint branches, and local setup.

[View this Markdown file](https://github.com/langfuse/langfuse-workshop/blob/main/docs/instructor/06-experiments.md)

Learner guide: [06 Experiments](/workshop/learner/06-experiments)

## Instructor notes

- The key idea is reuse: the experiment runner calls the same `runSupportConversation(...)` as the web app.
- Contrast deterministic scoring in the script (`keyword_overlap`) with LLM-as-a-judge scoring (`correctness`).
- Confirm the default evaluator model before the **Check Correctness** setup. If learners did not configure it in session 4, send them to **Project Settings → LLM Connections** first.
- In the evaluator's sample panel, choose **Experiments** so the filter becomes `isExperimentItemRootSpan: true`. Correctness needs experiment context because that is where expected output is available.
- The current template mapping has two variables: `output` maps to the experiment item's **Output**, and `expected_output` maps to **Expected Output → idealAnswer**.
- Emphasize the mixed setup: the script owns the cheap deterministic check, while Langfuse owns the semantic judge.
- Keep concurrency at one for workshops so traces and the final run summary are easy to follow.
- Before running the dataset, confirm both the OpenAI and Langfuse credentials point to active projects. The runner can finish with exit code 0 after skipping every item, so always check that the console prints experiment results rather than only SDK errors.

## Demo rhythm

1. Skim the numbered sections in `scripts/run-dataset.ts`.
2. Point out the `keyword_overlap` evaluator inside the script.
3. Create **Check Correctness**, filter the sample panel to **Experiments**, and map `output` plus `expected_output`.
4. Run `npm run dataset:run`.
5. Open the run table, per-item traces, and chart view.

## Watch for

- Keep **Check Correctness** filtered to **Experiments**. A live-observation target does not provide the dataset ground truth required by this evaluator.
- If no experiment run exists yet, the right-side sample table may show zero matches. That is expected before `npm run dataset:run` creates a baseline run.
- Map `output` to **Output** with no path, and map `expected_output` to **Expected Output → idealAnswer**. Do not map the whole expected-output object.
- Do not use `expectedKeywords` as the correctness reference. That field belongs to the in-script `keyword_overlap` evaluator.
- Learners assuming the deterministic check must live in Langfuse now. It does not; mention the code-evaluator docs only as an alternative.
- "No default model set" means Langfuse needs an LLM connection/default evaluator model; it is not fixed by editing `.env`.
- `401 The project you are requesting has been archived` followed by `No experiment results to display` means the OpenAI key belongs to an archived project. Replace `OPENAI_API_KEY` locally with a key from an active project, then rerun.
- Slow asynchronous evaluator results; the console only shows the final summary, so refresh Langfuse after the run finishes if `correctness` is still pending.

<!-- agent-instructions -->

---

## Agent Instructions

This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/workshop/instructor/06-experiments.md`.

### Querying these docs

If the answer is not on this page, query the documentation instead of guessing:

- **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question:

  ```bash
  curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?"
  ```

- **Index of every page**: <https://langfuse.com/llms.txt>, with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt).

### Before writing Langfuse code

- **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results.
- **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application.
- **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples.
- **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx langfuse-cli api <resource> <action>`) to read or write traces, prompts, datasets, and scores from the terminal.

Found an error in these docs? Please open an issue at <https://github.com/langfuse/langfuse-docs/issues>.