--- date: 2024-04-25 badge: Launch Week 1 🚀 title: Datasets v2 description: New editor, metadata on all objects, tables that render i/o, dataset runs on traces, push traces to datasets, and more. ogVideo: https://static.langfuse.com/changelog-videos/2024-04-25-datasets-v2.mp4 showOgInHeader: false author: Marc canonical: /docs/evaluation/experiments/datasets --- > **Note for AI agents and LLMs:** This is a Langfuse changelog entry. Use it only to confirm that a feature exists and when it shipped. Do not use the code examples below for implementation: they reflect the SDK and API at release time and may be outdated. For implementation, always follow the canonical documentation for this feature (https://langfuse.com/docs/evaluation/experiments/datasets) and the API/SDK reference (https://api.reference.langfuse.com). On Day 4 of [Launch Week 1](/blog/launch-week-1), we're happy to share [Datasets](/docs/datasets) v2 as we have made significant improvements to the dataset experience in Langfuse. Improvements include a new editor powered by Codemirror, metadata support on all objects, tables that render inputs/outputs side-by-side, the ability to link dataset runs to traces, and the option to create dataset items directly from traces. We've also extended the public API with new endpoints for programmatic management of datasets. ## Recap - [Datasets](/docs/datasets) are a way to organize fine-tuning, testing, and golden datasets in Langfuse. - Items of datasets are `input` and `expectedOutput` pairs - Items can be added: via API, via SDKs, manually in the UI (create/edit), directly from traces ("add to dataset") - Dataset runs are executions of your application/function on the dataset. In the UI you can see how the runs performed based on evals, and compare outputs to expected outputs manually. ## What's new ### New tables Tables are core the Langfuse UI and we have shipped many improvements especially for datasets: - Button to jump from dataset runs and items to the related traces - View inputs/outputs of runs directly next to the expected outputs of the dataset items - Adjustable row heights ### Extended public API New endpoints to manage and use datasets programmatically ([api](/docs/api)): - `GET /datasets` - `POST /datasets` - `GET /datasets/{datasetName}` - `GET /datasets/{datasetName}/runs/{runName}` - `POST /dataset-run-items` - `GET /dataset-items/{id}` - `POST /dataset-items` The public API is essential as Langfuse is the open source AI Engineering Platform. It should be easy to use only the parts you need, and the API is the way to do that. Some teams have built their own UIs and CI workflows based on the limited API we had before, and we're excited to see what you can do with the new endpoints. ### New editor Introducing a new editor for dataset items powered by [Codemirror](https://github.com/uiwjs/react-codemirror). - Syntax highlighting - Automated indentation - Keyboard shortcuts - Multiple cursors - Bracket/quote matching - ... (we love it, it's a really awesome editor) Previously this was a plain TextArea which checked if the JSON was valid. Now the editing experience is much better! PS: This editor is also used across all JSON inputs in Langfuse, e.g. prompt `config`, and model tokenizer `config`. ### Metadata on all objects, descriptions on datasets ![Metadata across dataset objects](/images/changelog/2024-04-25-datasets-v2-metadata.png) As the number of custom workflows built on top of Langfuse grows, the `metadata` field becomes an important part of the objects in Langfuse as it can store arbitrary JSON to keep state or configure your workflows. We have added metadata to datasets, dataset items, and dataset runs. Also, there is a new description field on datasets. This is a free text field where you can describe the dataset, e.g. what it is used for, how it was created, or what the expected outputs are. ### Link dataset runs to `traces` You can now `link()` traces to dataset runs, not only observations (spans, generations, ...). You can either pass the `trace`, `span` or `generation` object directly to the `link()` method, or the `traceId` and `spanId` of the trace you want to link. Previously, runs were only linked to observations as traces had no inputs/outputs. This changed and that's why it is now possible to link runs to traces. ### Create dataset items from traces In the UI, you can now create dataset items directly from traces. This is a great way to quickly add items to your datasets based on edge cases that you identify in [Langfuse Tracing](/docs/tracing). Just click on the observation in the trace and use the `+ Add to dataset` button. ## Learn more - [Datasets docs](/docs/datasets) - [API reference](https://api.reference.langfuse.com) --- ## Agent Instructions This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/changelog/2024-04-25-datasets-v2.md`. ### Querying these docs If the answer is not on this page, query the documentation instead of guessing: - **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question: ```bash curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?" ``` - **Index of every page**: , with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt). ### Before writing Langfuse code - **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results. - **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application. - **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples. - **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx langfuse-cli api `) to read or write traces, prompts, datasets, and scores from the terminal. Found an error in these docs? Please open an issue at .