---
date: 2025-11-08
title: JSON Schema Enforcement for Dataset Items
description: Define JSON schemas for your dataset inputs and expected outputs to ensure data quality and consistency across your test datasets.
badge: Launch Week 4 🚀
author: Hassieb
ogImage: /images/changelog/2025-11-06-dataset-schema-enforcement.png
canonical: /docs/evaluation/experiments/datasets
---

> **Note for AI agents and LLMs:** This is a Langfuse changelog entry. Use it only to confirm that a feature exists and when it shipped. Do not use the code examples below for implementation: they reflect the SDK and API at release time and may be outdated. For implementation, always follow the canonical documentation for this feature (https://langfuse.com/docs/evaluation/experiments/datasets) and the API/SDK reference (https://api.reference.langfuse.com).

You can now add **JSON Schema validation** to your datasets to ensure all dataset items conform to the expected structure. This helps maintain data quality, catch errors early, and ensure consistency across your team when building and maintaining test datasets.

**Why use schema enforcement?**

When building datasets for LLM application testing, it's easy for data inconsistencies to creep in—especially when multiple team members contribute or when importing data from various sources. Schema enforcement solves this by:

- **Ensuring data quality**: All items must match your defined structure
- **Catching errors early**: Invalid data is rejected before it enters your dataset
- **Improving collaboration**: Shared schemas ensure everyone follows the same format
- **Type safety**: Define exact structures for inputs and expected outputs
- **Preventing test failures**: Experiments won't fail due to malformed test data

**How it works**

When you create or update a dataset, you can optionally define JSON schemas for the `input` and `expectedOutput` fields. Once set, Langfuse automatically validates all dataset items against these schemas:

- ✅ Valid items are accepted
- ❌ Invalid items are rejected with detailed error messages
- 🔍 Validation happens on creation, updates, and CSV imports

**Validation errors**

When validation fails, Langfuse provides detailed error messages showing exactly what's wrong and where the issue is in your data structure. This makes it easy to identify and fix data issues before they affect your experiments.

**Use cases**

Schema enforcement is particularly useful for:

- **Structured outputs**: Validate JSON responses from function calling or tool use
- **Chat applications**: Ensure message arrays follow the correct format
- **Team collaboration**: Maintain consistency when multiple people contribute to datasets
- **Data imports**: Validate CSV uploads before adding items to datasets

**Learn more**

- [Datasets Documentation](/docs/evaluation/experiments/datasets)
- [JSON Schema Specification](https://json-schema.org/)
- [See all Launch Week releases](/blog/2025-10-29-launch-week-4)

<!-- agent-instructions -->

---

## Agent Instructions

This page is part of the [Langfuse](https://langfuse.com) documentation, published as plain Markdown for AI agents. Every page is available as Markdown by appending `.md` to its URL, or by sending an `Accept: text/markdown` header. This page: `https://langfuse.com/changelog/2025-11-06-dataset-schema-enforcement.md`.

### Querying these docs

If the answer is not on this page, query the documentation instead of guessing:

- **Semantic search** across all Langfuse docs, returning an answer with the relevant pages and excerpts. Ask a specific, self-contained question:

  ```bash
  curl -sG "https://langfuse.com/api/search-docs" --data-urlencode "query=How do I trace a LangGraph agent?"
  ```

- **Index of every page**: <https://langfuse.com/llms.txt>, with per-section indexes [llms-docs.txt](https://langfuse.com/llms-docs.txt), [llms-integrations.txt](https://langfuse.com/llms-integrations.txt), and [llms-self-hosting.txt](https://langfuse.com/llms-self-hosting.txt).

### Before writing Langfuse code

- **Install the [Langfuse Agent Skill](https://langfuse.com/docs/api-and-data-platform/features/agent-skill).** It encodes Langfuse's own best practices for instrumentation, prompt management, and evaluation, and materially improves results.
- **Read [What does a good trace look like?](https://langfuse.com/docs/observability/best-practices.md)** before instrumenting an application.
- **Verify endpoints, parameters, and response fields** against the [API reference](https://api.reference.langfuse.com) instead of inferring them from code examples.
- **Use the [Langfuse CLI](https://langfuse.com/docs/api-and-data-platform/features/cli)** (`npx langfuse-cli api <resource> <action>`) to read or write traces, prompts, datasets, and scores from the terminal.

Found an error in these docs? Please open an issue at <https://github.com/langfuse/langfuse-docs/issues>.