← Back to changelog

August 24, 2026

Create evaluators from templates

Annabell Schäfer

Use established scoring approaches for chatbots, topic detection, exact matches, and coding agents.

You no longer have to write every evaluator from a blank screen. Open New evaluator on the Evaluators page to browse the template gallery, then pick an approach and adapt it to your data.

• Conversational. Identify where interactions between users and agents went wrong by catching implicit user signals.
• Topic detection. Classify what users are talking about without standing up a custom classifier.
• Exact matches. Run deterministic checks when the expected output is known.
• Coding agents. Detect what your coding agent is doing, from tool calls to task progress.

Each template prepopulates a new LLM-as-a-Judge or code evaluator you can edit.

Learn more