← Back to changelog

July 10, 2026

Evaluate tool calls

Tobias Wochinger

Access recorded tool calls directly in code and LLM-as-a-Judge evaluators.

Code and LLM-as-a-Judge evaluators can now access tool calls through a dedicated structured field - no need to extract them from raw observation input.

For example, check whether an agent called search:

used_search = any(
    tool_call.name == "search" for tool_call in ctx.observation.tool_calls
)

const usedSearch = ctx.observation.toolCalls.some(
  (toolCall) => toolCall.name === "search",
);

Each call contains id, name, arguments, type, and index.

For LLM-as-a-Judge, map a variable to Tool Calls. Use $[*].name to pass only tool names or the full array to include arguments.

Learn more