Langfuse launches AI Gateway Learn more
FaqHow do I use TypeSafe Jev?

How do I use TypeSafe Jev?

Add a connection with the adapter typesafe, choose the upstream that serves Jev, and paste its API key. Then select that connection in a decision-model evaluator.

UpstreamAPI keyBilling
TypeSafeA key from the TypeSafe consoleBilled by TypeSafe
Vercel AI GatewayAn AI Gateway API keyBilled through AI Gateway
OpenRouterAn OpenRouter API keyBilled through OpenRouter

These upstreams need no base URL or extra headers. Create an account at typesafe.ai to generate a TypeSafe key. Access currently runs through a waitlist on TypeSafe's side. If you already use Vercel AI Gateway or OpenRouter, pick that upstream instead and use its API key.

Use jev-latest to follow new releases, or pin a version such as jev-1.13.0 when an evaluation threshold should stay stable.

Build the JSON state from observation input, output, metadata, tool calls, Expected Output, or Experiment Item Metadata. Reference those fields in question instructions with backticks, for example Compare `output` with `expectedOutput`. Jev Score levels require a description.

TypeSafe accepts roughly 32k tokens for the state plus the longest question. Keep the input focused on the data needed for the decision. Jev always picks an answer, so add an other or unclear option when input may be thin.

TypeSafe connections are only available to decision-model evaluators because Jev cannot generate text.


Was this page helpful?

Last updated on