Jev Directory

Evaluation / Scoring

Check a release note in one call

Ask independent questions about one document without resending the context.

NoulAdapted exampleSource reviewedNot live-tested

01 The questions

breaking_change noul

Does this release note describe a change that can break an existing integration?

true
An existing endpoint, format, or supported behavior is removed or changed incompatibly.
false
No incompatibility is described.
migration_steps noul

Does the release note provide an explicit action a user can take to migrate?

true
At least one concrete migration action is stated.
false
Migration is not addressed or only mentioned in general terms.

Copy includes the complete instructions, criteria, usage notes, and attribution.

02 Input

One complete release note shared by all questions.

Illustrative input · JSON
{
  "document": "Version 3 removes the legacy /v1/export endpoint. Migrate to /v2/export before upgrading. Existing saved reports are unaffected."
}

03 Answer & policy

Independent Noul probabilities for a breaking change and for migration instructions.

Your code decides what happens next.

Read each answer separately. The second question does not depend on the first answer, so both can share one request. The example logs signals rather than declaring a release safe.

04 Use it in your code

Node.js 24 · TypeSafe SDK 0.6.0 · Set TYPESAFE_API_KEY in your environment. Run on your server; API calls incur provider charges.

decision.ts · TypeScript
import { noul, TypeSafeClient, type JsonValue } from "@typesafe-ai/sdk";

// Illustrative input, not a recorded model test.
const state: JsonValue = {
  "document": "Version 3 removes the legacy /v1/export endpoint. Migrate to /v2/export before upgrading. Existing saved reports are unaffected."
};
const client = new TypeSafeClient();
try {
  const response = await client.systemOne({
    model: "jev-latest",
    state,
    questions: {
      breaking_change: noul(
        "Does this release note describe a change that can break an existing integration?",
        {
          "true": "An existing endpoint, format, or supported behavior is removed or changed incompatibly.",
          "false": "No incompatibility is described."
        }
      ),
      migration_steps: noul(
        "Does the release note provide an explicit action a user can take to migrate?",
        {
          "true": "At least one concrete migration action is stated.",
          "false": "Migration is not addressed or only mentioned in general terms."
        }
      )
    },
  });

  console.log({
    breakingChangeProbability: response.answers.breaking_change.noul,
    migrationStepsProbability: response.answers.migration_steps.noul,
  });
} catch (error) {
  console.error("Decision unavailable; use your fallback or human review.", error);
  process.exitCode = 1;
}
Download .ts ↓SDK setup ↗Copy and download include license notices.

The wrapper and example input are provided by Jev Directory. Checked against SDK types; no live model call was made. Pin a model version before evaluating production behavior.

Before you adapt it

  • If a question needs a previous answer, use a later request instead.
  • Batching benefits vary with workload; no speed or cost result is claimed here.
More about the original project or pattern

What it does

This cookbook evaluates thirteen independent questions against one long document. It compares one batched request with separate single-question requests, repeats each approach, and records answers, token usage, and latency. The example mixes Noul, Choice, and Score questions in one call.

What you can reuse

The useful pattern is to send shared state once and attach several atomic questions. Each question keeps a narrow key and rubric, while code reduces the typed answers into the values the application needs. The example also pins its source document revision and caches API responses, making repeated inspection reproducible without new calls.

How it fits

A document is the shared state. Independent questions form a keyed map. Jev returns answers under the same keys, and downstream code reads the probability, selected choice, or normalized score required by each workflow.

Setup and compatibility

At review time, the cookbook specified jev-1.12, Python, typesafe-sdk>=0.5.7, and a TypeSafe API key. Consult the current page before running it because SDK and model identifiers can change.

Limitations

The published speed and cost comparison is TypeSafe’s own measurement on a document-heavy workload. Sequential single calls make the latency contrast larger than concurrent calls would, while repeated document tokens still affect cost. Independent questions are essential: questions that depend on one another need explicit orchestration.

Sources

Primary source: TypeSafe parallel-questions cookbook.