langgraph-decision-models

langchain-typesafe is alpha (0.0.1a3) and TypeSafeClassifier is marked @beta — pin it and expect churn.

uv add langchain-typesafe

Three ways to reach a model. The classifier POSTs to {base_url}/v1/systemone with Authorization: Bearer {api_key}, so switching providers is constructor arguments only:

import os
from langchain_typesafe import TypeSafeClassifier

# 1. TypeSafe directly (Jev). Reads TYPESAFE_API_KEY when api_key is omitted.
classifier = TypeSafeClassifier(model="jev-latest")

# 2. SemIf, hosted on the LangSmith Gateway. Note: LangSmith key, not a TypeSafe key.
classifier = TypeSafeClassifier(
    model="semif-qwen3.5-4b",
    api_key=os.environ["LANGSMITH_API_KEY"],
    base_url="https://gateway.smith.langchain.com",
)

# 3. Jev through the Gateway (BYOK). The `typesafe/` prefix routes to a
# TYPESAFE_API_KEY stored in LangSmith workspace secrets. Without that secret
# every `typesafe/*` id returns 424 Failed Dependency -- before the model name is
# even validated, so a 424 does not confirm the id is real.
classifier = TypeSafeClassifier(
    model="typesafe/jev-1.13.0",
    api_key=os.environ["LANGSMITH_API_KEY"],
    base_url="https://gateway.smith.langchain.com",
)

Ask every question about a page/item in one request, put the typed response in state, and let a plain function route on it. The router is ordinary Python — testable without touching a network.

from typing import TypedDict
from langchain_typesafe import ClassifierResponse, Noul, Score, TypeSafeClassifier
from langgraph.graph import StateGraph, START, END

QUESTIONS = {
    "relevant": Score(
        instructions="How relevant is this ticket to a billing problem?",
        criteria=["Unrelated.", "Possibly related.", "Directly about billing."],
    ),
    "angry": Noul(instructions="Is the customer expressing anger?"),
}

class State(TypedDict):
    text: str
    answers: ClassifierResponse
    route: str

classifier = TypeSafeClassifier(model="jev-latest")

def classify(state: State) -> dict:
    # One request, every question. They are answered independently.
    return {"answers": classifier.invoke(
        {"state": state["text"], "questions": QUESTIONS}
    )}

def route(state: State) -> str:
    a = state["answers"]
    if a.nouls["angry"].noul > 0.7:
        return "escalate"
    if a.scores["relevant"].score < 0.5:
        return "close"
    return "handle"

Reading answers: response.nouls[id].noul, response.choices[id].choice, response.scores[id].score. Each view is keyed by your question id; response.answers holds them all.

These cause silent misrouting, not exceptions.

1. Score.score is an expected value, not a level. It is a probability-weighted average over the rubric and is routinely fractional. score == 0 almost never fires — a "not responsive" item lands at 0.07, not 0. Always compare against a band.

if a.scores["relevant"].score < 0.5:   # correct
if a.scores["relevant"].score == 0:    # WRONG -- nearly never true

2. Confidence measures distribution shape, not correctness. On a Score, confidence reports how concentrated the rubric distribution is. An item sitting cleanly between two levels scores low confidence even when the model is entirely clear about it. A blanket confidence < X -> escalate rule therefore escalates items the model already decided. Gate on confidence only inside the ambiguous middle:

if score < NOT_RELEVANT:                          # decisive -- trust it
    return "close"
if score < RELEVANT or confidence < MIN_CONF:     # ambiguous -- escalate
    return "human_review"
return "handle"

3. Thresholds do not transfer between models. Calibration is part of the model. The same policy over the same items routes differently on Jev vs SemIf vs an LLM adapter. Re-tune thresholds whenever you change models, and pin the model id.

A loose question produces confident wrong answers, and no threshold fixes it. Use criteria to say what each outcome means, including what should not count.

In a measured case, "Is this a confidential communication with a lawyer?" scored a routine finance memo at 0.798. Rewriting it to name the actual test — written by or to a lawyer, with an explicit carve-out for finance and accounting content — moved the same page to 0.005 while a genuinely privileged page held at 0.991.

Noul(
    instructions=(
        "Was this written by or to a lawyer, or does it convey a lawyer's legal "
        "advice? Answer no for ordinary business or accounting discussion, even "
        "when the subject is litigation-sensitive."
    ),
    criteria=NoulCriteria(
        true="A named attorney is author or recipient, or it relays legal advice.",
        false="Business or accounting content with no attorney involved.",
    ),
)

Before blaming the model, rewrite the question and re-measure.

To find where a decision model fits, look for these in the codebase — see references/conversion-playbook.md for the full walkthrough.

The last two matter most: cheap semantic judgments change what you can build, not just the bill. If evaluating every item became affordable, what would you stop sampling?

Measured on a 24-item batch, identical LangGraph graph and routing policy, only the classifier swapped:

Routing agreed on 4–5 of 6 items across engines; disagreements clustered on genuinely borderline items. Treat these as shape, not benchmarks — measure on your own workload.

If you compare against an LLM baseline, use method="json_schema" so the comparison is fair. LangChain's with_structured_output defaults to method="function_calling", which injects a tool schema into every request — 556/35 tokens versus 228/12 for the native output_config.format path on the same one-field probe.

Classification is per-item and independent, so fan out with Send and let each item route on its own.

from langgraph.types import Send

def fan_out(state):
    return [Send("classify_item", {"text": t}) for t in state["items"]]

builder.add_conditional_edges(START, fan_out, ["classify_item"])

Fan-out hides latency, so it flatters slow classifiers most: in the run above, Sonnet gained 8x from concurrency and Jev only 1.4x — yet Jev still finished first. Compare throughput, not the speedup multiple.

• langgraph-fundamentals — StateGraph, Send, Command, conditional edges
• langgraph-human-in-the-loop — interrupt() for the escalation branch above
• langchain-middleware — structured output when you need an LLM, not a classifier