Jev in Practice: Four Copy-Paste Patterns and One Path Tested and Rejected

> Four primary sources read in full: TypeSafe design doc, hermes-jev-skills, fast-jev-compaction, LangChain integration | Vendor benchmarks noted as such; hands-on data labeled with source | Continues the 9/17 customs report (topic 178634914)

The previous post covered what Jev is: a TypeSafe "System One" model that only answers yes/no-style judgments, never writes text. This one covers how to use it. The material is four primary documents, all read in full, including two community evaluations. There are numbers, pitfalls, and one path that was rejected by testing.

First, the price tag. Official figures: $0.042 per million input tokens (traditional LLM input prices run $0.20–$10), free output, end-to-end latency of 70–500ms. LangChain relays the vendor's claim: 200x faster and 400x cheaper than comparable LLMs on classification tasks. There are only three question types: Choice (pick one from options), Score (ordinal rating), and noul (yes/no probability). The most counterintuitive property: all questions in a single request are evaluated in parallel — adding ten questions adds almost no latency, only token cost. The claim "writing code itself accounts for only 4–10% of tokens" comes from video coverage; the hard citation is Microsoft's fastcontext team analysis of GPT-5.4 trajectories: reading and searching account for 56.2% of tool-call rounds and 46.5% of the main agent's total tokens.

1. The Design Doc's Core: The Tyranny of KV Cache

The most valuable part of TypeSafe's design document is not the product pitch but the opening question: if LLMs had no KV cache, how would you design a coding agent?

The answer is a list the document calls the tyranny of KV cache — six items. Here are the harshest three.

Routing doesn't work. The doc runs the numbers: opus at $5/$25, sonnet at $3/$15. A session that goes opus → sonnet → back to opus costs more than staying on opus throughout, because the context must be re-processed by the large model and the KV cache is invalidated. At typical load ratios, pure opus costs only two-thirds as much. The hidden cost of routing is not the model price difference — it's cache rebuilding. The longer the context, the more expensive each switch.

Tool calls are a strange tradeoff. Tools must be pre-loaded into the system message whether or not they're used this turn; parameters eat context; models are unreliable on high-cardinality, off-policy tool calls. This is also why skills beat raw MCP.

Compaction exists because everyone assumes all future turns share a single state. The doc asks: why assume that? Compression is hard, and likely inferior to query-aware compression — if you know what you're looking for, compressing is far easier.

TypeSafe's own answer is called Meta-attention: demolish the assumption that "context is static"; re-score every piece of context per query with four tiers — hide, short summary, long summary, full text. In the doc this is still a vision, not a shipped feature. But the direction matches the two community projects below exactly.

2. Four Patterns You Can Copy Directly

Pattern 1: Model routing. hermes-jev-skills (community project, unofficial) wires Jev into Hermes/Claude Code/Codex to decide which model to use each turn — measured at roughly 0.4 seconds. LangChain released official ModelRouterMiddleware, which picks a model from the latest user message for the whole run. The key isn't selection, it's rollout: run shadow mode first — decide and log only, never switch; review a day of logs before going live. hermes' guardrails are worth copying too: risky keywords (production, delete, migration, payment, legal) never route to the cheapest tier; large contexts are never downgraded mid-session.

Pattern 2: Context filtering. fast-jev-compaction's approach is blunt: don't use Jev to write summaries — have it decide what to delete. Each tool call and result gets two scores: should this call be kept (do we know it happened + do the parameters still matter), and should this result be kept verbatim (is the content still needed + can it be re-fetched). Below threshold, delete. What's kept stays untouched, character for character. User and assistant text are never modified. Summarization is lossy — file paths, exact errors, constraints, and commands vanish in a summary. Deletion is lossless — what remains is guaranteed verbatim. One engineering discipline from practice: if the compression ratio is under one-quarter, don't bother running it.

Pattern 3: Skill selection. Picking which of 377 installed skills this turn needs takes about 2.8 seconds. It pairs well with the design doc's conditional AGENTS.md idea: don't keep all instructions resident in context — load dynamically by condition — attach the style guide for frontend tasks, attach the footguns file for subdirectories. The doc has a line worth quoting: every subdirectory should have a gotchas file.

Pattern 4: Tool-call guardrails. LangChain's AutoModeMiddleware uses Jev to intercept risky tool calls before execution. The blog puts it plainly: such safety-classifier steps used to be locked inside closed-source harnesses (Claude Code, Codex, Cursor internals); now that a cheap, fast classifier exists, every agent can use the same pattern. hermes' version is stricter: Jev can only return an id from a predefined action table — a closed-set guardrail that caps the blast radius of a wrong choice.

3. Limits, and One Rejected Path

Three hard limits first. First, probability is not proof: fast-jev-compaction's Limitations section says verbatim, "a probability is not a proof that a result is safe to delete" — the saving grace is that agents can always re-run tools, which is deletion-style compaction's safety net. Second, accuracy is traded away: DataCamp's phrasing is "trades peak accuracy for cost and latency savings" — peak accuracy is not its selling point. Third, the privacy default still sends data off-site: routing by default sends redacted user turns to Jev (2,500-character cap, head and tail read); sensitive profiles require private_profiles, which only sends coarse features like length, presence of code, and risk keywords.

The rejected path: conversation handoff summaries. In hermes' hands-on testing, having Jev do keep/summarize/drop conversation compaction performed worse on recall than pure transcription: single query, raw transcript 58.7% vs Jev-summarized 37.5%; with one search, 75.0% vs 68.3%. It lost at both tiers. What they shipped instead is the full-text option: deliver the complete 1,200-word conversation plus a backtracking path. A community project writing a negative result into its README deserves respect.

This negative result fits the previous post's conclusion perfectly: closed-set judgments can be outsourced; open-set judgments cannot. Conversation summarization is an open-set problem; routing and triage are closed-set problems. Jev's适用 domain is exactly the ring outside the former.

There's also an engineering disagreement worth studying: hermes is fail-open across the board (if Jev dies, keep the status quo — worst case, lose 2.5 seconds), while fast-jev-compaction is fail-loud (throw an exception and let the caller decide whether to fall back to Claude Code's built-in summarization). Same model, two failure philosophies, depending on whether the cost of error is "one turn gets slower" or "context silently loses things." Where failure would be silent, prefer loud.

4. Tying It Back

Delete-only compaction is the third sample of "don't re-compress what was compressed": Stanford's Prefix Sliding (drop-tier), CliffCompaction's residual propagation, and fast-jev-compaction's scored deletion — three teams, within two weeks, with different implementations, saying the same thing: rewriting intermediate artifacts is always lossy; delete, don't rewrite. The KV-cache tyranny arithmetic is a new entry in interface-tax pricing: context is prefix-structured, switching models means switching interfaces, and the entire prefix tax is repaid. Routing doesn't save intelligence — it saves tax.

One falsifiable prediction: within 12 months, mainstream agent frameworks will ship closed-set-judgment routing as a default middleware layer (LangChain has already started), with judgments and generation priced separately at the harness level. Looked back on from then, the $0.042 price tag won't define a cheap model — it will define a new category.

---

*Sources: TypeSafe design doc (Google Docs), github.com/kerpopule/hermes-jev-skills, langchain.com/blog/building-a-harness-with-jev (9/17), github.com/tamaratran/fast-jev-compaction, typesafe.ai official blog. hermes-jev-skills and fast-jev-compaction are independent community projects. Benchmarks are vendor figures; hands-on data comes from both repos' evals docs.*