Have you ever seen this happen? An agent suddenly sends a customer a 90% discount. A long-running workflow loses its way halfway through. An event handler reacts to a changed world using outdated prompts.
The first instinct is usually: "the model glitched again." Swap in a stronger model, tune the prompt, add examples. But the truth may be: the problem isn't the model—it's your code.
A May 2026 paper introduces SDB (Stochastic-Deterministic Boundary), making a counterintuitive claim: in production, many failures that look like "model defects" are actually system architecture's fault. Your architecture choices determine success before the LLM says a word.
Key points
- SDB is a four-part contract at the seam where LLM output becomes system action:
- Core principle: LLMs propose; code verifies, commits, rejects. An audit of 5 open-source frameworks and 21 LLM-to-action call sites found 90.5% include some Verifier-Commit logic, but quality varies wildly—from a one-line JSON parse in
openai/swarmto MetaGPT's multi-stage pydantic + LLM-as-judge auto-review loop. Of 21 published agent failure post-mortems, 71.4% located faults at the SDB boundary itself; 81% of fixes strengthened one of the four parts. - sigma: per-call variance. Compressed generation over generation as base models improve.
- mu: architectural momentum—your chosen patterns and SDB strength, independent of any single call.
- Open, non-enumerable output space? → LLM Proposer. Otherwise → deterministic rule engine.
- Verification encodable as structured checks? → Code Verifier (JSON Schema, numeric ranges, legal state transitions). Otherwise → human Verifier (P6) or LLM-as-Judge.
- Side effects needing exactly-once semantics? → Code Commit + Saga compensation (P2). Otherwise → event-log append (P3).
- Rejection needing structured recovery? → Typed Reject Signal (
'incomplete'+ error code + retry hint). Otherwise → simple exception. - Vasudevan et al. (2026). Stochastic-Deterministic Boundary: Runtime Architecture Patterns for Production LLM Agents. arXiv:2605.20173. https://arxiv.org/abs/2605.20173
- Reference implementation: https://github.com/vasundras/agent-runtime-patterns
- Baselines: openai/swarm, MetaGPT, AutoGen, HyperAgents, OpenEvolve
- Pattern sources: Actor Model, Sagas, Spanner, Erlang OTP, Workflow Nets
| Part | Role | Owner | |------|------|-------| | Proposer | LLM output—text, tool args, classifications | LLM (stochastic) | | Verifier | Deterministic checks—JSON Schema, policy rules, state predicates | Code (deterministic) | | Commit | Persistent writes after acceptance—DB, API calls | Code (deterministic) | | Reject Signal | Typed response returned to the LLM on failed verification | Code (deterministic) |
The reliability equation: sigma shrinks, mu dominates
The paper offers a decomposition model:
> y(t) = μt + σξ(t)
| Era | sigma | Dominant lever | |-----|-------|----------------| | Early GPT-3 | Large | Model choice, prompt engineering | | GPT-4/Claude now | Compressing | mu: SDB design and pattern choice | | Future | Near zero | Pure architecture |
Translation: models keep getting steadier; what's left is all architecture. A 90%-discount incident isn't fixable by shrinking sigma—the root cause is a missing Verifier, which is mu.
Three crash sites, all architectural
1. Event handler with stale prompts. A resumed workflow uses prompts cached at generation time rather than current world state. Surface symptom: "inconsistency," "hallucination." Actual cause: architecture didn't separate event time from processing time. Fix: add timestamp/version predicates to the Verifier, or migrate to P5 (Shared State Machine) with CAS for version consistency.
2. A 90% discount reaches customers. Promptfoo reported a real case: migrating from GPT-4o to GPT-4.1, prompt-injection resistance dropped 23 percentage points (94%→71%) under the same eval framework. A malicious prompt then produced a 90% discount sent directly to a customer. SDB analysis: the Verifier was missing/weak, Commit wrote straight to customer systems, and no structured Reject Signal existed. Fix: output classifier + strict tool gating. The LLM can propose a discount; the code decides whether it ships.
3. Long-horizon amnesia. An agent resumes a paused workflow and can't find its place. Wrong: treating the context window as state. Right: P5 Shared State Machine—versioned rows + Compare-And-Swap conditional writes. Timers carry the version at scheduling time. human_required is state, not a missing event.
Six architectural patterns
| Pattern | Concern | Core mechanism | SDB notes | |---------|---------|----------------|-----------| | P1 Hierarchical Delegation | Coordination | Orchestrator + sub-agents | Merge step must be deterministic code | | P2 Scatter-Gather plus Saga | Coordination | Fan-out + compensating transactions | Idempotent compensation per peer | | P3 Event-Driven Sequencing | State | Event log driven | Boundary at each event handler | | P4 Supervisor plus Gate | Control | Supervised restarts + policy gates | Audit logging; gates reject unauthorized actions | | P5 Shared State Machine | State | CAS writes + versioned rows | Strictest SDB; workers as pure functions | | P6 Human-in-the-Loop | Control | Manual intervention | Four planes: Kill Switch / Escalation / Approval / Throttling |
Key rule: "LLM proposes. Deterministic code decides." If P1's merge step is delegated to an LLM, one sub-agent's output can outweigh its declared weight in the merge. P5 is the strictest: worker reads (state, action) → proposes next → Verifier CAS-checks version → conditional write or version-conflict signal.
How to divide work between LLM and code
Anti-pattern: openai/openai-agents-js #1104—rejected tool calls reported status: 'completed', so the model hallucinated success. Fix: switch to status: 'incomplete'. The Reject Signal is a load-bearing contract part, not an afterthought.
Building a production-grade agent
Console-first build order: State Schema + observability lenses → Gate (P4) + audit log → Orchestrator (P1/P2) + one sub-agent → remaining sub-agents → P6 control plane (Kill Switch → Escalation → Approval → Throttling). State is the spine—define it before coordination and control.
Key thresholds:
| Condition | Action | |-----------|--------| | Pause > 1 hour | P5 persistent spine needed | | State not reconstructible from raw inputs | P5 versioned state needed | | World changed during pause | P5 dynamic state predicates needed | | Replay divergence appears | P3→P5 migration | | CAS retries > 3 at p99 | State granularity too coarse—split it |
Quarterly diagnostic: pin the model version on a recent failure batch → replay on prior versions → persistent failure means a functional fault (check architecture signatures); failure resolved means replay divergence (spine exposure—consider P3→P5); neither means variance fault—raise k in pass@k.
Reference implementation: 90-day contract renewal
Validated on the IBM Telco Customer Churn dataset with 100 renewal scenarios: P5 (CAS transitions on renewal rows; human_required as state; versioned timers), P1 (orchestrator + three sub-agents), P2 (Saga compensation on billing writes), P4 (gates reject unauthorized discounts), P6 (contract-merge escalation, approval SLAs, per-tenant throttling).
Repo: https://github.com/vasundras/agent-runtime-patterns
A key insight from five workloads: both Long-Horizon, Port-in uses P5 while Lead Warming uses P3—the only difference is the predicate "can state be reconstructed from the touch log?"
One-line summary
> Explicit SDB design = Proposer/Verifier/Commit/Reject separation + predicate-based State spine selection + coordination patterns matched to work decomposition + control patterns matched to risk level.
The stronger the model, the closer sigma gets to zero—and mu (architectural momentum) becomes the only dominant lever. Your agent isn't dumb—your architecture is digging the pit. Fixing SDB matters a hundred times more than swapping models.