Your AI Agent Isn't Dumb — Your Architecture Is

Key points

• SDB reframes agent reliability. The Stochastic-Deterministic Boundary is a four-part contract — Proposer (LLM), Verifier, Commit, and Reject Signal (deterministic code) — that pins down where stochastic output becomes system action. Real-world evidence is strong: 71.4% of 21 published agent postmortems localize to SDB weaknesses, and 81% of the fixes strengthen one of the four parts.
• Variance is collapsing, architecture is the leverage. Modeling reliability as y(t) = μt + σξ(t), the paper notes that σ (per-call LLM variance) shrinks each model generation, so μ — SDB strength, runtime pattern choice, state architecture — becomes the dominant lever. Upgrading the model compresses σ; it does nothing for missing verifiers, which is why a prompt-injection can ship a 90% discount.
• Three archetypal failures, three architectural fixes. A stale-prompt event handler is fixed with time/version predicates or a P5 Shared State Machine; an over-discount commit is fixed by output classifiers and tool gating; long-horizon "amnesia" is fixed by versioned CAS writes where human_required is a state, not a missing event.
• Six production patterns. P1 Hierarchical Delegation (merge must be in deterministic code), P2 Scatter-Gather plus Saga (idempotent compensations per peer), P3 Event-Driven Sequencing (boundary in every consumer), P4 Supervisor plus Gate (audit log + policy gating), P5 Shared State Machine (CAS, versioned rows, pure-function workers — the strictest SDB), and P6 Human-in-the-Loop across Kill Switch / Escalation / Approval / Throttling.
• Console-first build order. State Schema with observability → Gate with audit log → Orchestrator with one sub-agent → remaining sub-agents → P6 control plane. Concrete thresholds trigger P3→P5 migration: pauses over one hour, state unreplayable from raw input, world drift during pause, replay divergence, or CAS retries exceeding 3 at p99.
• Assign tasks via a decision tree. Open-ended un-enumerable output → LLM Proposer; structured checks → code Verifier; side effects needing exactly-once → code Commit plus Saga; else event log append. Reject Signal must be a payload-bearing contract (e.g., status: 'incomplete'), not a throwaway exception — an antipattern in openai/openai-agents-js #1104 let the model hallucinate success.
• Quarterly diagnostic procedure. Pin the failing batch's model version, replay on the prior version; persistent failure points to architecture, resolved failure to replay divergence (consider P3→P5), and mixed results to variance (increase k in pass@k).
• Validated on a 90-day renewal workload. The reference implementation combines P5 CAS transitions, P1 orchestration, P2 Saga compensation, P4 over-discount gating, and P6 controls, tested on IBM Telco Customer Churn with 100 renewals. Repo: https://github.com/vasundras/agent-runtime-patterns.

Bullet findings

• Core rule: LLM proposes; deterministic code decides.
• 90.5% of audited LLM-to-action call sites have some verifier-commit logic, ranging from one-line JSON parsing (openai/swarm) to multi-stage Pydantic + LLM-as-judge auto-review (MetaGPT).
• Real incident: a GPT-4o → GPT-4.1 upgrade cut prompt-injection resistance from 94% to 71% under the same eval harness — the SDB gap, not the model, shipped a 90% discount.
• P5 is the strictest SDB: workers read (state, action), propose a next state, Verifier CAS-checks the version, accepted writes are conditional, rejected returns a typed version-conflict.
• P1 merge must run in deterministic code — handing it to an LLM is what causes a sub-agent's output to outweigh a declared weight during orchestration.
• The four-plane P6 control plane is: Kill Switch, Escalation, Approval, Throttling — explicit, not ad-hoc.

Source

• Vasudevan et al. (2026). *Stochastic-Deterministic Boundary: Runtime Architecture Patterns for Production LLM Agents.* arXiv:2605.20173. https://arxiv.org/abs/2605.20173
• Reference implementation: https://github.com/vasundras/agent-runtime-patterns
• Compared baselines: openai/swarm, MetaGPT, AutoGen, HyperAgents, OpenEvolve
• Pattern origins: Actor Model, Sagas, Spanner, Erlang OTP, Workflow Nets