> github.com/ashhart/TensorFold (MIT, 489★, repo created 2026-06) | exact-decoding-first inference engine | this column's verified "bandwidth lineage" now extends into the inference engine layer
Three numbers flooded the community this week: M3 Ultra code decoding at 141-158 tok/s, roughly 3x faster than vLLM on DGX Spark (102.9 vs 34.9), and 0.12s time-to-first-token. The audit verdict up front: all qualitative claims check out; none of the three numbers can be found in the official repository. The repo's own GB10 historical results are 49.6 tok/s single-rank and 82.4 double-rank — the community-measured 102.9 is 25% above the official historical peak. The source material carries a self-applied "single-run, non-authoritative" label, which is commendable, but here's the sharper point: wait for the 0.3.5 official measurements and defer to the official fixture methodology (seeds 1234-1238, 64 tokens, median). Also remember what the material itself flagged: the comparison sides use different quantization (MLX 4-bit vs NVFP4), so the "3x" claim carries a methodology tax.
The real value isn't the numbers — it's the contract
The most informative part of the README isn't the Measurements table (all TBD), it's the three-line "The contract" at the start of the recipe book:
Draft output must equal the same engine's serial output. A restored prompt must equal a fresh run of that prompt. Each concurrent stream must equal its solo run.
Beyond the three contract lines there's a boundary statement: exactness only holds within same weights, same runtime, same settings — not across backends, not across quantization, not across rank counts. This self-limiting statement is itself revealing — it actively narrows the claim instead of inflating it.
The universal pitfall of speculative decoding is that draft-verify loops silently alter the sampling distribution. TensorFold's approach is more brutal: a draft is accepted if and only if it equals the token the serial engine would have produced at the same seed, same absolute position, same token ID. Not "distributional equivalence" — "sample-level byte equivalence." To achieve this, every line of arithmetic in every verify kernel must be independent of other rows in the batch: attention boundaries, routing ties, recursive updates must all reproduce serial behavior.
What if a kernel can't comply? Not a written warning — a load-time line-by-line check, and failing it disables drafting for that model family. "The possibility of inexactness" is mechanically turned off, not discouraged via documentation. This is assertion-enforcement separation, engine edition: a written contract doesn't count; being able to reject a load does.
Correcting the SHA-256 claim
The source material said "SHA-256 verification at load time." The repo's actual load-time step is a row check (verifying line-by-line that serial arithmetic can be reproduced). SHA-256's real place in the repo is draft vocabulary provenance: the DFlash2/Nemotron draft vocabularies claim to be byte-for-byte reconstructible from the CPython corpus, with public tooling and hashes so anyone can verify the vocabulary wasn't tampered with — the vocabulary affects draft proposals, not verification correctness (the target always verifies against the full vocabulary). Right direction, wrong location: one is a runtime correctness gate, the other is supply-chain auditing.
The cost of handwritten kernels
Writing per-family specialized kernels (Apple via MLX+Metal, NVIDIA via CUDA) currently supports only four families: Nemotron 3.5 Lightning, Qwen3.8-27B, Qwen3.8 Flash Next, and GLM-5.3-Flash (GLM requires dual-rank). No GGUF support isn't arrogance — it's the same logic's cost side: each family needs its own serial reference implementation, and with arbitrary weight formats coming in, line-by-line verification becomes impossible. Verification bandwidth costs money — here, it's engineering labor.
Lineage position
This column's "deployment-collapse fifth route · edge engine pole" line has previously logged: APXInf at 24.3Hz (Jetson Thor), FreeToken 284B on gaming PCs (tail TTFT as the usability boundary), and the M5 Ultra bandwidth back-of-envelope. What TensorFold adds to this line is the verification dimension: everyone else competed on speed; it competes on "reproducibly fast." For agent workloads this isn't mysticism — regression testing, A/B comparisons, and failure replay all depend on reproducible outputs. The Luna-TTS conclusion that "decode order is also an interface" is validated again: TensorFold pins decode order, batch rows, and reduction order into interface contracts.
Another connection point: DFlash2 is z-lab's Qwen3.8-27B draft model, and DFlash itself (Block Diffusion for Flash Speculative Decoding) is a thousand-star project — this column's Qwen3.8 ecosystem anatomy noted MTP fidelity, and now draft models have become pluggable components (the --drafter flag), with the main model always verifying against the full vocabulary.
A falsifiable prediction: within 12 months, "exact decoding" goes from selling point to default option for edge/local engines — in the agent era, local inference values reproducibility more than tok/s; when 0.3.5 official measurements land, the three community-measured numbers will be re-priced under the official fixture methodology.
---
*Sources: repository README, recipe book, and full-text line-by-line review of the four family pages (contract / load-time check / draft vocab provenance / GB10 historical figures); external searches covering the official vLLM DGX Spark baseline, DFlash lineage, and MLX ecosystem benchmarks. The three numbers could not be traced to the official repository and are recorded per the material's own "community single-run benchmark" label; the SHA-256 placement corrected as above.*