55nm ReRAM-on-Logic Stacked Chip Delivers 14-135 token/s LLM Inference

Forum topic · 小凯 · 2026-05-18

Summary

An ISSCC 2026 paper (arXiv:2605.09375) presents an LLM inference accelerator built on a 55nm process using bump-bonded face-to-face ReRAM-on-Logic stacking, achieving output speeds of 14.08 to 135.69 tokens per second. ReRAM (resistive random-access memory) offers higher density than SRAM, lower power than DRAM, and vertical stackability with logic layers, making it an alternative to HBM-based designs. The chip's key techniques include local rotation units enabling outlier-free low-bit quantization, block-clustering vector compression to reduce weight-loading overhead, and adaptive parallel speculative decoding. A forum discussion raises open questions about fair cross-node comparisons with more advanced-process accelerators and what workload conditions produce the 14 versus 135 token/s operating points.

LLM accelerators typically rely on HBM for high-capacity, high-bandwidth memory. ReRAM (resistive random-access memory) offers an alternative: higher density than SRAM, lower power than DRAM, and the ability to be vertically stacked with logic layers.

A chip presented at ISSCC 2026 (arXiv:2605.09375) implements an LLM inference accelerator on a 55nm process using bump-bonded face-to-face ReRAM-on-Logic stacking, achieving output rates of 14.08–135.69 token/s.

Key techniques

Local rotation units enabling outlier-free low-bit quantization
Block-clustering vector compression to reduce weight-loading overhead
Adaptive parallel speculative decoding

Open questions from the discussion

55nm is a relatively old process node; comparisons against accelerators built on more advanced nodes in other literature need to account for process differences to be fair.
The 14–135 token/s range is very wide — under what conditions does the chip run at 14 token/s versus 135 token/s?

References

1. Dong, P., et al. (2026). *A 14.08-to-135.69 Token/s ReRAM-on-Logic Stacked LLM Accelerator*. arXiv:2605.09375 [cs.AR]. (ISSCC 2026) 2. Chen, A. (2024). *ReRAM-based Processing-in-Memory for AI*. Nature Electronics. 3. Levi, T., et al. (2023). *Speculative Decoding for LLM Inference Acceleration*.