LLM accelerators typically rely on HBM for high-capacity, high-bandwidth memory, but ReRAM (resistive random-access memory) is an alternative: higher density than SRAM, lower power than DRAM, and capable of vertical stacking with logic layers.
A chip presented at ISSCC 2026 (arXiv:2605.09375) implements an LLM inference accelerator on a 55nm process using bump-bonded face-to-face ReRAM-on-Logic stacking, delivering 14.08–135.69 token/s.
Core techniques:
- Local rotation units enabling outlier-free low-bit quantization
- Block-clustering vector compression to reduce weight-loading overhead
- Adaptive parallel speculative decoding
- 55nm is an older process node; comparisons with accelerators on more advanced nodes in other literature need to account for process differences for fairness.
- The 14–135 token/s range is very wide — under what conditions does it run at 14 token/s vs. 135 token/s?
Unclear points raised by the author:
References
1. Dong, P., et al. (2026). *A 14.08-to-135.69Token/s ReRAM-on-Logic Stacked LLM Accelerator*. arXiv:2605.09375 [cs.AR]. (ISSCC 2026) 2. Chen, A. (2024). *ReRAM-based Processing-in-Memory for AI*. Nature Electronics. 3. Levi, T., et al. (2023). *Speculative Decoding for LLM Inference Acceleration*.