55nm ReRAM-on-Logic Stacked Chip Runs LLM Inference at up to 135 tokens/s (ISSCC 2026)

LLM accelerators typically rely on HBM for high-capacity, high-bandwidth memory. ReRAM (resistive random-access memory) is an alternative: it has higher density than SRAM, lower power than DRAM, and can be vertically stacked with logic layers.

A chip presented at ISSCC 2026 (arXiv:2605.09375) implements an LLM inference accelerator in 55nm CMOS using bump-bonded face-to-face ReRAM-on-Logic stacking, delivering 14.08–135.69 token/s. Core techniques include:

• Outlier-free low-bit quantization using local rotating units
• Block-clustering vector compression to reduce weight-loading overhead
• Adaptive parallel speculative decoding

Open questions raised by the author

• 55nm is a relatively old process node. When comparing against accelerators in the literature built on more advanced nodes, fairness requires accounting for process differences.
• The 14–135 token/s range is very wide — under what conditions does the chip run at 14 token/s versus 135 token/s?

References

1. Dong, P., et al. (2026). *A 14.08-to-135.69Token/s ReRAM-on-Logic Stacked LLM Accelerator*. arXiv:2605.09375 [cs.AR]. (ISSCC 2026) 2. Chen, A. (2024). *ReRAM-based Processing-in-Memory for AI*. Nature Electronics. 3. Levi, T., et al. (2023). *Speculative Decoding for LLM Inference Acceleration*.