CPU Prefetching Rethought: Predict by Instructions, Not Addresses (ICP, ISCA 2026)

Forum topic · 小凯 · 2026-05-18

Summary

A Chinese tech forum post discusses ICP, a new CPU prefetching approach presented at ISCA 2026 (arXiv:2605.15645). Conventional prefetchers rely on address patterns—predicting the next address from repeated strides—but irregular workloads such as linked-list traversal, graph neighbor access, and sparse matrices produce addresses that never repeat. ICP instead exploits instruction correlation: although the memory addresses vary, the instructions that generate them recur. If instruction A produces an address that instruction B then uses as an offset, the prefetcher can predict B's next access whenever A appears again. The author reports that ICP outperforms the state-of-the-art prefetcher by 14% on SPEC CPU and GAP benchmarks while using only 2.1KB of storage—three orders of magnitude less than traditional methods. The post also raises an open question: since ICP depends on stable data dependencies between instructions, its effectiveness on highly dynamic code (JIT-compiled or self-modifying code) remains unclear. References include the ICP paper and Triangle (ISCA 2022) on memory-access-driven prefetching.

Reading data from memory is slow—about two orders of magnitude slower than the CPU's own compute speed. Prefetching is the technique of having the CPU guess which memory addresses will be accessed next, and pull the data into the cache ahead of time.

Traditional prefetchers rely on address patterns: if reading A is followed by reading A+64, the next access is predicted as A+64. But many data access patterns are irregular—linked-list traversal, graph neighbor access, sparse matrices—so the addresses simply never repeat.

ICP (arXiv:2605.15645, ISCA 2026) takes a different approach: instead of guessing addresses, it observes which instructions produced them. Even though memory addresses do not repeat, the instructions that generate those addresses often do. If you see instruction A produce an address, and instruction B then use that address as an offset to access another address, then the next time instruction A appears, you can predict what instruction B will access.

Results

14% faster than the current best prefetcher on SPEC CPU and GAP benchmarks
Only 2.1KB of storage required—three orders of magnitude smaller than traditional methods

Open question

ICP depends on data dependencies between instructions. If the code is highly dynamic—JIT-compiled or self-modifying code—can these inter-instruction correlations still be captured reliably? The post leaves this unresolved.

---

References

1. Li, M., et al. (2026). *ICP: Exploiting Instruction Correlation for Prefetching Irregular Memory Accesses*. arXiv:2605.15645 [cs.AR]. (ISCA 2026) 2. Bera, R., et al. (2022). *Triangel: Memory Access Driven Prefetching*. ISCA 2022.