A New Idea for CPU Prefetching: Predict by Instructions, Not Addresses

Forum topic · 小凯 · 2026-05-18

Summary

Prefetching helps CPUs hide slow memory latency by predicting which addresses will be accessed next and loading them into cache ahead of time. Traditional prefetchers rely on repeated address patterns, which fails for irregular accesses like linked-list traversal, graph neighbor visits, and sparse matrix operations. A paper called ICP (arXiv:2605.15645, ISCA 2026) proposes a different approach: instead of predicting addresses, it observes which instructions produce them. Although addresses rarely repeat, the instructions generating them do. If instruction A produces an address and instruction B uses it as an offset for another access, seeing A again lets the prefetcher predict B's access. On SPEC CPU and GAP benchmarks, ICP outperforms the best existing prefetchers by 14% while using only 2.1KB of storage—three orders of magnitude smaller than conventional methods. The author also raises an open question: whether instruction correlations remain stable in highly dynamic code such as JIT-compiled or self-modifying code.

Reading data from memory is slow—about two orders of magnitude slower than the CPU's own compute speed. Prefetching is the technique of having the CPU predict memory addresses it will access soon, and move that data into cache ahead of time.

Traditional prefetchers rely on address patterns: if a read of A is followed by a read of A+64, the next time they guess A+64 again. But many data access patterns are irregular—linked-list traversal, graph neighbor visits, sparse matrices—where addresses simply never repeat.

ICP (arXiv:2605.15645, ISCA 2026) proposes a different idea: don't predict addresses; instead, observe which instructions produced them. Although memory addresses don't repeat, the instructions that generate them often do. If you see instruction A produce an address, and then instruction B use that address as an offset to access another address, then the next time instruction A appears, you can predict what instruction B will access.

Results: 14% faster than the current best prefetchers on the SPEC CPU and GAP benchmarks, using only 2.1KB of storage—three orders of magnitude smaller than traditional methods.

Open question raised by the author

ICP relies on data dependencies between instructions. If the code is highly dynamic (JIT compilation or self-modifying code), can these inter-instruction "correlations" still be captured reliably?

References

1. Li, M., et al. (2026). *ICP: Exploiting Instruction Correlation for Prefetching Irregular Memory Accesses*. arXiv:2605.15645 [cs.AR]. (ISCA 2026) 2. Bera, R., et al. (2022). *Triangel: Memory Access Driven Prefetching*. ISCA 2022.

A New Idea for CPU Prefetching: Predict by Instructions, Not Addresses

Summary

Open question raised by the author

References

Tags