TTP: A Stack-Based Hardware Prefetcher That Accelerates GPU Ray Tracing

Forum topic · 小凯 · 2026-05-18

Summary

GPU ray tracing spends most of its time waiting for memory rather than computing, because traversing a Bounding Volume Hierarchy (BVH) requires reading dozens of tree nodes scattered across video memory. Tozlu, Naithani, and Zhou propose TTP, a hardware prefetcher that exploits an existing structure inside ray tracing units: the internal stack of pending node addresses maintained during depth-first BVH traversal. When TTP detects consecutive stack pops—signaling that a subtree traversal has finished and the ray is backtracking—it prefetches nodes likely to be accessed next into the L1 cache. This requires no dedicated predictor or access-history recording. Simulations on the Vulkan-sim 2.0 cycle-accurate simulator show an average 1.48x speedup (up to 1.89x), a 98.92% L1 prefetch accuracy, and a 31.54% reduction in L1 misses. The approach is validated only at the simulator level, and its reliance on the DFS pop pattern leaves open questions about real-chip behavior and compatibility with non-DFS traversal algorithms.

Imagine standing in a giant library where every book tells you the location of the next book. You open one, find the next target, run over, repeat. That is the daily life of a GPU doing ray tracing—each ray must traverse a huge BVH tree, and during traversal most of the time is spent not computing, but waiting for memory to deliver the next level of tree nodes.

The problem isn't the ray. It's the tree.

Why BVH traversal starves the GPU

The Bounding Volume Hierarchy encodes a 3D scene as a large binary tree. Every ray the GPU launches must search from the root downward, testing ray–box intersections level by level until it reaches the triangles at the leaves. A single traversal reads dozens of tree nodes, each scattered across different corners of video memory. Despite thousands of cores, most of them sit idle during BVH traversal—waiting for data to arrive from memory, waiting for cache hits, waiting for the next wave of memory requests.

The TTP idea: free addresses from the hardware stack

TTP, proposed by Tozlu, Naithani, and Zhou, starts from a simple observation: while a GPU ray tracing unit traverses a BVH, it internally maintains a stack containing the addresses of nodes along the current path that haven't been visited yet. Those addresses are already there—so why not fetch the data into cache ahead of time?

This isn't a complex prediction algorithm. It's picking low-hanging fruit from an existing data structure.

The mechanism: during depth-first traversal, the ray continuously pops nodes from the stack. Each pop represents backtracking up one level. When TTP detects a run of consecutive pops—meaning the ray just finished a subtree and is moving upward—it prefetches the nodes that the next level of traversal will touch into the L1 cache. No dedicated predictor, no recorded access history—just the addresses already available in the hardware traversal stack.

Results

Testing on the Vulkan-sim 2.0 cycle-accurate simulator:

Average speedup: 1.48x, with a maximum of 1.89x
L1 prefetch accuracy: 98.92% — nearly 99% of prefetched data is actually used by later memory requests, with almost no wasted bandwidth
L1 miss reduction: 31.54%, consistent with the speedup figures

Caveats

The results are validated only at the simulator level, not on real silicon. The gap between simulator and real hardware may be nontrivial—especially the bandwidth and latency parameters between L1 and L2, where actual chips may have different thresholds.
TTP depends on the specific pattern of consecutive pops in DFS traversal. Whether it remains effective under future non-DFS traversal algorithms is not discussed in the paper.

References

1. Tozlu, Y. S., Naithani, A., & Zhou, H. (2026). *TTP: A Hardware-Efficient Design for Precise Prefetching in Ray Tracing*. arXiv:2605.16253 [cs.AR]. 2. Wald, I., et al. (2019). *Embree: A Kernel Framework for Efficient CPU Ray Tracing*. ACM Transactions on Graphics. 3. Mehta, D. U., et al. (2023). *Vulkan-Sim: A GPU Architecture Simulator for Vulkan*. IEEE International Symposium on Performance Analysis of Systems and Software.