Xiaohongshu's engine architecture team had a paper accepted at OSDI 2026: "The Clustering Strikes Back: Building Cost-Effective and High-Performance ANNS at Scale with HELMSMAN." The paper confronts a growing problem — the hardware cost of vector retrieval is expanding exponentially.
The problem
Xiaohongshu's search, recommendation, and advertising business maintains single indexes with tens of billions of high-dimensional vectors in production, serving millions of queries per second under low-latency requirements. The pure in-memory HNSW route has reached PB-scale DRAM occupancy, costing several million US dollars per year in hardware — unsustainable as demand grows. DRAM-SSD hybrid approaches like DiskANN and SPANN cannot replace in-memory deployments in high-SLA, high-QPS scenarios: graph-based ANNS has inherent serial I/O dependencies (each neighbor read waits on the previous SSD return), so even high SSD bandwidth goes underutilized.
HELMSMAN's key insight
Graph indexes suit memory, not SSDs; cluster-based ANNS is what can actually exploit high-bandwidth SSDs. Locating a set of centroids first, then batch-reading corresponding clusters, produces independent accesses that naturally form batch I/O.
Three layers of engineering
1. Custom ANNS storage stack. Traditional Linux I/O path software overhead can account for up to 58% of end-to-end latency. HELMSMAN uses SPDK user-space storage, bypassing the filesystem, block layer, and kernel drivers, managing NVMe queues directly and storing fixed-size cluster lists on raw logical blocks — an entire cluster list is fetched in a single submission.
2. Layered Learned Search Pruning (LLSP). A fixed nprobe wastes I/O on easy queries and misses recall on hard ones. A router model predicts a search tier, then a GBDT model computes the actual nprobe needed from centroid distances and other features. All reads are still submitted as one batch, preserving the batch-I/O advantage.
3. GPU-accelerated distributed build pipeline. A three-stage heterogeneous pipeline: GPUs run coarse-grained k-means, an elastic CPU pool handles partitioning and balancing, and multi-core CPUs do merging and LLSP model training. Virtual Kubelet borrows idle CPU from online clusters at low-traffic periods.
Measured results
- 40 all-flash servers handle the online load previously requiring 35,000 CPU cores + 350 TB DRAM — over 90% hardware cost savings
- 2-16× throughput vs. DiskANN, Starling, PipeANN, and SPANN
- Up to ~85% of pure in-memory HNSW throughput, meeting 5-10 ms average latency and strict tail latency
- SSD bandwidth utilization: under 20% for graph-based systems, ~55% for SPANN on Gen4, ~85% for HELMSMAN on Gen4, ~70% on Gen5
- Index building: 0.1B vectors in under 1 hour, 10B vectors in 4-7 hours — roughly a 10× speedup
What this changes
The previous vector-retrieval infrastructure narrative was "add memory, add QPS, add SLA." HELMSMAN replaces it with "add storage, add bandwidth, add batch I/O." Any embedding retrieval service in 2026 H2 — search, recommendation, advertising, content safety, RAG — will likely re-evaluate its hardware stack; the in-memory route may cap out around tens of billions of vectors before the cost curve becomes unmanageable.
This is also a notable entry among recent Chinese infrastructure contributions, and unlike prior ones (models, benchmarks), HELMSMAN is an engineering systems paper accepted at OSDI — meaning globally peer-reviewed recognition.
Limitations
1. Test workloads are Xiaohongshu's search/recommendation/ads business; cross-business reproduction requires recalibration. 2. The 85% SSD bandwidth utilization was achieved on Gen4; on Gen5 it drops to ~70% — Gen5 drives have lower per-device latency but higher concurrent IOPS, and the team is still tuning. 3. Paper and code are fully open-sourced (GitHub: Red-EAD/helmsman), but production deployment requires operational capabilities quite different from the in-memory route. 4. HELMSMAN is engineering evidence for "cluster-based beats graph-based," not a law of physics — graph ANNS still has advantages on some low-dimensional, small datasets.
Links: OSDI paper on arXiv | GitHub repository | Tencent News coverage