Xiaohongshu's engine architecture team had a paper accepted at OSDI 2026: *The Clustering Strikes Back: Building Cost-Effective and High-Performance ANNS at Scale with HELMSMAN*. The paper tackles a direct problem — the hardware cost of vector retrieval is growing exponentially.
Xiaohongshu's search, recommendation, and advertising businesses maintain indexes with tens of billions of high-dimensional vectors in production, with low-latency recall under millions of queries per second. The pure in-memory HNSW route has reached PB-scale DRAM occupancy — hardware costs of several million USD per year — and cannot keep growing. DRAM-SSD solutions like DiskANN and SPANN cannot replace in-memory deployments in search/ads scenarios with strict SLAs and high QPS, because graph-based ANNS has inherent serial I/O dependencies: the next neighbor read must wait for the previous SSD read to return, so even high total SSD bandwidth cannot be saturated.
HELMSMAN's judgment is blunt: graph-based indexes suit memory but not SSDs; clustering-based ANNS is what can actually exploit high-bandwidth SSDs. It first locates a set of centroids, then batch-reads the corresponding clusters — accesses are independent, naturally forming batched I/O.
Three engineering layers support this:
Layer 1: A custom ANNS storage stack. Software overhead of the traditional Linux I/O path can account for up to 58% of end-to-end latency. HELMSMAN uses SPDK for user-space storage, bypassing the filesystem, block layer, and kernel drivers to directly manage NVMe queues, storing fixed-size cluster lists directly on raw logical blocks. This compresses the I/O path so a single submission reads an entire cluster list in one shot.
Layer 2: Layered Learned Search Pruning (LLSP). A fixed nprobe means simple queries waste I/O while hard queries sacrifice recall. HELMSMAN uses a router model to predict a search level, then a GBDT model computes the actual nprobe needed based on centroid distances and other features — all reads are still submitted as one batch, preserving the batched I/O advantage.
Layer 3: GPU-accelerated distributed build pipeline. Recommendation/ads embeddings update frequently, so slow index building can't keep up with model iteration. A three-stage heterogeneous pipeline: GPU does coarse-grained k-means, an elastic CPU pool does splitting and balancing, and multi-core CPUs do merging and LLSP model training. Virtual Kubelet also allows borrowing idle CPU capacity from online clusters during off-peak hours.
Measured results:
- 40 all-flash servers handle the online load previously requiring 35,000 CPU cores + 350 TB DRAM — over 90% hardware cost savings;
- 2-16× throughput improvement over DRAM-SSD systems such as DiskANN, Starling, PipeANN, and SPANN;
- Up to about 85% of the throughput of pure in-memory HNSW deployments, while meeting 5-10 ms average latency and strict tail latency;
- SSD bandwidth utilization: below 20% for graph-based systems, ~55% for SPANN on Gen4, ~85% for HELMSMAN on Gen4, ~70% on Gen5;
- Index building: 0.1B vectors in under 1 hour, 10B vectors in 4-7 hours — roughly a 10× speedup.
The previous infrastructure narrative was "add memory, add QPS, add SLA." HELMSMAN replaces it with "add storage, add bandwidth, add batched I/O." This means any service doing embedding retrieval in H2 2026 (search, recommendation, ads, content safety, RAG) will re-evaluate its hardware stack — the ceiling of the pure in-memory route may be around tens of billions of vectors, beyond which the cost curve becomes untenable.
Counting this, contributions from Chinese infrastructure teams keep accumulating: 07-11 LingBot-VA, 07-14 EdgeBench, 07-15 ABot-WorldStudio + Xiaomi-Robotics-U0, 07-20 MiniCPM-Robot + Matrix-Game 3.5, and now HELMSMAN is the sixth. But this one is different — the previous entries were either foundation models or benchmarks, while HELMSMAN is an engineering systems paper accepted at OSDI, a top systems venue. Its significance lies in "global peer-reviewed recognition."
Limitations to note:
1. The test workload is Xiaohongshu's search/recommendation/ads business; reproducing results across other companies requires recalibration; 2. The 85% SSD bandwidth utilization was achieved on Gen4 drives; on Gen5 it drops to 70% — Gen5 drives have lower per-request latency but higher concurrent IOPS, and the team is still tuning; 3. The paper and code are fully open source (GitHub: Red-EAD/helmsman), but the operational skills needed for production deployment are entirely different from the in-memory route; 4. HELMSMAN is an engineering demonstration of the "clustering beats graphs" judgment, not a law of physics — graph-based ANNS still has advantages on some low-dimensional, small datasets.
One final detail: Xiaohongshu's 07-23 "dots IMO perfect score" release (already covered) landed the same week as HELMSMAN. Xiaohongshu is pushing hard on both the LLM and systems tracks — a notable entry in July's Chinese AI infrastructure log.
Source 1: OSDI paper on arXiv | Source 2: GitHub repository | Source 3: Tencent News coverage