Xiaohongshu Open-Sources RedKnot: Head-Wise Sparse KV Cache for Long-Context Inference
> Published: 2026-06-29 · Category: Product release · Source: Xiaohongshu Tech (Redtech) + arXiv 2606.06256 > Paper: https://arxiv.org/abs/2606.06256
What Happened
On June 29, Xiaohongshu's engine architecture AI Infra team open-sourced RedKnot, a long-context inference engine rebuilt around "head-wise KV Cache sparsification." The paper is on arXiv and the code is publicly maintained.
The starting observation: today's KV Caches are dense tensors of shape [B, H, L, D] (batch, heads, sequence length, head dim). All heads for a set of tokens are stored, moved, and computed together. But Xiaohongshu's empirical finding is that KV Cache value is not uniformly distributed across tokens — it is strongly differentiated by head. Some heads genuinely need full context (e.g., global semantic alignment); others mostly attend to local windows (e.g., local syntax patterns). Current inference systems ignore this and process all heads as one bundle.
RedKnot does three things:
1. Head-classified sparsity. Offline classification of every (layer, head) pair: a minority of "global heads" (12-15%) see full context, while the majority "local heads" (85-88%) use only a sliding window. Validated on Mistral-7B, Qwen3-32B, Llama-3.3-70B, Qwen3.5-397B, and DeepSeek-V4-Flash, local-head share is stable at 83.4-96.8%.
2. Sparse FFN. Dense FFN runs only on the top-k tokens by attention score; other tokens take a residual identity path. This targets the 2-8K segments common in agent workloads, where FFN — not attention — is the real prefill bottleneck (57-62% of TTFT).
3. SegPagedAttention storage. Replaces the dense layout with per-(layer, head) segmented paged KV storage plus a fused variable-length attention kernel that physically stores only the tokens each head actually needs. Every head stays on the FlashAttention fast path with no attn_mask construction.
The three techniques act on orthogonal dimensions (heads, storage, channels), so gains multiply rather than compete.
Measured Results (8x H800, 3 models, 6 QA datasets, 8K-128K context)
- TTFT speedup: DeepSeek-V4-Flash goes from 3.51x at 16K to 5.16x at 128K — speedup grows with context, which token-level baselines cannot achieve.
- Llama-3.3-70B HotpotQA: exact match improves from 0.60 (dense baseline) to 0.80; first-token top-1/top-10 agreement with the dense path is 0.93/0.87.
- Per-GPU concurrency at 32K context: from 4 to 31.
- KV transfer bytes in PD-disaggregation: up to 6.3x savings.
- Prefill compute: reduced by 67-79.5%.
- Prefix caching — compute the KV for a shared conversation prefix once and reuse it (e.g., DeepSeek MTP, Qwen PD disaggregation).
- Position-independent KV reuse (PIC) — reuse KV for identical document fragments at different positions (CacheBlend, Epic, ProphetKV).
- Results are on H800; reproduction on consumer GPUs (RTX 4090/5090) and domestic accelerators is unverified.
- Offline head classification stability under distribution drift needs long-term observation; misclassification could hurt accuracy.
- Whether the "residual identity" sparse-FFN trick truly preserves model behavior needs more downstream validation.
- Section 4.4 of the paper admits ~5% accuracy degradation on some long-tail tasks for DeepSeek-V4-Flash — production tolerance depends on the use case.
Accuracy generally stays at or above 95% of dense F1, and can even exceed dense at long contexts — sparsification suppresses low-value token noise while preserving high-quality attention structure.
The implementation is based on SGLang and architecture-agnostic: the same runtime covers standard GQA, hybrid attention + MoE (Qwen3.5), and MLA compressed attention (DeepSeek-V4).
Why the "Per-Head" View Matters
Long-context inference optimization has mainly followed two paths over the past year:
But RedKnot's Figure 1 shows why PIC methods are "unreliable": they select token subsets to recompute at token granularity, yet different heads attend to different token subsets. Satisfying all heads requires taking the union of their important tokens — often a large fraction of the fragment — negating the reuse benefit. In other words, token-level is the wrong granularity; head-level is the workload's true sparsity structure.
Why Sparse FFN Matters for Agent Workloads
RAG, coding agents, and long-session systems typically run prompts of 2-8K tokens. In that range, attention is not the bottleneck — FFN accounts for 57-62% of TTFT. But naively cutting FFN for some tokens breaks the residual stream, so RedKnot uses "attention as a router for FFN sparsification": dense FFN only on top-k tokens, residual identity for the rest.
Sparsification as denoising is the more counterintuitive result. As the paper puts it: the longer the context, the more concentrated attention becomes — RedKnot can *exceed* dense accuracy at long contexts because it suppresses low-value token noise while keeping high-quality attention structure. RedKnot is not just "faster dense compute"; it may be "better compute."
Relation to DeepSeek DSpark (06-28)
Yesterday's DSpark accelerates decoding via speculative decoding (60-85% overall generation speedup); today's RedKnot sparsifies prefill (accelerating first-token latency). The two are orthogonal — generation vs. input, model architecture vs. system scheduling. China's open-source LLM inference infrastructure cadence is becoming systematic.
Why It Matters
1. Cost structure of long-context agents may be rewritten. For coding/deep-research/customer-service agents with half-hour sessions, TTFT and concurrency dominate cost. If RedKnot's speedups hold in production, unit costs could drop to 1/3-1/5. 2. "Per-head" may become the common language of long-context inference — future vLLM, SGLang, and TensorRT-LLM versions will likely adopt head classification. 3. Sparsification-as-denoising opens a new research direction — long-prompt accuracy may be about using context more precisely, not adding more of it. 4. First full reveal of Xiaohongshu's AI Infra team — previously known for training-side work (TensorFusion), RedKnot is their first disclosed inference engine release.