Helium: Rethinking LLM Serving for Agentic Workflows with Database Techniques (1.56x Speedup)

Forum topic · 小凯 · 2026-06-28

Summary

Helium, a system from the National University of Singapore, treats multi-agent LLM workflows as database query plans rather than sequences of independent inference calls. Traditional serving engines like vLLM optimize individual calls, missing cross-call redundancy such as repeated prompt prefixes, recomputed intermediate results, and speculative branches that are never used. Helium models agent workflows as DAGs with LLM calls as first-class operators and introduces three techniques: proactive caching that precomputes and pins static KV caches of shared prefixes in GPU memory; a Templated Radix Tree (TRT) that separates static prompt text from dynamic placeholders to expose sharing opportunities; and cache-aware scheduling with nested sequences to maximize prefix reuse and avoid cache thrashing. Evaluations on primitive workflows (Map-Reduce, multi-agent debate, reflection, iterative refinement, parallel chains) show speedups up to 1.56x over strong baselines including vLLM, LangGraph, AgentScope, Parrot, and KVFlow, and 1.34x on a complex financial trading workflow. Ablations show proactive caching contributes the largest share of gains. Helium is built on vLLM v0.16.0 and is limited to on-premise deployment with a shared base model.

Helium (NUS, SIGMOD 2026) applies database query-optimization principles to multi-agent LLM serving, achieving up to 1.56x speedup by eliminating hidden redundancy across LLM calls.

Key points

The problem: Serving engines like vLLM treat each LLM call as an independent request. In agent workflows, shared system prompts are re-prefilled dozens of times, intermediate results are recomputed across agents, and speculative branches are executed but never consumed.
Core insight: Agent workflows differ from SQL pipelines in three ways — LLM operators are stateful (KV cache, continuous batching), inter-operator sharing extends to KV cache reuse, and inter-workflow prefix sharing is possible across queries.
Gap in existing systems: LLM serving optimizes a single call; orchestration frameworks (LangGraph, AutoGen) treat the LLM as a black-box UDF; data systems (Spark, Dask) mismatch classic batching with continuous batching.

Helium's three techniques

1. Proactive Caching — At compile time, analyze the workflow DAG to identify static prompt prefixes; precompute their KV caches and pin them in GPU memory so they survive LRU eviction. A global prompt cache fully bypasses deterministic operators (zero temperature) with identical inputs. 2. Templated Radix Tree (TRT) — Prompts are split into static text and dynamic placeholders filled by upstream operators. Shared prefixes share tree paths, so agents like a debate's shared conversation history automatically reuse KV caches. 3. Cache-Aware Scheduling — The scheduler uses the TRT to order operators for maximal reuse: group operators sharing prefixes, critical-path-first ordering (Kelley-Walker heuristic), and nested sequences to avoid cache thrashing, minimizing makespan.

Results

| Workflow | Speedup | |---|---| | Map-Reduce | 1.56× | | Parallel Chains | 1.52× | | Multi-Agent Reflection | 1.45× | | Multi-Agent Debate | 1.34× | | Iterative Refinement | 1.28× | | Trading Workflow (end-to-end) | 1.34× vs. strongest baseline |

Baselines: vLLM, OpWise, LangGraph, AgentScope, Parrot, KVFlow. Ablations: removing proactive caching costs 28% performance, cache-aware scheduling 19%, prompt cache 11%, common-subexpression elimination 7%.

Practical implications for developers

For multi-agent workflows, optimization must be global across calls, not per-call.
Highest benefit: shared system prompts, deterministic intermediate steps, batched similar requests, speculative execution. Lowest benefit: fully distinct prompts, high temperature everywhere, independent linear requests.
Temperature matters: the prompt cache only works for deterministic operators; use zero temperature for extraction/context steps.
Helium complements vLLM (built on vLLM v0.16.0), sitting at the orchestration layer: workflow → Helium (optimizer + scheduler) → vLLM → GPUs.

Limitations and future work

Current limitations: on-premise only (needs GPU/KV control, no cloud APIs), assumes a single shared base model, requires offline profiling, and the prompt cache requires determinism. Future directions: heterogeneous models per agent, dynamic workload adaptation, cloud API integration via a proxy layer, and approximate/semantic caching.

Conclusion

Helium's message goes beyond 1.56x: agent workflows are computational graphs needing global optimization, not strings of independent API calls. Before optimizing individual inference, check the workflow for structural redundancy — the biggest wins often come from smarter execution plans, not faster engines.

References:

Wadlom, N., Shen, J., & Lu, Y. (2026). "Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective (Extended)." arXiv:2603.16104. SIGMOD 2026.
Code: https://github.com/mlsys-io/helium_demo