AdaSR Explained: Adaptive Streaming Reasoning with Hierarchical Relative Policy Optimization

Forum topic · 小凯 · 2026-06-15

Summary

This forum post explains AdaSR (Adaptive Streaming Reasoning with Hierarchical Relative Policy Optimization), a framework by Junlong Tong and colleagues that enables large language models to reason in real time as information streams in, rather than following the traditional read-then-think paradigm. AdaSR splits inference into two phases: lightweight streaming reasoning that tracks key information on the fly, and deep reasoning at checkpoints or stream end. The framework is trained with HRPO, a reinforcement learning method featuring hierarchical advantage decomposition, relative (group-based) rewards, and a triple reward combining format, accuracy, and adaptive-thinking signals. Models learn to allocate computation based on uncertainty, thinking deeply on complex inputs and skipping redundant reasoning on simple ones. Benchmark results on audio streaming, video streaming, and long-dialogue tracking show AdaSR outperforming SFT, standard RL, and StreamingLLM baselines on both streaming and final accuracy, while cutting latency by 20-30% and reducing token usage on easy inputs. The post discusses parallels to human dual-process cognition (Kahneman's System 1/System 2), relevance to embodied AI such as autonomous driving, current limitations (training cost, reward tuning, long-range memory), and future directions including multimodal and multi-agent streaming reasoning.

AdaSR: Adaptive Streaming Reasoning with Hierarchical Relative Policy Optimization — A Deep Dive

> This post is a detailed walkthrough of the paper *AdaSR: Adaptive Streaming Reasoning with Hierarchical Relative Policy Optimization* (Tong et al., arXiv:2606.14694).

Why Streaming Reasoning Matters

Current LLMs follow a read-then-think paradigm: consume the full input, reason in a static context, then answer. This fails in three ways:

Real-time constraints: voice assistants, autonomous driving, and live translation cannot wait for all input.
Memory explosion: long videos/audio/conversations can reach millions of tokens.
Poor adaptability: new information can overturn earlier conclusions; the model must revise continuously.

Streaming reasoning means reasoning, updating, and responding while information keeps arriving — like a simultaneous interpreter or a debater who starts formulating rebuttals mid-sentence.

The Three Core Challenges

1. When to think? Humans think more when information density is high and less when it is redundant. How can a model sense this? 2. How deeply to think? Trivial facts need only an intuitive note; complex claims need multi-step reasoning. 3. Accuracy vs. latency trade-off. The optimal balance point should shift dynamically with the situation.

AdaSR Architecture

Two-Phase Reasoning

Phase 1 — Streaming reasoning: lightweight, on-the-fly inference (analogous to taking shorthand notes while listening), capturing key points and flagging conflicts without full analysis.
Phase 2 — Deep reasoning: at checkpoints or stream end, systematic multi-step inference over everything accumulated, producing the final answer.

This mirrors human working memory: track in real time, offload key facts, then do deep processing for decisions.

HRPO: Hierarchical Relative Policy Optimization

Standard RL (PPO, GRPO) spreads a single sequence-level reward uniformly across tokens. HRPO introduces:

Advantage decomposition: streaming tokens receive advantages based on timely capture of key information; deep-reasoning tokens receive advantages based on final answer correctness.
Relative policy optimization: candidate responses are compared against the group average reward, reducing variance and stabilizing training.
Triple reward function:

Adaptive Computation Allocation

The model maintains an internal uncertainty estimate; high uncertainty triggers more thinking tokens, low uncertainty skips redundant reasoning — yielding efficiency on easy inputs and depth on hard ones.

Experimental Results

Evaluated on audio-stream understanding, video-stream reasoning, and long-dialogue tracking:

| Method | Streaming acc. | Final acc. | Latency | Compute efficiency | |--------|---------------|-----------|---------|-------------------| | SFT | 62.3% | 74.5% | High | Low | | Standard RL | 65.1% | 76.2% | Medium | Medium | | StreamingLLM | 68.4% | 78.1% | Medium | Medium | | AdaSR | 72.8% | 81.5% | Low | High |

Key findings:

Streaming accuracy up +4.4% over StreamingLLM.
Final answer quality beats even non-streaming baselines.
Average response latency reduced by 20–30% via adaptive allocation.
~40% fewer tokens than SFT on simple inputs.

A case study on a climate-debate recording shows AdaSR barely thinking on simple facts, launching deep reasoning on complex statistical claims, flagging contradictions for later verification, and synthesizing a structured conclusion at stream end.

Broader Significance

Batch-to-real-time paradigm shift: toward online intelligence that mirrors perception-action loops in humans.
Dual-process analogy: streaming reasoning resembles Kahneman's System 1 (fast, automatic); deep reasoning resembles System 2 (slow, deliberate). HRPO trains the coordination between them.
Embodied AI: self-driving cars and robots must decide mid-perception; adaptive compute allocation suits resource-constrained embedded systems.

Limitations and Future Directions

Limitations: high RL training cost, delicate reward weighting, limited memory over very long streams (hours of video), and heuristic (not provably optimal) compute allocation.

Future work: multimodal streaming reasoning (video + audio + screen in live meetings), metacognitive self-improvement, multi-agent social streaming reasoning, and neuro-symbolic streaming reasoning that verifies intuitive judgments with symbolic logic.

References

Tong, J., Xu, W., Fan, Y., Zhao, A., Lu, X., Tan, Y., & Shen, X. (2026). AdaSR: Adaptive Streaming Reasoning with Hierarchical Relative Policy Optimization. *arXiv preprint arXiv:2606.14694*.

Xiao, C., et al. (2024). StreamingLLM: Efficient Streaming Language Models with Attention Sinks. *ICML*.
Kahneman, D. (2011). *Thinking, Fast and Slow*. Farrar, Straus and Giroux.
Schulman, J., et al. (2017). Proximal Policy Optimization Algorithms. arXiv:1707.06347.