DynaTree: A Persistent Semantic Retrieval Tree for Time-Sensitive News Retrieval (KDD 2026)

Forum topic · 小凯 · 2026-06-01

Summary

DynaTree, a KDD 2026 paper from Shanghai Jiao Tong University and Orion Arm AI, addresses a core paradox in news retrieval: query semantics persist while document distributions shift daily. Instead of re-running agentic reasoning for every query, DynaTree materializes semantic query expansions into a persistent retrieval tree built offline by four cooperating agents (planning, retrieval, augmentation, and reflection), each handling path-aware expansion, adaptive strategy routing, budget-aware evidence normalization, and structural self-correction. At inference time, a lightweight daily subtree-selection mechanism uses a time-localized proxy corpus to pick the best-performing subtree without LLM inference. On BEIR, DynaTree consistently outperforms standard RAG and agentic baselines in recall and NDCG. In an online A/B test on the Syft news platform covering 100 topic channels, daily dynamic selection roughly doubled platform survival rate compared to a fixed subtree, outperforming all existing production retrievers every day. A one-time tree build costs about 70K tokens and four minutes per topic, amortizing agentic inference costs by orders of magnitude. The paper argues for decoupling expensive reasoning (offline) from lightweight selection (online), treating semantic expansions as reusable persistent assets rather than transient artifacts.

DynaTree: Dynamic Agentic Retrieval Tree for Time-Sensitive News Retrieval

Paper metadata

| Attribute | Content | |---|---| | Title | DynaTree: Dynamic Agentic Retrieval Tree for Time-Sensitive News Retrieval | | Authors | Siyuan Qi, Xinyuan Wang, Yingxuan Yang, Haochuan Guo, Jianghao Lin, Weiwen Liu, Yong Yu, Weinan Zhang | | Institutions | Shanghai Jiao Tong University, Orion Arm AI | | arXiv | 2605.31377 | | Date | 2026-05-29 | | Venue | KDD 2026 (cs.IR) | | Core claim | Semantic query expansion should be materialized as a persistent, reusable structured tree rather than a transient artifact discarded after each query; offline agentic reasoning builds the tree, online lightweight selection picks subtrees, balancing high recall with low latency for time-sensitive news retrieval |

The paradox of news retrieval: same query, changing answers

In news retrieval, a user's high-level interest is stable, but the relevant subtopics and surface expressions drift daily. Traditional systems either re-run semantic expansion per query (high latency and cost) or use fixed expansion vocabularies (unable to track topic drift). Agentic RAG was supposed to fix this, but existing methods tightly couple semantic expansion with retrieval decisions in short reasoning loops, re-running the agent on every query at large token cost.

DynaTree's answer: don't re-think every time — plant a semantic tree in advance and just decide daily which branches to prune. Where traditional agentic RAG is an explorer redrawing the map every trip, DynaTree is a settler with a permanent, well-surveyed camp, choosing routes daily based on conditions.

The retrieval tree: making semantic space persistent

Existing agentic RAG discards expansions after each use. DynaTree instead materializes them as a persistent retrieval tree: the root is the original query, child nodes are semantic expansions, and edges encode specialization. Each root-to-leaf path defines a coherent, progressively refined semantic trajectory.

Example for an "AI consumer electronics" query:

AI consumer electronics → chips & compute → mobile SoC competition → Apple vs Qualcomm
AI consumer electronics → on-device deployment → privacy computing → federated learning on devices
AI consumer electronics → imaging algorithms → computational photography → night-mode enhancement
AI consumer electronics → wearables → health monitoring → non-invasive glucose detection

Unlike flat expansions, the tree captures structural relationships, avoiding both redundancy between expansions and structural blind spots.

Why a tree, not a graph? Stage I expansion is path-conditioned and recursive, making each path an ordered expansion unit; Stage II subtree selection becomes tractable under a fixed online budget. General graphs would cause path explosion and unstable structure choices. The tree is both a representation choice and a computational constraint.

Four agents growing the tree (offline)

1. Planning agent — path-aware planning: generates expansions conditioned on the entire root-to-node path, encouraging progressive, complementary, and de-redundant growth. 2. Retrieval agent — adaptive routing: selects BM25, dense, or hybrid retrieval per query; technical terms favor lexical retrieval, conceptual expansions favor semantic retrieval. 3. Augmentation agent — budget-aware evidence normalization: deduplication, summarization, and quality filtering within a token budget; news evidence is highly redundant. 4. Reflection agent — structural self-correction: prunes overly narrow branches, merges semantically overlapping siblings (diversity check), detects coverage gaps, and enforces depth control.

These agents form an iterative loop with feedback rather than a simple pipeline, producing a well-designed semantic map rather than a bag of expansions.

Daily subtree selection (online)

Once built, the tree becomes persistent semantic infrastructure. The online phase uses a time-localized evaluation proxy: the most recent day's news subset evaluates candidate subtrees via Recall@α, choosing the highest-recall subtree as the day's retrieval strategy. Each subtree induces root-to-leaf paths encoded as text embeddings; document scores aggregate similarity across paths. The process involves no LLM inference, tree modification, or retraining — only lightweight vector operations.

This shifts cost from frequent online reasoning to one-time offline construction (~70K tokens per topic), amortizing per-query agentic cost toward zero for channels serving thousands of daily queries.

Offline results: BEIR benchmarks

On BEIR and a multi-day Syft News dataset, DynaTree consistently beats standard RAG and agentic baselines in recall and NDCG. Shapley-value analysis across 16 agent combinations shows the planning and augmentation agents contribute most, with retrieval-strategy diversity and reflection providing robustness gains. Notably, the analysis reveals non-additive interactions: removing the reflection agent matters far less when planning quality is already degraded, indicating strong synergistic dependence among the four agents. Surprisingly, simple uniform score aggregation slightly outperforms softmax-weighted aggregation in most cases — an engineering insight that complex weighting is not always better.

Production deployment: A/B test on Syft

DynaTree is deployed in the production pipeline of Syft, an AI news platform. The A/B test ran January 28 – February 6, 2026, over 100 topic channels, with the tree built once on January 27.

Port A (static): fixed offline-optimal subtree — survival rate 0.32–0.53
Port B (dynamic): daily subtree selection — survival rate 0.59–0.73

Both competed against Syft's five production retrievers (relation, decomposition, semantic, agentic, web-search recall) under identical downstream filtering and ranking. Port B beat all existing production retrievers every day, a ~1.5× improvement. Three takeaways: (1) subtree effectiveness genuinely drifts over time; (2) structure-aware unified selection beats heterogeneous static strategies; (3) offline cost is amortizable — roughly an 870:1 cost ratio versus per-query agentic reasoning under realistic assumptions.

Design philosophy

Decoupling reasoning from retrieval: run expensive reasoning offline, lightweight selection online — analogous to compile-time optimization vs. runtime execution.
Persistent structures vs. transient artifacts: treat semantic expansions as accumulable, reusable assets, shifting from per-query processing to long-term investment in the query's semantic space.
Tree as a computational constraint: chosen not for expressiveness but for the best expression–efficiency trade-off under latency constraints.

Limitations and future work

Cold start: building a new tree takes ~4 minutes, during which the system falls back to existing retrievers — potentially problematic for breaking news.
Maintenance cost: total offline cost grows linearly with channel count; incremental branch-level updates could help at platform scale.
Recall focus: ranking precision and diversity remain open; high recall needs to be paired with finer ranking.
Cross-topic transfer: unexplored — reusing shared subtrees across related topics (e.g., "AI consumer electronics" and "AI automotive") could reduce maintenance cost from linear to sub-linear, akin to inheritance in software engineering.

Conclusion

DynaTree's significance goes beyond survival-rate gains: it is an architectural paradigm shift. While most agentic RAG systems optimize per-query reasoning speed, DynaTree asks a more fundamental question: if the query's semantic space is persistent, why re-reason every time? The answer is a tree — grown offline by collaborative agents, pruned daily by lightweight selection, amortizing expensive reasoning across days of service. It suggests that the next frontier of AI efficiency may lie not in model capability but in architecture: organizing, storing, and reusing what models already produce.

References

1. Qi, S., Wang, X., Yang, Y., et al. (2026). *DynaTree: Dynamic Agentic Retrieval Tree for Time-Sensitive News Retrieval*. KDD 2026. arXiv:2605.31377 [cs.IR]. 2. Lewis, P., et al. (2020). *Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks*. NeurIPS 2020. 3. Yao, S., et al. (2023). *ReAct: Synergizing Reasoning and Acting in Language Models*. ICLR 2023. 4. Asai, A., et al. (2024). *Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection*. ICLR 2024. 5. Lavrenko, V., & Croft, W. B. (2001). *Relevance-Based Language Models*. SIGIR 2001.