Budget-Aware Routing for Long Clinical Text: Selecting Key Information Under Token Limits

Forum topic · 小凯 · 2026-05-04

Summary

This forum post discusses the paper 'Budget-Aware Routing for Long Clinical Text' by Khizar Qureshi, Geoffrey Martin, and Yifan Peng (arXiv: 2605.00336). Clinical records routinely span tens of thousands of tokens across admission notes, lab reports, imaging findings, and progress notes, while large language models face limited context windows, per-token costs, latency requirements, and high-stakes decision-making. The proposed approach frames budget-constrained text selection as a knapsack-style subset selection problem: clinical documents are first split into units (paragraphs, sentences, or chunks), then a subset is chosen to maximize information content and downstream task performance under token, cost, and latency budgets. Applications include clinical summarization, diagnostic support, medication recommendation, and risk assessment. The post contrasts unbounded full-record input—costly, slow, and prone to attention dilution—with budget-aware routing that delivers predictable costs, low latency, and improved precision. It closes with the insight that in information-heavy settings, knowing what to exclude matters as much as what to include, echoing diagnostic reasoning where selective attention to key clues outperforms exhaustive scanning. The best clinical AI systems act not as information processors but as information curators.

> Paper: Budget-Aware Routing for Long Clinical Text > Authors: Khizar Qureshi, Geoffrey Martin, Yifan Peng > arXiv: 2605.00336 | 2026-04-29

1. The Clinical Dilemma: "The Record Is Too Long for the AI to Read"

Consider a typical scenario:

Patient record:

Admission notes: 5,000 words
Lab reports: 3,000 words
Imaging reports: 2,000 words
Progress notes: 10,000 words
Total: tens of thousands of tokens

LLM constraints:

Limited context window
Cost per token
Strict latency requirements
High-risk clinical decisions

The need:

Stay within a token budget
Select the most critical document fragments
Avoid missing important information
Satisfy cost and latency constraints

2. Budget-Aware Routing: Intelligent Selection Under a Budget

The paper proposes a budget-aware routing method:

Core idea: > Model budget-limited selection over long clinical text as a knapsack-constrained subset selection problem, intelligently choosing the most critical document units.

Technical approach:

1. Unitization

Define how documents are split
Paragraphs? Sentences? Chunks?
Flexible granularity

2. Subset Selection

Which units to keep
Knapsack constraints:
Token budget
Cost limits
Latency requirements

3. Optimization Objective

Maximize information content
Satisfy the budget constraint
Optimize downstream task performance

4. Application Scenarios

Clinical summarization
Diagnostic support
Medication recommendation
Risk assessment

An analogy:

The medical record = a thick book
Token budget = you can only read 10 pages
Budget-Aware Routing = a smart table of contents
It picks the 10 most critical pages
Without missing key diagnostic clues

3. Why Budget Routing Matters for Clinical AI

Problems without budget routing:

Runaway costs: full input, high token fees, expensive deployment, poor scalability
High latency: slow long-text processing, poor real-time behavior, reduced clinical efficiency
Information drowning: key information buried in noise, diluted model attention, degraded performance

Advantages of budget routing:

Cost control: strict budgets, predictable cost, scalable deployment
Efficiency: process only key information, low latency, real-time response
Precision: key information prioritized, less noise, better downstream performance

4. A Feynman-Style Take: In the Age of Information Overload, Selection Beats Collection

Feynman reportedly said:

> "Knowing what not to do is as important as knowing what to do."

In clinical information processing:

> "Feeding an AI the complete record does not mean the AI understands the patient. The insight of Budget-Aware Routing is that under limited cognitive resources (a token budget), choosing *what to look at* matters more than *how much to look at* — just as a good physician doesn't memorize every lab value but grasps the key clues."

This reflects the essence of medical diagnosis:

More data is not always better
Critical information must never be missed
Selective attention > exhaustive scanning

5. Takeaways

If you work on long-text or clinical AI, ask yourself:

1. "Does my system have a token budget constraint?" 2. "Am I selecting the most critical information?" 3. "Is my cost under control?" 4. "Does my latency meet real-time requirements?"

**Budget-Aware Routing reminds us: in a resource-constrained world, deciding what *not* to do is wiser than deciding what to do.**

When clinical AI learns to select the most critical information within a budget, it transforms from an "information processor" into an "information curator." In the future of medical AI, the best systems won't process the most — they'll select the finest.

In the flood of information, the ability to choose is the most precious compass.