Proactive Agents Don't Need an LLM Every Time — A Small Graph Model Is Enough, Up to 83x Faster

Forum topic · ✨步子哥 · 2026-05-30

Summary

A new paper (arXiv:2605.30152) argues that proactive AI agents should not call an LLM for every event to decide whether to intervene. Instead of flattening structured user activity (actor, verb, object, timestamp) into text and asking an LLM to judge each event, the proposed temporal graph learning (TGL) model encodes the natural event graph directly and outputs two things: a trigger probability and a routing score. Only when an event is deemed relevant is a compact structured handoff sent to the LLM to generate a user-facing response. Across 14 LLM backbones, TGL improves trigger-decision F1 by 16.7 points on average (up to 46.0), runs at 11.13 ms/event on GPU servers (4-7x faster) and 13.99 ms/event on consumer laptops (12-83x faster), and its 220 MiB (BF16) footprint allows on-device deployment alongside privacy-sensitive activity data, with more stable thresholds across environments. The paper also candidly notes limitations: TGL is distilled from LLM labels, was validated only for triggering, depends on structured event streams, and may need retraining across domains.

Proactive Agents Don't Need to Wake the LLM Every Time — A Small Graph Model Is Enough, Up to 83x Faster

> Source: Do Proactive Agents Really Need an LLM to Decide When to Wake and What to Anchor?, https://arxiv.org/abs/2605.30152

1. Motivation: Your AI assistant is woken 3,600 times an hour — 3,599 of them are "nothing"

Imagine a proactive agent that monitors everything you do — opening files, switching windows, receiving emails, editing your calendar — and decides whether to proactively help.

The current approach: feed every event to an LLM and let it decide whether the event is worth acting on.

Sounds reasonable, but consider: you open a PDF. The system converts the event into text — "user opened quarterly_report.pdf" — and sends it to GPT-4: "Does this event require proactive action?" GPT-4 thinks: "No." That took 200 ms and consumed hundreds of tokens. You close a browser tab. Same round trip, same answer.

Over a day, 99% of the judgments are "no need" — but you pay 99% of the compute bill for them.

The paper asks a counterintuitive question: does deciding whether to act really require an LLM?

2. Key insight: user activity is naturally a graph, not text

User activity is not text — it is a structured event stream. Every event is a 4-tuple: (actor, verb, object, timestamp). These events naturally form a temporal graph: entities are nodes, events are edges, timestamps are edge attributes.

The current practice flattens this graph into text, feeds it to an LLM, and has the LLM reconstruct the graph in its head before making a judgment — like reading a map aloud to a blind person who must redraw it from memory before giving directions. An unnecessary round trip.

3. Approach: replace the LLM trigger with a small graph model

1. Temporal graph learning (TGL) model as encoder: a tiny graph neural network whose single forward pass outputs:

Trigger probability: is this event worth waking the agent?
Routing score: if yes, which entities should the agent focus on?

LLM is invoked only on trigger

Tiny model

Analogy: TGL is the gate guard; the LLM is the expert inside. The guard checks credentials, rejects 99% of visitors, and only lets genuinely important ones through.

4. Results: the small model wins decisively

Accuracy: average +16.7 F1, up to +46.0

Across 14 LLM backbones, TGL beats LLM-as-trigger on trigger decisions across the board — not "about equal," but a comprehensive win. The LLM's forced text round trip loses structural information; TGL operates directly on the graph and preserves it.

Speed: 4-83x faster

GPU server: 11.13 ms/event, 4-7x faster than the fastest LLM-as-trigger configuration
Consumer laptop: 13.99 ms/event, 12-83x faster than LLM-as-trigger

If the LLM takes ~1 second per event, TGL takes ~12 ms.

Stability: more robust thresholds

A single TGL checkpoint yields stable trigger thresholds across deployment environments; LLM-as-trigger thresholds fluctuate and are hard to tune.

Privacy: 220 MiB enables on-device deployment

TGL can stay on-device with user activity data — no need to ship every activity log to a cloud LLM.

5. Broader implication: not every "smart" task needs a "big" model

"Use an LLM for everything" is a lazy architecture. LLMs excel at natural language understanding, fluent generation, and complex reasoning. But "decide whether a structured event deserves attention" requires pattern matching and temporal reasoning — exactly what graph neural networks do well. Using an LLM as a trigger is like delivering parcels with a sports car: it works, but wastes speed and fuel; a bicycle suffices.

Future AI systems may not be "one LLM to rule them all," but many small models with clear roles, with the LLM appearing only where it matters: TGL as gatekeeper, retrieval as librarian, the LLM as consultant.

6. Honest assessment

1. Where does TGL's training data come from? The paper trains TGL on LLM-labeled data, so TGL is essentially a distillation of the LLM's judgment — including its biases. 2. Only triggering is validated: whether TGL can handle more complex decisions (e.g., which action to take) is unaddressed. 3. Graph structure dependency: TGL needs structured event streams from the OS; incomplete or inconsistent event formats will degrade results. 4. Generalization: TGL is trained on specific domains of activity streams and may need retraining elsewhere.

7. One-line takeaway

Deciding "whether to act" doesn't need a large model — a small graph model is enough: 83x faster, 16.7 F1 points more accurate, and privacy-preserving. Sometimes the smartest architectural decision is not "use a stronger model" but "use the right model."

Paper link: https://arxiv.org/abs/2605.30152