Proactive Agents Don't Need to Wake an LLM Every Time — A Small Graph Model Is 83x Faster
> Source: *Do Proactive Agents Really Need an LLM to Decide When to Wake and What to Anchor?*, https://arxiv.org/abs/2605.30152
The Problem: Your AI Assistant Gets Woken 3,600 Times an Hour — 3,599 for Nothing
Imagine a proactive agent that monitors your activity — opening files, switching windows, receiving emails, editing your calendar — and decides whether to step in and help.
The current approach: feed every single event to an LLM and ask "is this worth acting on?"
You open a PDF. The system converts the event to text ("user opened quarterly_report.pdf") and sends it to GPT-4: "Does this event require proactive action?" GPT-4 thinks: "No." — 200 milliseconds and hundreds of tokens spent.
You close a browser tab. Same conversion, same LLM call, same "no."
Over a day, 99% of these judgments are "not needed." But you pay 99% of the compute bill for those "noes."
The paper asks a counterintuitive question: does deciding "whether to act" actually require an LLM?
Key Insight: User Activity Is Naturally a Graph, Not Text
The paper's core observation: user activity is not text — it's a structured event stream.
Each event is a 4-tuple: (actor, verb, object, timestamp). For example:
- (User A, open, quarterly_report.pdf, 14:32:01)
- (User A, close, Chrome Tab 7, 14:32:15)
- (User B, send, email "Q3 budget", 14:33:00)
- Trigger probability: is this event worth waking the agent?
- Routing score: if yes, which entity should it anchor on?
- GPU server: 11.13 ms/event, 4-7x faster than the fastest LLM-as-trigger setup
- Consumer laptop: 13.99 ms/event, 12-83x faster than LLM-as-trigger
Operating systems already maintain this structured data, and these events naturally form a temporal graph: entities are nodes, events are edges, timestamps are edge attributes.
The current approach flattens this graph into text, feeds it to an LLM, and lets the LLM mentally rebuild the graph before making a judgment.
It's like having a map in hand, but insisting on describing it aloud to a blind person, who draws it from memory in their head, and then gives you directions based on that mental map.
An unnecessary round trip.
The Solution: A Small Graph Model Replaces the LLM for Trigger Decisions
The proposed approach is remarkably simple:
1. A temporal graph learning (TGL) model as encoder: a very small graph neural network that, in one forward pass, outputs:
2. The LLM is only called when triggered: only when the TGL says "this event matters" is a small structured handoff packaged and sent to the LLM to generate a user-visible response.
3. The TGL model is tiny: 220 MiB (BF16), deployable on-device alongside privacy-sensitive activity data.
Analogy: the TGL is the doorman, the LLM is the expert inside. The doorman glances at credentials, says "not needed" 99% of the time, and only lets the truly important 1% through to see the expert.
Results: The Small Model Wins Decisively
Accuracy: Average F1 gain of 16.7, up to 46.0
Across 14 LLM backbones, TGL's trigger decisions beat every LLM-as-trigger configuration — not "about the same," but a clean sweep. Why? Because forcing an LLM to convert structured data into text and back loses information. TGL operates directly on the graph, preserving structure natively.
Speed: 4-83x faster
Stability: More robust thresholds
A single TGL checkpoint yields stable trigger thresholds across deployment environments, whereas LLM-as-trigger thresholds fluctuate and are hard to tune.
Privacy: 220 MiB enables on-device deployment
At 220 MiB, the TGL model can stay on-device with the user's activity data — no need to ship every activity log to the cloud for an LLM to inspect.
The Bigger Lesson: Not Every "Intelligence" Needs a "Big Model"
The paper's implications go beyond agent triggering:
In AI system design, "use an LLM for everything" is lazy architecture.
LLMs excel at understanding language, generating fluent text, and complex reasoning. But "decide whether a structured event is worth attention" doesn't need language understanding, text generation, or deep reasoning. It needs pattern matching and temporal reasoning — exactly what graph neural networks do well.
Using an LLM for trigger decisions is like using a sports car for deliveries: it works, but wastes the car's speed and burns fuel. A bicycle suffices.
Future AI systems may not be "one LLM to rule them all," but many small models doing their jobs, with the LLM appearing only where it matters most: TGL as doorman, retrieval models as librarians, LLM as consultant — each playing to its strengths.
Honest Assessment
1. Where does TGL's training data come from? The paper trains TGL on LLM-labeled data, so TGL is essentially a distillation of LLM judgment. Any bias in the LLM's decisions is inherited. 2. Only trigger decisions were validated: whether TGL can handle more complex decisions (e.g., "what action should be taken") is not explored. 3. Dependence on graph structure: TGL requires the OS to provide structured event streams. Incomplete or inconsistent streams would degrade performance. 4. Generalization: TGL is trained on domain-specific activity streams and may require retraining in new domains.
One-Sentence Takeaway
Deciding "whether to act" doesn't need a large model — a small graph model suffices: 83x faster, 16.7 F1 points more accurate, and privacy-preserving. Sometimes the smartest architectural decision isn't "use a stronger model" — it's "use the right model."
Paper link: https://arxiv.org/abs/2605.30152