Your phone's AI assistant does something incredibly wasteful every moment: it translates your life into text, then reads it back to itself.
You open an app—it logs "user opened WeChat." A message arrives—"user received a message from Zhang San." You switch songs—"user switched playlist track." Then it stuffs all this text into a large language model and asks: "Should I proactively do something?"
99% of the time, the LLM says: "No."
This is how today's proactive agents work: call an LLM for every single event to decide whether to act. It sounds intelligent, but it's absurd—you've hired a PhD to be on 24-hour standby just to glance at "user swiped the screen again" every second and say "nothing to do."
The paper behind this post asks a pointed question: do you actually need an LLM to make this decision?
The Core Problem: Structure Gets Flattened
The key insight: user activity is naturally structured, not textual.
OS-level user behavior is recorded as (actor, verb, object, timestamp) tuples—e.g., (user, open, WeChat, 14:30:02). These tuples are already organized as a graph structure inside the OS: who did what to what, when, and how it relates to prior events.
But existing proactive agents take this graph, flatten it into text, and ask an LLM to recover the structure from the text. It's like tearing a map into pieces, turning it into a sentence, and asking someone to redraw the map from the sentence—pointless, and information is necessarily lost.
The paper's core claim: treat always-on signals as graph updates, not text.
The Solution: Small Model as Gatekeeper, LLM as Writer
The proposed architecture is remarkably simple:
Tier 1: A temporal graph learning (TGL) model as gatekeeper. A tiny graph neural network consumes the structured event stream and outputs two things:
- Trigger probability: Is this event worth waking the LLM?
- Routing score: If so, which downstream agent should handle it?
- GPU server: TGL runs at 11.13 ms per event, 4-7x faster than the fastest LLM-as-trigger configuration
- Consumer laptop: 13.99 ms per event, 12-83x faster than the fastest LLM-as-trigger configuration
One forward pass, 11 milliseconds.
Tier 2: The LLM is only called when triggered. It receives not the raw text stream but a compact, structured "handoff packet" containing only relevant context. The LLM's sole job is converting structured information into fluent, user-facing natural language.
Analogy: the TGL is a building's security guard; the LLM is the front-desk receptionist. The guard watches the monitors (event stream) 24/7, decides who needs attention (triggering), and only brings those people to the front desk (routing). The receptionist never watches all the monitors—they only talk to whoever is brought over.
Results: The Small Model Crushes the Big One
Across all 14 backbone models, TGL improved F1 scores, averaging +16.7, up to +46.0. All of them—no exceptions.
Speed gap is even more dramatic:
Trigger stability: a single TGL checkpoint delivered the strongest trigger AUC and the most stable deployment thresholds across all tested scenarios. By contrast, LLM-as-trigger performance fluctuated heavily across scenarios.
Why Is the Small Model Better?
This counterintuitive result has deep reasons:
1. Structured information naturally fits graph models. The (actor, verb, object, timestamp) event structure is literally graph nodes and edges. Graph neural networks natively excel at this; LLMs must first "understand" implicit structure from text.
2. Trigger decisions are pattern matching, not reasoning. Judging whether "user opened WeChat, then Alipay 30 seconds later" is worth triggering is fundamentally sub-pattern matching on a graph. It doesn't need LLM reasoning—just efficient structured pattern recognition.
3. The LLM's "understanding" is wasted here. LLMs excel at fluent text generation and complex reasoning, but triggering needs neither. Using an LLM for this is like using a sports car for package delivery—it works, but it's absurdly expensive.
4. Latency is fatal. Proactive agents need real-time responses. LLM inference latency (even a single forward pass) becomes a bottleneck under dense events. TGL's 11 ms vs. an LLM's 50-900 ms is a difference of orders of magnitude.
The Deeper Lesson: Not Every AI Problem Needs an LLM
The implications go far beyond proactive agents:
LLMs are general-purpose reasoning engines, but not omnipotent. Many AI applications need not "natural language understanding" but "efficient structured data processing." Flattening structured data into text for an LLM is an architectural anti-pattern.
The right division of labor: small models for perception and filtering, large models for reasoning and generation. This isn't new—classic two-stage detectors (like Faster R-CNN) work exactly this way: a lightweight network proposes candidates, a heavyweight network does fine-grained classification. The idea was forgotten in the LLM era because everyone fell in love with the elegance of "one end-to-end giant model."
End-to-end doesn't mean one model does everything. True end-to-end means optimizing the whole system end-to-end—not cramming all functionality into one model. When your system has clear perception-decision-action layers, using models of different scales and types at each layer is the engineering-correct choice.
An Honest Assessment
The limitations are obvious: the TGL model must be trained for a specific event stream format and, unlike LLMs, doesn't zero-shot generalize. If the event stream schema changes, retraining may be needed. Also, the experiments were mainly conducted in simulated environments; real-world deployment remains to be validated.
But the direction is right. While the entire AI world sprints toward "bigger, more general," this paper reminds us: sometimes, small and specialized is the correct answer. Not every problem needs GPT-4—sometimes a 220 MiB model is enough, and does it better.
---
Paper link: https://arxiv.org/abs/2605.30152
Authors: Xiaoze Liu, Ruowang Zhang, Amir H. Abdi, Michel Galley, Zhikai Chen, Siheng Xiong, Xiaoqian Wang, Jing Gao