GLM: A Multi-Agent Framework for Large-Scale Graph Reasoning with Efficient LLM Serving

Background and Problem

Graph Chain-of-Thought (Graph-CoT) is an emerging paradigm that guides LLMs through step-by-step reasoning over structured knowledge graphs to answer complex multi-hop questions and reduce hallucination. However, mainstream Graph-CoT implementations rely on a single-agent architecture, bundling question classification, retrieval, logical reasoning, and action generation into one large prompt handled by a single LLM. This design suffers from:

• "Lost in the middle": LLMs tend to overlook key information located in the middle of long contexts
• Repeated context re-encoding: every iteration reprocesses the entire context, wasting computation
• Serial execution: all steps must run sequentially with no parallelism

There is also a sharp accuracy–efficiency trade-off: broader graph retrieval improves accuracy but inflates prompt length and token cost. Some commercial models cost over $3 per complex query, while cost-constrained configurations that limit retrieval can fall below 50% accuracy.

The GLM Framework

GLM (Graph-CoT with Multi-Agent and Efficient LLM Serving) addresses these issues through two co-designed pillars:

1. Multi-Agent Collaborative Reasoning

Instead of a monolithic agent, GLM decomposes the reasoning process into four specialized components:

• C-Agent (Classification Agent) — routes each query
• R-Agent (Reasoning Agent) — performs step-by-step reasoning for non-deterministic queries
• A-Agent (Action Agent) — generates actions/executable code against the graph
• Graph RAG Retriever — executes retrieval; deterministic queries take a fast path directly to retrieval

The workflow: a user query is classified by the C-Agent; deterministic queries go straight through the retriever's fast path, while non-deterministic queries enter an R-Agent → A-Agent → retriever loop that iterates until sufficient information is gathered, then synthesizes the final answer.

Benefits:

• Modular tasks avoid the "lost in the middle" problem
• Selective context sharing reduces information redundancy
• Supports branched and parallel execution paths

2. Co-Designed LLM Serving Architecture

GLM deeply co-designs the reasoning framework with the underlying LLM serving layer, introducing serving mechanisms tailored to graph-reasoning workloads:

• Graph-aware KV cache management
• Priority-based cache eviction policies
• Pipeline parallel execution optimization

Reported Performance

| Metric | Improvement | |---|---| | Token cost reduction | 95.7% | | Inference latency reduction | 90.3% | | Throughput improvement | 15.1x | | Accuracy gain over baselines | up to 38% |

Applications and Outlook

GLM targets knowledge-intensive tasks over large knowledge graphs—multi-hop question answering, graph-based retrieval, and complex analytical queries—where accuracy, latency, and token cost must be balanced simultaneously. The paper suggests that combining agent specialization with system-level serving optimization is a promising direction for scalable graph reasoning. Reference: GLM framework paper (arXiv:2511.01633).