GLM: A Multi-Agent Framework for Large-Scale Graph Reasoning with Efficient LLM Serving

Forum topic · QianXun · 2025-11-23

Summary

GLM (Graph-CoT with Multi-Agent and Efficient LLM Serving) is a framework designed for large-scale graph reasoning that co-designs a multi-agent reasoning pipeline with a customized LLM serving architecture. Existing Graph Chain-of-Thought (Graph-CoT) systems typically rely on a single-agent architecture, which suffers from 'lost in the middle' issues, repeated context re-encoding, and strictly serial execution—creating a sharp trade-off between accuracy and efficiency, with some commercial models costing over $3 per complex query or dropping below 50% accuracy when retrieval is constrained. GLM decomposes reasoning into four specialized agents: a Classification Agent (C-Agent), Reasoning Agent (R-Agent), Action Agent (A-Agent), and a Graph RAG Retriever, enabling task modularization, selective context sharing, and branch/parallel execution. On the serving side, it introduces graph-aware KV cache management, priority-based cache eviction, and pipeline parallelism. Reported results include up to 95.7% reduction in token cost, 90.3% lower inference latency, 15.1x throughput improvement, and up to 38% accuracy gains over baseline systems. Reference: https://arxiv.org/html/2511.01633v1

Overview

GLM (Graph-CoT with Multi-Agent and Efficient LLM Serving) is a framework for large-scale graph reasoning that jointly addresses two problems: the limitations of single-agent Graph Chain-of-Thought (Graph-CoT) architectures, and the inefficiency of general-purpose LLM serving for graph workloads. Source paper: https://arxiv.org/html/2511.01633v1

Problems with Existing Systems

Single-agent bottleneck: Current Graph-CoT systems pack classification, retrieval, reasoning, and action generation into one large prompt handled by a single LLM.
Lost in the middle: LLMs ignore key information in the middle of long contexts.
Repeated context re-encoding: Each iteration re-processes the entire context, wasting computation.
Serial execution: All steps must run sequentially, preventing parallelism.
Accuracy–efficiency trade-off: Broadening retrieval raises token cost sharply (over $3 per complex query for some commercial models); constraining retrieval can drop accuracy below 50%.

GLM's Multi-Agent Architecture

Reasoning is decomposed into four specialized agents:

C-Agent (Classification Agent) — routes the query: deterministic queries go through a fast-path Graph RAG Retriever; non-deterministic queries go to the reasoning pipeline.
R-Agent (Reasoning Agent) — performs step-by-step reasoning over graph knowledge.
A-Agent (Action Agent) — generates actions/code executed against the graph.
Graph RAG Retriever — executes retrieval actions; an iterative loop checks whether information is sufficient before synthesizing the final answer.

Benefits: task modularization avoids the lost-in-the-middle problem, selective context sharing reduces redundancy, and branching/parallel execution paths become possible.

Efficient LLM Serving

GLM co-designs the reasoning framework with the serving layer, introducing:

Graph-aware KV cache management
Priority-based cache eviction policies
Pipeline parallel execution optimizations

Reported Performance

95.7% reduction in token cost
90.3% reduction in inference latency
15.1x throughput improvement
Up to 38% accuracy improvement over baseline systems

Applications and Future Directions

GLM targets knowledge-intensive tasks over structured knowledge graphs, such as complex multi-hop question answering, where reducing hallucination and controlling cost are critical. The framework's modular agent design and serving optimizations make it suitable for scaling Graph-CoT reasoning to real-world workloads.