GLM-5 In-Depth Technical Research Report: Zhipu AI's 744B Open-Source Agentic Engineering Model

Forum topic · ✨步子哥 · 2026-02-21

Summary

GLM-5 is Zhipu AI's flagship open-source large language model (released February 11, 2026), positioned as the state-of-the-art open model for agentic software engineering. It scales to 744B total parameters with only ~40B activated per token via a 256-expert MoE architecture, was pretrained on 28.5T tokens, and supports a 202K-token context window with 128K-token output. Architecturally, it adopts DeepSeek's Sparse Attention (DSA) with a Lightning Indexer and fixed Top-k=2048 selection, plus Multi-Token Prediction (MTP) for faster generation. Its training pipeline is headlined by Slime, a novel asynchronous reinforcement-learning infrastructure that decouples rollout generation from training, enabling fine-grained iteration and delayed credit assignment for long-horizon agent tasks. Trained entirely on domestic Chinese AI chips (Huawei Ascend, Cambricon, Moore Threads, and others), GLM-5 is released under an MIT license. Benchmarks show open-source SOTA results: 77.8 on SWE-bench Verified, 56.2 on Terminal Bench 2.0, and $4,432 on Vending Bench 2—approaching Claude Opus 4.5—and it ranks fourth globally and first among open models on the Artificial Analysis index. The report also notes limitations, including undisclosed architectural details and reliance on DeepSeek-derived components.

GLM-5 Deep Technical Research Report (Structured Summary)

*Note: The source is a long-form Chinese technical report. Below is a structured English summary preserving key facts, figures, and claims.*

1. Overview and Positioning

GLM-5 (released Feb 11, 2026) marks Zhipu AI's shift from "Vibe Coding" (lightweight chat-based code assistance) to "Agentic Engineering" — autonomous planning, multi-step execution, long-term memory, and end-to-end software lifecycle completion.
It ranks 4th globally and 1st among open models on the Artificial Analysis index, the first open model to exceed 50 points on the composite intelligence score.
Before release, GLM-5 was anonymously tested on OpenRouter as "Pony Alpha", topping popularity charts with >91% of users correctly guessing its identity.
Released under the MIT license (fully open weights); trained entirely on domestic Chinese compute — acknowledgments include Huawei Ascend, Moore Threads, Cambricon, Kunlun, MetaX, Enflame, and Hygon.

2. Key Specifications

| Dimension | GLM-5 | Comparison | |---|---|---| | Total parameters | 744B | DeepSeek-V3.2: 685B; 2.1× GLM-4.5's 355B | | Activated params | ~40B–44B | 256 experts, 8 routed per token (~5.9% sparsity) | | Context window | 202K in / 128K out | DeepSeek-V3.2: 128K | | Pretraining data | 28.5T tokens | vs 23T for GLM-4.5 (+24%) | | Depth | 78 layers | First 3 dense FFN, remaining 75 MoE | | License | MIT | vs Llama's tiered, Qwen's limited commercial terms |

3. Architecture Innovations

Mixture-of-Experts (MoE)

256 expert networks, 8 activated per token; activated params rose only 25% (32B→40B) while total params grew 109%, implying efficiency via finer-grained routing. Shared experts likely explain the gap between routed-expert params (~23B) and reported ~44B activation.

DeepSeek Sparse Attention (DSA)

Reduces attention complexity from O(L²) to O(L·k); for 128K context with k=2048, attention pairs drop from ~8.2B to ~0.26B (~97% compression).
Two-stage pipeline: (1) Lightning Indexer — a lightweight scorer using ReLU instead of Softmax (~5% of full-attention compute) producing relevance scores; (2) full attention on fixed Top-k=2048 selected tokens.
Code review shows GlmMoeDsaForCausalLM inherits from DeepseekV2ForCausalLM — an architectural reuse, not original DSA research.

Multi-Token Prediction (MTP)

Speculative decoding (vLLM config: num_speculative_tokens 1) reduces generation latency and complements DSA's compute savings.

4. Training: Slime Asynchronous RL Framework

Slime is an original asynchronous RL infrastructure that decouples rollout generation from gradient updates via dedicated worker pools and a shared experience buffer, pushing GPU utilization toward 100%.
Handles off-policy drift with importance sampling correction; supports fine-grained (sub-episode) iteration and trajectory replay.
Enables agentic RL at scale: automatic decomposition of tasks into hundreds of sub-steps and delayed credit assignment for long-horizon tasks.
Alignment uses mixed supervision: synthetic trajectory distillation, human preference feedback, and self-play verification.

5. Benchmark Results

| Benchmark | GLM-5 | Closest comparison | |---|---|---| | SWE-bench Verified | 77.8 | Claude Opus 4.5 ~79 | | Terminal Bench 2.0 | 56.2 | Claude Opus 4.5 ~58 | | Humanity's Last Exam (tool reasoning) | 50.4% | GLM-4.7: 42.8% | | Vending Bench 2 | $4,432 | Claude Opus 4.5 ~$4,500 | | BrowseComp / MCP-Atlas / τ²-Bench | Leading | Open-source first |

Internal CC-Bench-V2 shows >20% average improvement over GLM-4.7 on frontend, backend, and long-horizon tasks.
Demonstrated end-to-end capabilities: generating deployable microservices with tests and CI from requirement docs, full GBA emulator development, and macOS UI simulation.

6. Deployment and Ecosystem

Adapted to Ascend, Moore Threads, Cambricon, Kunlun, MetaX, Enflame, Hygon; quantization (W4A8/INT4/INT8) and kernel fusion reportedly halve deployment cost vs. dual-GPU international clusters.
FP8 needs ≥8×H200; ~1–2s first-token latency, 30–60 tokens/sec sustained. Supported by vLLM, SGLang, KTransformers, xLLM.
Much of the architectural detail was community-mined from inference-framework PRs rather than officially published.

7. Limitations and Future Directions

Undisclosed details include: Lightning Indexer internals, Slime's communication protocol, pretraining data composition, and MoE routing/load-balancing mechanics. Future directions: DSA generalization beyond 1M-token contexts, multi-agent collaborative engineering, continual learning, and deep multimodal integration.