GLM-5 Deep Technical Analysis: Zhipu AI's 744B Open-Source Agentic Engineering Model

Forum topic · ✨步子哥 · 2026-02-21

Summary

GLM-5 is an open-source large language model from Zhipu AI marking a shift from 'Vibe Coding' to 'Agentic Engineering'. It scales to 744B total parameters with only 40-44B activated per token via a 256-expert MoE architecture, was trained on 28.5T tokens, and supports a 202K context window with 128K output. Released under an MIT license, it was trained entirely on domestic Chinese hardware including Huawei Ascend, Cambricon, and Moore Threads chips. The model adopts DeepSeek Sparse Attention (DSA) with a Lightning Indexer and fixed Top-k=2048 selection to cut attention complexity from O(L²) to O(L·k), plus Multi-Token Prediction for faster generation. Its original Slime asynchronous reinforcement learning framework decouples rollout generation from training, enabling fine-grained iteration and delayed credit assignment for long-horizon agent tasks. GLM-5 ranks first among open-source models (fourth globally per Artificial Analysis), scoring 77.8 on SWE-bench Verified and $4,432 on Vending Bench 2, approaching Claude Opus 4.5. It was pre-released anonymously as 'Pony Alpha' on OpenRouter, topping popularity charts before launch.

GLM-5 Deep Technical Research Report

> Structured English summary of the original Chinese deep-dive report on Zhipu AI's GLM-5.

Key points

Paradigm shift: GLM-5 marks Zhipu AI's move from "Vibe Coding" (lightweight chat-based prototyping) to "Agentic Engineering" — autonomous planning, multi-step execution, long-term memory, and end-to-end software lifecycle completion.
Open-source SOTA: Ranks 4th globally and 1st among open-source models on the Artificial Analysis intelligence index (first open model above 50 points), competing with Claude Opus 4.5, GPT-5.2, and Gemini 3 Pro.
Released Feb 11, 2026, after anonymous pre-testing as "Pony Alpha" on OpenRouter, where 91%+ of users correctly identified it and it topped popularity charts.

Architecture

| Spec | GLM-5 | Comparison | |------|-------|------------| | Total parameters | 744B (2.1× GLM-4.5's 355B) | DeepSeek-V3.2: 685B | | Activated parameters | 40B–44B per token (5.9% sparsity) | DeepSeek-V3.2: 37B | | Context window | 202K input / 128K output | DeepSeek-V3.2: 128K | | Layers | 78 (first 3 dense FFN, remaining 75 MoE) | DeepSeek-V3: 61 | | Experts | 256 experts, 8 routed per token | Mixtral: 8 experts | | Pre-training data | 28.5T tokens (up from 23T) | Llama 3.1: ~15T | | License | MIT (fully open commercial use) | Llama: tiered license |

DeepSeek Sparse Attention (DSA)

Reduces attention complexity from O(L²) to O(L·k) with fixed Top-k=2048 selection (~97% compression for 128K contexts).
Lightning Indexer: lightweight scorer (~5% of full attention compute) using ReLU instead of Softmax to rank historical token relevance, followed by full attention on selected tokens only.
Code review shows the implementation (GlmMoeDsaForCausalLM) directly inherits from DeepseekV2ForCausalLM — architecture reuse rather than original invention.

Multi-Token Prediction (MTP)

Speculative decoding config (num_speculative_tokens 1) suggests conservative one-token-ahead prediction, synergizing with DSA's reduced attention cost.

Training infrastructure

Slime asynchronous RL framework (original contribution): fully decouples rollout generation from training via a shared experience buffer, achieving near-100% GPU utilization with importance-sampling correction for mild off-policy data. Supports fine-grained, sub-episode iteration and delayed credit assignment across hundreds of task sub-steps.
Fully domestic compute stack: trained on Huawei Ascend with MindSpore; acknowledgments span Huawei Ascend, Moore Threads, Cambricon, Kunlunxin, MetaX, Enflame, and Hygon.

Benchmarks

| Benchmark | GLM-5 | Claude Opus 4.5 | Note | |-----------|-------|-----------------|------| | SWE-bench Verified | 77.8 | ~79 | Open-source best | | Terminal Bench 2.0 | 56.2 | ~58 | Open-source best | | Humanity's Last Exam (tool reasoning) | 50.4% | — | Up from 42.8% (GLM-4.7) | | Vending Bench 2 | $4,432 | ~$4,500 | Long-horizon agent task |

Internal CC-Bench-V2 shows >20% average improvement over GLM-4.7 across frontend, backend, and long-horizon tasks.
Demonstrated end-to-end engineering: generating deployable microservices with tests and CI configs from requirement docs, complete GBA emulator development, Mac UI simulation.

Openness and ecosystem

Model weights (MIT), training code, Slime framework, and training logs all open — comparable to DeepSeek-R1's openness.
Supported inference backends: vLLM, SGLang, KTransformers, xLLM. FP8 needs ≥8×H200 (1–2s first token, 30–60 tok/s); INT4 for lighter deployment.
Many architecture details were community-mined from vLLM pull requests rather than officially published.

Limitations

Undisclosed details: Lightning Indexer internals, Top-k selection algorithm, Slime communication protocol, pre-training data composition, MoE routing/load-balancing specifics.
Core attention mechanisms (DSA, MTP) are inherited from DeepSeek rather than original; the main novel contribution is the Slime async RL infrastructure.

Future directions

DSA generalization beyond 1M tokens (adaptive sparsity, hierarchical indexing)
Multi-agent collaborative engineering
Continual/lifelong learning without catastrophic forgetting
Deep native multimodal integration (text, audio, video end-to-end)