GLM-5: A New Era of Open-Source Agentic Engineering

This post is a deep-dive technical analysis of GLM-5, the latest large language model from Zhipu AI, based on a Chinese forum report.

Key Points

From "Vibe Coding" to "Agentic Engineering"

• GLM-5 represents a fundamental strategic shift: from lightweight, intuition-driven code generation ("Vibe Coding") to autonomous, multi-step Agentic Engineering.
• Capabilities emphasized: autonomous planning and multi-step execution, long-term memory retention, continuous learning/adaptation, and end-to-end coverage of the full software engineering lifecycle.
• Three technical drivers:
• Scaled pretraining: total parameters increased from 355B to 744B; pretraining data grew from 23T to 28.5T tokens.
• DeepSeek Sparse Attention (DSA): preserves long-context modeling while drastically cutting compute.
• Slime asynchronous RL framework plus new agentic RL algorithms, enabling learning from complex, long-horizon interactions.

Open-Source SOTA Positioning

• Ranks #4 globally and #1 among open-source models on Artificial Analysis; the first open model to surpass 50 on the composite intelligence index.
• Comparison with competitors:

| Dimension | GLM-5 | Competitors | Differentiator | |---|---|---|---| | Total parameters | 744B | DeepSeek-V3.2: 685B; MiniMax-M2.5: 230B | Largest open-source scale | | Activated parameters | 40B/44B | DeepSeek-V3.2: 37B; GPT-4o: ~80B (est.) | Sparse efficiency | | Context window | 202K | DeepSeek-V3.2: 128K; Claude-3.5: 200K | Long agent tasks | | License | MIT | Llama: tiered; Qwen: limited commercial | Fully open | | Training hardware | Huawei Ascend | Mostly NVIDIA GPUs | Domestic, self-controlled stack |

Core Specifications

• Total parameters: 744B (40B/44B activated)
• Pretraining data: 28.5T tokens
• Context window: 202K tokens (max output 128K)
• SWE-bench Verified: 77.8%

Architecture Innovations

• Mixture-of-Experts (MoE): 256 expert networks; each token routes to 8 experts (~5.9% sparsity). Layered design: first 3 layers use dense FFN for stability; the remaining 75 layers use MoE — 78 hidden layers in total.
• DeepSeek Sparse Attention (DSA): reduces self-attention complexity from O(L²) toward (near-)linear. For a 128K-token context, full attention would require ~8.2 billion attention pairs; DSA compresses effective computation to ~260 million pairs — a 97% compression ratio. Uses a Lightning Indexer with top-2048 sparse selection.
• Multi-Token Prediction (MTP) for decoding efficiency.

Training & Infrastructure

• Slime asynchronous RL framework with novel agentic RL algorithms.
• Trained entirely on Huawei Ascend chips with MindSpore, fully validating the domestic Chinese compute stack.

Open-Source Strategy

• Model weights released under the MIT license, permitting unrestricted commercial use, modification, and redistribution — a deliberate move tied to compute-sovereignty considerations.

Limitations & Future Directions

• The source report lists a section on research limitations and future directions, but the truncated content does not include the specific details.

*Note: figures above are as reported in the original forum analysis.*