On July 1, 2026, the China Academy of Information and Communications Technology (CAICT), jointly with the China Artificial Intelligence Industry Alliance (AIIA), officially released the Agentic AI Capability Assessment Standard 1.0 and launched a matching official evaluation platform. It is described as the world's first system-level Agentic AI evaluation framework issued by a national-level official research institution.
What the standard evaluates
Unlike prior LLM benchmarks, the evaluation target is not a single model but a complete agent system, assessed across six core capabilities:
- Task planning: decomposing high-level goals (e.g., "build me a quant backtesting system") into 5–15 executable subtasks, including dependencies, resource calls, and failure fallback strategies.
- Tool calling: accurate tool invocation, exception handling, and adapting next actions based on tool feedback.
- Long-term memory: maintaining context over 100+ conversation turns with traceable key decisions.
- Environment awareness: understanding real-world state changes (file creation, database writes, API status codes) and adjusting behavior accordingly.
- Security and compliance: proactively avoiding unauthorized operations, data leakage, and prompt injection attacks.
- Multi-agent collaboration: correct division of labor, conflict avoidance, and result merging in multi-agent systems.
- LLM vendors: Zhipu, Alibaba, Moonshot, ByteDance, DeepSeek and others will seek certification for their B-end products over the next 12 months, turning "can this model do work" from market self-assessment into official certification. Buyers can order by star rating.
- AI coding tools: Foreign tools (Cursor, Claude Code, GitHub Copilot) selling to government, SOE, and financial clients in China will likely need certification — and may struggle to reach 5 stars due to weak adaptation to local toolchains (DingTalk, Feishu, WeCom, Alibaba Cloud, Huawei Cloud), strengthening domestic tools (Tongyi Lingma, Trae, CodeGeeX, CodeArts).
- Embodied agents: Version 2.0 will add an embodied intelligence evaluation module, meaning humanoid robot vendors (Unitree, Zhiyuan, Robot Era) may be brought into the framework by end of 2026, potentially cleaning up the current self-reported-benchmark chaos.
- Static benchmarks vs. dynamic reality: A static 5-star rating may degrade to 3 stars in real scenarios with task complexity, environmental noise, and user variability — a fundamental limitation of all benchmarks.
- Policy vs. technical neutrality: CAICT's past standards (e.g., DSMM) have shown implicit preference for domestic over foreign solutions. No such bias is visible yet, but actual evaluation cases over the next 6–12 months should be watched.
- International alignment is unclear: Interoperability with ISO/IEC JTC1 SC42, IEEE P7000 series, and ETSI remains undefined. Without mutual recognition, Chinese AI products going overseas face duplicated certification and doubled compliance costs.
Each capability contains 3–5 sub-indicators with explicit scoring rubrics. Systems receive a 1–5 star rating: 5 stars means capable of autonomous operation in high-risk scenarios (finance, healthcare, government); 1 star means only simple single-turn, single-tool tasks.
Why it matters
The standard marks a watershed between "chat-capable" and "task-completing" models. Previous benchmarks — MMLU, HumanEval, SWE-bench, GPQA Diamond — measure model intelligence, not whether a model can finish real work. A model scoring 70% on SWE-bench may still fail to independently complete a full engineering task in production. CAICT's 1.0 shifts evaluation from *model intelligence* to *task completion*, echoing the July 2 Stanford/Apple paper questioning the marginal utility of multi-agent teamwork.
Why China issued it first
1. China's application layer moves fastest. China's ToB AI market reached RMB 450 billion in 2025 (~60% of the US market). In the US, over 60% of ToB AI spend goes to chatbots/content generation; in China, over 55% goes to agent execution/process automation — making system-level agent evaluation a harder requirement. 2. Standards-first regulation. The EU's AI Act Phase 2 implementing rules (June 2026) and NIST's AI Risk Management Framework (2025) focus on risk management. CAICT's standard directly defines capability levels, functioning as a de facto "Agentic AI product license."
Three layers of impact
Risks and caveats
Source: CAICT, "Agentic AI Capability Assessment Standard 1.0 Officially Released," 2026-07-01, https://www.caict.ac.cn/kxyj/qwfb/bps/202607/t20260701_647283.htm