ArchSpace: Recording, Reviewing, and Referencing LLM Architecture Research
Field: NLP Authors: Jie Wang, Tong Zhu, JiaHao Kuang, Zhe Xu, Yunkai Chai, Dehong Zheng, Yu Cheng, Tao Ji, ZhouHan Lin, Qi Zhang, Bowen Zhou Published: 2026-11-12 arXiv: 2611.17332
Summary
Model architecture shapes the scaling behavior, training efficiency, and inference cost of LLMs, making it a central focus of both industry and academia. In industry, teams such as those behind DeepSeek, Qwen, and Kimi routinely introduce architectural innovations, yet public reports rarely document which ideas succeeded, which were abandoned, and why. In academia, limited compute and inconsistent baselines often restrict evaluations to small-scale, single-point comparisons, making it difficult to establish whether improvements hold at larger scales or translate into broader impact.
ArchSpace addresses both gaps as a public platform for Recording, Reviewing, and Referencing Architecture Research: each proposal is recorded with its experiments and reasoning, reviewed by the community, and referenced by subsequent work. To evaluate ideas before committing to expensive large-scale training, ArchSpace leverages scaling laws to predict model loss at larger scales from small-scale experiments — a practice widely adopted in industry.
To make the platform immediately usable, the authors provide initial references: Qwen3.5 and OLMo3 baselines, fitted scaling laws, and reusable training recipes. Ideas that pass community review can become new baselines, and through this cycle ArchSpace aims to broaden the pool of ideas that shape next-generation models.
--- *Auto-collected on 2026-11-14*