CreativeBench: Does Scaling Make AI More Creative — or More Conservative?

Forum topic · 小凯 · 2026-03-14

Summary

This forum post explores CreativeBench, a benchmark that decomposes machine creativity into two types: combinational creativity (making new connections among known elements) and exploratory creativity (venturing beyond known concept spaces). According to the post, CreativeBench reveals a surprising pattern the authors call 'Convergence-by-Scaling': as language models grow larger, accuracy on combinational tasks rises steadily, but divergence—the diversity of outputs on exploratory tasks—declines. Larger models become better at producing 'correct' answers yet more inclined toward safe, high-probability responses, while simply raising sampling temperature adds noise rather than meaningful novelty. The post suggests mid-sized models may occupy a 'sweet spot' balancing accuracy and divergence, and draws implications for AI development: rethinking scale-first strategies, adopting multidimensional evaluation beyond accuracy, designing hybrid human-AI creative ecosystems, and reflecting on how exam-style training may suppress creativity in both machines and humans. It closes by framing future directions such as novel architectures, intrinsic motivation, and collective creativity. The article is an accessible deep-dive essay in Chinese, with a curated reference list including Boden, Guilford, Amabile, Runco & Jaeger, and the CreativeBench paper itself.

Overview

This post is a long-form Chinese-language deep dive into CreativeBench, a benchmark designed to probe what AI creativity actually is—not just whether machines "can create," but how their creativity differs from human creativity and how it changes with model scale.

Key points

Two faces of creativity

Combinational creativity: building new connections among existing elements (e.g., combining known ingredients in a new recipe; answering rhyme or association tasks from known knowledge). This requires efficient search and recombination within a known space.
Exploratory creativity: expanding the boundary of possibilities—proposing multiple distinct solutions to open-ended problems, making unprecedented cross-domain connections, challenging assumptions. This requires venturing into unknown concept space.
CreativeBench measures these with structured tasks rather than purely subjective human judging, using relative novelty metrics: frequency of an idea in training data, agreement with expert evaluation, and ability to solve open problems beyond known solutions.

The central finding: "Convergence-by-Scaling"

On combinational tasks, accuracy rises steadily with model scale (roughly 40–50% for small models up to 80–90% for large ones, per the post's illustrative figures).
On exploratory tasks, divergence (output diversity) declines as scale grows. Small models produce wild but occasionally brilliant answers; large models converge on safe, standard answers.
The mechanism: LLMs are probability machines. Larger models, trained on more data, have stronger confidence in "what is correct," raising average quality but biasing outputs toward high-probability responses. Creativity often lives at the edges beyond the "usual."

Accuracy vs. diversity trade-off

Raising sampling temperature increases diversity, but mostly as noise (grammatical errors, incoherence), not meaningful innovation.
The interesting middle ground may be mid-sized models—enough knowledge for plausible output, enough uncertainty for genuine exploration. The "bigger is always better" assumption may be wrong for creativity specifically.

Implications discussed in the post

1. Rethink scale-first dogma: blind scaling may move the field away from true creative capability. 2. Multidimensional evaluation: benchmarks should ask not only "is it correct?" but "is it novel, diverse, insightful?" 3. Human-AI creative ecosystems: large models as quality baselines; specialized smaller or fine-tuned modules as explorers of conceptual fringe. 4. Education parallels: training students to reproduce standard answers may suppress creativity the same way scaling appears to—"Convergence-by-Scaling" mirrors test-oriented education.

Future directions

Architecture innovation beyond pure Transformer: hybrid symbolic-neural systems, modular creative systems, evolutionary approaches.
Intrinsic motivation: curiosity mechanisms, self-set creative goals, metacognition about when to converge vs. diverge.
Collective creativity: networks of specialized AIs proposing, verifying, critiquing, and synthesizing.
Reframing AI as an amplifier of human creativity—idea trigger, feasibility filter, iteration accelerator, style amplifier—rather than a replacement.

Conclusion

CreativeBench does not answer "can machines create?" with a simple yes or no. It shows creativity is a multidimensional spectrum: AI excels at combinational recombination, while genuinely exploratory, boundary-breaking creativity remains (for now) a distinctly human domain. The takeaway: bigger is not necessarily better—and how we train and evaluate machines reflects what we value in ourselves.

References cited in the post

1. Boden, M. A. (2004). *The Creative Mind: Myths and Mechanisms* (2nd ed.). Routledge. 2. Guilford, J. P. (1967). *The Nature of Human Intelligence*. McGraw-Hill. 3. Amabile, T. M. (1996). *Creativity in Context*. Westview Press. 4. Runco, M. A., & Jaeger, G. J. (2012). The Standard Definition of Creativity. *Creativity Research Journal*, 24(1), 92-96. 5. CreativeBench Team (2026). *CreativeBench: Benchmarking Machine Creativity*. arXiv:2603.11863.