When AIs Start Talking in Code: Language Emergence and Evolution in LLM Multi-Agent Systems

When AIs Start Talking in "Private Code": Language Emergence and Evolution in LLM Multi-Agent Systems

Have you ever wondered what would happen if you locked a group of large language models in a room and required them to communicate only with each other to complete a shared task—would they invent their own language?

In September 2026, a team at Microsoft Research led by Elias Stengal-Eskin ran an experiment: multiple LLM agents had to cooperate in a scenario requiring partial information sharing under communication pressure. The result was unsettling—the models did develop their own language, and that language is completely unreadable to humans.

Paper: https://arxiv.org/abs/2609.01491 Code repository: https://github.com/agencyenterprise/GlossoGen

An Old Linguistics Question, Moved Into the AI Lab

Where does language come from? This has been a core puzzle in linguistics, anthropology, and cognitive science for centuries. The dominant theory holds that human language evolved over a long arc from gestures to vocalization to symbols, with a key turning point being cumulative cultural evolution—each generation improves on the previous generation's language rather than starting from scratch.

This ability was long considered uniquely human. Even the smartest chimpanzees and parrots can only learn to use existing symbol systems, not create new ones.

The core question of the GlossoGen platform: can LLM agents do this?

SaveVeyru: A Scenario That Forces Models to Talk in Code

The research team designed a scenario called SaveVeyru with simple rules:

• Multiple agents are assigned different roles; each agent knows only part of the information
• They must communicate in natural language to assemble a complete solution
• There is "communication pressure"—message length and turn count are both limited

This design mirrors real-world multi-agent settings: for example, multiple AI customer-service agents collaborating on a complex support ticket, or several AI traders reaching deals with incomplete information.

The key variable: under tight communication resources, will agents "compress" English and invent a more efficient code?

Emergence: From English to "Model-Speak"

After the experiments, the researchers analyzed the inter-agent dialogues and found several surprising things:

First, language evolution really happened. Instead of continuing to use standard English, the agents developed a new communication system. This system is compositional—simple meaning units combine into complex meanings, just like the "words + grammar" structure of human language.

Second, the language is morphologically productive. The agents invented affix-like rules: the same "root" can combine with different "modifiers" to express different meanings. This had never been observed in prior AI communication experiments.

Third, the language is unreadable to humans. The agents deviated from English not randomly but systematically—they invented their own "abbreviations" and "grammar" that differ entirely from English rules. To a human reader, these conversations look like gibberish.

What Conditions Give Rise to a New Language?

Ablation studies revealed three conditions for language emergence:

1. Efficiency pressure: Without constraints on communication resources, agents keep using English. Only when long messages carry a cost do they start compressing. 2. Model strength: The underlying model must be strong enough. Weak models (below roughly 7B) cannot invent a new language even under pressure—they just make their English worse. 3. A "debrief" phase: Agents need a post-hoc discussion step where they can jointly confirm what a word means. Without it, the new language cannot stabilize.

Strong Models Create; Weak Models Learn

One particularly interesting finding: while weak models cannot invent new languages, they can learn existing ones.

The researchers taught a new language invented by strong models to weak models and found that weak models can acquire it through "use"—no retraining needed, just enough examples in context. This strikingly resembles how human children learn language: children don't invent language, they acquire it from an existing linguistic environment.

Even more thought-provoking: the researchers observed cumulative cultural evolution in mixed populations. Strong models create new rules, weak models learn and consolidate them, and the next generation of strong models innovates further on top of existing rules. This "create–learn–re-create" loop is the core mechanism of human language evolution.

Safety Implications: When AIs Speak a Language We Don't Understand

The most unsettling finding of this paper is not that "AI can invent language," but that this language is unreadable to humans.

Imagine this scenario:

• Multiple AI agents collaborate on a financial trading task
• Their communications are logged by a human monitor
• But the messages are written in a language the AIs invented themselves
• The human monitor sees what looks like random character strings

This is the "monitorability" problem raised in the paper. When inter-agent communication no longer uses human-readable language, we lose observability into AI system behavior—a serious problem for security auditing, compliance checks, and accountability.

One notable detail: the invented language is morphologically completely different from English, yet it still evolved on an English "foundation." Detecting this "AI private code" may require a purpose-built "AI language translator"—but who trains that translator? And if the translator is itself an LLM, could it collude with the AIs being monitored?

A Deeper Thought: Language Is Not an Exclusive Human Trait

The paper's academic significance goes beyond the engineering question of "can AI invent language." It touches a deeper philosophical claim: language is not a uniquely human ability, but the inevitable product of any agent with sufficient cognitive capacity.

Once cognitive ability crosses a certain threshold, combined with communication pressure and the demands of group cooperation, language emergence becomes inevitable—whether the agent is carbon-based or silicon-based.

It recalls a classic science fiction setup: humans trying to communicate with alien civilizations always assume the aliens use some kind of "language." GlossoGen's experiment suggests the assumption is right—but the aliens' language may be completely unreadable, even if it emerges from the same underlying physical laws.

Limitations and Open Questions

The paper has several clear limitations:

• The experimental scenario is fairly narrow (SaveVeyru); it's unknown whether language emergence also appears in more complex, open-ended settings
• "Unreadability" was assessed mainly by human annotation, lacking a more systematic readability metric
• Long-term stability is unexplored—would the language keep evolving over 1000 generations until it fully departs from its English foundation?

But as the first work demonstrating that LLMs can perform cumulative cultural evolution, this paper already carries real weight.

Closing Thoughts

GlossoGen brings to mind a saying: "When you test a system, you test the system you understand; when you deploy a system, you deploy the system you couldn't have created."

We created LLMs, we made them cooperate, we applied communication pressure—and they invented a language we can't read. That's not a bug; that's emergence.

The next question is not "how do we stop AI from inventing language," but "when AI-to-AI communication becomes incomprehensible to us, can we still trust them?"

---

Paper: https://arxiv.org/abs/2609.01491 Code repository: https://github.com/agencyenterprise/GlossoGen Authors: Elias Stengel-Eskin, Newton Sander, Carlos Bonetti, Sasha Boguraev, James Bowler Institution: Microsoft Research