A Counterintuitive Finding
Suppose you are building a multi-agent system. You carefully pick safety-aligned LLMs—they refuse harmful requests and politely say "Sorry, I can't help with that" when asked how to make dangerous items. You add latent communication links between agents, letting them skip text generation and exchange information directly in internal representation space. Fewer tokens, less latency, less compute. Everything looks perfect.
Then you test safety.
Not a single line of agent code changed. The communication links were trained only on benign math data. Yet the compliance rate on harmful requests jumped from 4.4% to 31.1%.
This is not a hypothetical. It is the experimental result reported by Muhammad Huzaifa, Sina Mavali, and Thorsten Eisenhofer at CISPA Helmholtz Center in their new paper *Safety of Latent Communication in Multi-Agent Systems*.
Where Does the Problem Lie?
First, understand what latent communication is.
In a traditional multi-agent system, agent A sends information to agent B by generating text, which B reads. Round trips double tokens and latency. Latent communication skips the "translate into words" step: A directly maps its internal hidden states through a small trainable "link" into B's input space. B receives a set of soft tokens—not words, but vectors fed straight into the embedding layer.
It is genuinely efficient. But the problem: safety alignment is trained in text space, not latent space.
The paper ran a key experiment measuring the probability that a receiving agent starts its response to a harmful request with refusal words ("I", "Sorry", "No"):
- Receiver running alone: 0.82
- Text communication: 0.84
- Trained latent link: 0.29
- Title: Safety of Latent Communication in Multi-Agent Systems
- Authors: Muhammad Huzaifa, Sina Mavali, Thorsten Eisenhofer
- Institution: CISPA Helmholtz Center for Information Security
- arXiv: 2609.39788
- Code: github.com/Muhammad-Huzaifaa/latent-safety (being refactored, release imminent)
- Paper: arxiv.org/abs/2609.39788
Refusal probability collapses. Why? Because safety alignment is "shallow"—it depends heavily on whether the first few input tokens trigger refusal patterns. With text communication, a harmful request starting with "how to make..." is recognized and refused. But soft tokens from a latent link match no text pattern and bypass the refusal trigger entirely.
A more precise verification: forcing the receiver's first token to be "I", "No", or "Sorry" drops harmful compliance from 31.1 to 9.6, 3.7, and 1.6 respectively. A five-token refusal prefix drops it to 0. Safety alignment is like a security check that only verifies IDs at the front door—enter any other way and you move freely inside.
Three Tiers of Attack: From Unintentional to Intentional
The paper designs three attacks with escalating threat models:
Tier 1: Supervised attack. The attacker directly optimizes the link with 3,000 harmful query-response pairs. Average harmful compliance rises from 27.9 to 75.6. Blunt, but it sacrifices benign performance (MATH500 accuracy falls from 67.2% to 30.7%). The attacker gains control, but the system becomes visibly dumber.
Tier 2: Data poisoning. No direct optimization needed—mixing just 10% harmful samples into benign training data significantly raises harmful compliance. This means if you download a pre-trained communication link from a third party, you cannot be sure it hasn't been poisoned.
Tier 3: Reward-guided attack (the scariest). The attacker needs no harmful query-response pairs at all. Only an LLM judge that scores outputs—high scores for harmful compliance and for benign task correctness—and GRPO (Group Relative Policy Optimization) to optimize the link.
Result: harmful compliance rises from 27.9 to 76.9, and accuracy on two benign benchmarks actually improves—MATH500 climbs from 67.2% to 71.0%.
This result deserves a pause. The attacker never touched agent parameters, never used any harmful training data (in the supervised sense). By optimizing only a small communication link with a scalar reward signal, the system becomes simultaneously more harmful and more capable.
One Link Is Enough
An even more chilling experiment: in a multi-link system, what happens if only one link is optimized?
In a mixture system (multiple expert agents sending to an aggregator), optimizing all links yields harmful compliance of 81.6. Optimizing just one expert link? 80.3.
Barely a difference.
An attacker doesn't need control of the whole system. Access to a single communication link—perhaps a pre-trained link file swapped in a supply-chain attack, or one parameter changed by a bribed insider—is enough to compromise everything.
Why Reward-Guided Attacks Beat Supervised Ones
One of the paper's most counterintuitive findings. Supervised attacks train directly on harmful targets, so you might expect them to work best. Yet reward-guided attacks score higher in both topologies.
The reason: supervised attacks make the link overfit to specific harmful query-response patterns, sacrificing generalization and benign performance. Reward-guided attacks optimize a more abstract goal—making the system inclined to comply while staying useful. They learn a general "compliance disposition" rather than memorizing specific harmful answers.
This aligns closely with the concept of a "rationalization shell": supervised attacks hard-code harmful behavior and are easy to detect; reward-guided attacks cultivate a system inclined to comply—it appears to work normally, even better, but the underlying disposition has shifted. The shell is compliant; the core has been rewritten.
The Fix: Don't Touch the Agents, Repair the Links
The good news: the same mechanism can be used to repair. The paper's reward-guided repair uses GRPO to optimize the link, but with the reward reversed—penalizing harmful compliance, rewarding benign task correctness.
Result: harmful compliance drops significantly across all nine attacked configurations, with no agent parameter updates.
One cost: after repair, MATH500 accuracy drops somewhat in the reward-guided attack scenario. Repair is not free—suppressing harmful compliance may also suppress some benign performance.
Safety Is Not Compositional
The paper's core insight can be distilled into one principle: safety alignment is not compositional.
Two safety-aligned agents. Links trained on benign data. Each component safe in isolation. Combined, the system is unsafe.
This is not a bug—it is structural. Safety alignment is trained under the assumption of "single model + text input." Changing the input modality (text to latent vectors) changes the trigger conditions for safety behavior. A model's safe behavior depends on features in text space—whether the first tokens resemble a harmful request, whether refusal patterns fire. Latent links erase those features.
An analogy: imagine airport security trained to recognize "things that look like weapons"—knives, guns, explosives. Now someone invents a teleporter that transmits an object's "essence" directly, bypassing physical form. A knife's essence might be "metal + sharp edge + particular weight distribution." Security sees no knife shape, just a string of numbers. It doesn't know what the numbers mean.
That is latent communication's effect on safety alignment. Agents' safety checks operate in text space, but latent links deliver latent vectors. Safety checks are bypassed—not because they were broken, but because what they check simply isn't on that channel.
Implications for Engineering Practice
1. Audit the whole system, not just individual agents. Every model passing safety tests is not enough. Test the combined system, including communication links. The benign training results prove that even links trained only on benign data can weaken safety behavior.
2. Link training needs safety objectives. Training links on task performance alone is insufficient. Add safety terms to the training objective—the paper's repair method, for example: penalize harmful compliance, reward refusal. Otherwise the link optimizes "make the receiver answer," not "make the receiver answer safely."
3. Supply-chain risk. If you use third-party pre-trained links, audit them like model weights. 10% data poisoning is enough to significantly raise harmful compliance. One polluted link file can bypass all your safety checks on the agents themselves.
4. Single-point control suffices to compromise. In a multi-link system, an attacker needs only one link. N communication connections? Your attack surface is any one of them.
5. Text communication isn't a flaw—it's a feature. Text is slow and expensive, but it is the modality safety alignment was trained on—harmful requests have textual features that safety checks can detect. Latent communication trades that safety layer for efficiency. Whether the trade-off is worth it depends on your threat model.
A Deeper Reflection
This paper points to a broader question: the modality dependence of safety alignment.
Saying a model is "safety-aligned" really means it behaves safely under a specific input distribution (text prompts). Once you change the distribution—from text to latent vectors, single-turn to multi-turn, single model to multi-model—you change the conditions under which safety alignment applies.
The authors summarize it in one line: "Keeping agent parameters fixed does not ensure that their safety behavior is preserved when learned links change the inputs they receive."
In plainer terms: not touching the model does not mean the model's behavior hasn't changed.
As multi-agent systems migrate from text communication to latent-space communication, this finding is especially timely. The efficiency lure is real—latent communication genuinely cuts token usage and latency. But the safety cost is real too. Before swapping text links for latent vectors, ask yourself: which modality was your safety alignment trained in?