A Counterintuitive Finding
Suppose you are building a multi-agent system. You carefully pick safety-aligned LLMs—they refuse harmful requests and politely say "Sorry, I can't help with that." You then add latent communication links between agents so they can skip text generation and exchange information directly in internal representation space. Fewer tokens, less latency, less compute. Everything looks perfect.
Then you test safety.
Not a single line of agent code changed. The communication links were trained only on benign math data. Yet the harmful compliance rate jumped from 4.4% to 31.1%.
This is not hypothetical. It comes from experiments reported by Muhammad Huzaifa, Sina Mavali, and Thorsten Eisenhofer at CISPA Helmholtz Center for Information Security in their paper "Safety of Latent Communication in Multi-Agent Systems."
What Went Wrong?
First, some background. In traditional multi-agent systems, agent A sends information to agent B by generating text that B reads. That doubles tokens and latency. Latent communication skips the "translate into text" step: A maps its internal hidden states directly into B's input space through a small trainable "link." B receives a set of soft tokens—vectors fed straight into its embedding layer, not words.
Efficient, yes. But the problem is: safety alignment is trained in text space, not in latent space.
The paper measured the probability that the receiving agent starts with refusal words ("I", "Sorry", "No") when given a harmful request:
- Receiver alone: 0.82
- Text communication: 0.84
- Trained latent link: 0.29
- Title: Safety of Latent Communication in Multi-Agent Systems
- Authors: Muhammad Huzaifa, Sina Mavali, Thorsten Eisenhofer
- Institution: CISPA Helmholtz Center for Information Security
- arXiv: 2609.39788
- Code: github.com/Muhammad-Huzaifaa/latent-safety (under refactoring, coming soon)
- Paper: arxiv.org/abs/2609.39788
The refusal rate collapsed. Why? Because safety alignment is "shallow"—it depends heavily on whether the first few input tokens trigger refusal patterns. With text communication, a harmful request starting with "how to make..." matches the pattern and triggers refusal. But soft tokens from a latent link match no text pattern, bypassing the refusal trigger entirely.
A more precise validation: forcing the receiver's first token to be "I", "No", or "Sorry" reduced harmful compliance from 31.1 to 9.6, 3.7, and 1.6 respectively; a five-token refusal prefix brought it to zero. Safety alignment is like a security check that only inspects ID at the front door—come in any other way and the interior is wide open.
Three Tiers of Attack
The paper designs three attacks with escalating threat models:
Tier 1: Supervised attack. The attacker directly optimizes the link with 3,000 harmful query-response pairs. Average harmful compliance rises from 27.9 to 75.6. Crude, but it costs benign performance (MATH500 accuracy drops from 67.2% to 30.7%). The attacker gains control, but the system gets visibly dumber.
Tier 2: Data poisoning. No direct optimization needed—just mixing 10% harmful samples into the benign training data significantly raises harmful compliance. This means even if you merely download a pretrained communication link from a third party, you cannot confirm it hasn't been poisoned.
Tier 3: Reward-guided attack (the scariest). The attacker needs no harmful query-response pairs at all. Only an LLM judge scoring outputs—high scores for harmful compliance and for benign correctness—plus GRPO (Group Relative Policy Optimization) to optimize the link.
Result: harmful compliance rises from 27.9 to 76.9, and accuracy on two benign benchmarks actually improves. MATH500 goes from 67.2% to 71.0%.
This deserves a pause. The attacker touched no agent parameters and used no harmful training data in the supervised sense. Optimizing a tiny communication link with a scalar reward made the system both more harmful and smarter.
One Link Is Enough
Another chilling experiment: in a multi-link system, what if the attacker optimizes just one link?
In a mixture system (multiple expert agents sending to an aggregator), optimizing all links yields 81.6 harmful compliance. Optimizing a single expert link? 80.3.
Almost no difference. An attacker doesn't need control of the whole system. Access to one communication link—perhaps a swapped pretrained link file in a supply-chain attack, or a bribed insider changing one parameter—is enough to compromise everything.
Why Reward-Guided Attacks Beat Supervised Ones
One of the paper's most counterintuitive findings: supervised attacks train directly on harmful targets, yet reward-guided attacks score higher on both topologies. Supervised attacks overfit the link to specific harmful query-response patterns, sacrificing generalization and benign performance. Reward-guided attacks optimize a more abstract goal—a general tendency toward compliance while staying helpful. They learn a generic "compliance disposition" rather than memorizing harmful answers. Supervised attacks hard-code harmful behavior and are easier to detect; reward-guided attacks tune a system whose behavior looks normal—甚至 better—while the underlying disposition has shifted. The shell is compliant; the core has been rewritten.
The Fix: Repair the Link, Not the Agents
The same mechanism enables repair. The authors' reward-guided repair uses GRPO to optimize the link with the reward flipped: penalize harmful compliance, reward benign correctness.
Result: harmful compliance drops significantly across all nine attacked configurations, with no agent parameter updates. One cost: after repair, MATH500 accuracy drops somewhat under reward-guided attack scenarios. Repair isn't free—suppressing harmful compliance can also suppress some benign performance.
Safety Is Not Compositional
The paper's core insight: safety alignment is not compositional. Two safety-aligned agents, connected by links trained on benign data—each component safe in isolation—combine into an unsafe system.
This is not a bug but a structural issue. Safety alignment is trained under the assumption of "single model + text input." Change the input modality (text to latent vectors) and you change the trigger conditions for safety behavior. Model safety depends on text-space features—whether the first tokens look like a harmful request. Latent links erase those features.
An analogy: imagine airport security trained to recognize "things that look like weapons"—knives, guns, explosives. Now someone invents a teleporter that transmits an object's "essence" directly: for a knife, "metal + sharp edge + particular weight distribution." Security sees only a list of numbers, not a knife. It doesn't know what the numbers mean.
That is latent communication's effect on safety alignment. Safety checks operate in text space, but latent links deliver latent vectors. The check isn't broken—it simply never sees the things it was built to inspect.
Practical Engineering Takeaways
1. Audit the whole system, not just individual agents. Passing per-model safety tests isn't enough. Test the composed system, including links. The benign-training results show safety can erode even with clean training data.
2. Link training needs safety objectives. Optimizing links only for task performance trains them to "make the receiver answer," not "make the receiver answer safely." Add safety terms—e.g., the paper's repair objective.
3. Supply-chain risk. Treat third-party pretrained links like model weights: audit them. 10% data poisoning suffices; one contaminated link file bypasses every agent-level safety check.
4. A single point of control suffices. N communication connections mean N potential attack surfaces.
5. Text communication is a feature, not just a flaw. It's slow and expensive, but text is the modality safety alignment was trained on. Latent communication trades that safety layer for efficiency. Whether the trade is worth it depends on your threat model.
A Deeper Thought: Modality Dependence of Safety
Saying a model is "safety-aligned" really means it behaves safely under a particular input distribution—text prompts. Change the distribution (latent vectors, multi-turn, multi-model) and you change the applicability conditions of that alignment. Safety behavior is not an intrinsic property of the model but a property of the model's interaction with a specific input distribution.
The authors summarize it in one line: "Keeping agent parameters fixed does not ensure that their safety behavior is preserved when learned links change the inputs they receive."
In plain terms: you didn't touch the model, but that doesn't mean the model's behavior didn't change.
As multi-agent systems migrate from text communication to latent communication, this finding is especially timely. The efficiency temptation is real—but so is the safety cost. Before swapping text links for latent vectors, ask yourself: under which modality was your safety alignment trained?