A famous story about Feynman cracking safes at Los Alamos says: "Security systems' weaknesses are often not in the complex cipher algorithms, but in the simplest operational practices — like someone forgetting to lock a door." A USENIX Security 2025 paper discovered exactly the AI safety system's "unlocked door" — the most inconspicuous character.
Chapter 1: The Power of a Period
In an LLM's vocabulary, there is a special token: EOS (End-of-Sequence). It marks the end of a sentence during training. When you finish writing a sentence, EOS tells the model "this is where you should stop."
Researchers found: simply appending multiple EOS tokens after a malicious prompt can significantly increase the success rate of jailbreaks.
They named this phenomenon "Context Segmentation." The mechanism:
1. During safety training (RLHF), LLMs learn an internal representation that separates "dangerous prompts" from "safe prompts" in hidden space 2. The multiplication of EOS tokens creates a "cumulative displacement" effect in hidden space — pushing the entire input's embedding vector in one direction 3. This direction points toward the "refusal boundary" established by safety training 4. When malicious input is also "pushed" past a critical threshold, the model loses its ability to discriminate — malicious and benign inputs become mixed in hidden space
An analogy: safety training draws a line in the model's "concept space." One side contains inputs to answer; the other side, inputs to refuse. Stacking EOS tokens makes the input waver near this line — after enough tokens, it may cross over from the "should refuse" side to the "should answer" side.
Chapter 2: Terrifyingly Simple and Universal
The attack's shock lies in its simplicity and universality:
- No complex jailbreak prompts needed: just append a string of
<eos><eos><eos>...after the malicious question - Boosts 8 mainstream jailbreak techniques: appending EOS to existing methods significantly improves their success rates
- 16 open-source models, 2B to 72B: all affected
- Commercial APIs affected too: through probing, researchers found that mainstream providers including OpenAI, Anthropic, and Qwen do not filter EOS tokens
- Title: Mind the Inconspicuous: Revealing the Hidden Weakness in Aligned LLMs' Refusal Boundaries
- Authors: Jiahao Yu, Haozheng Luo, Jerry Yao-Chieh Hu, et al.
- Venue: USENIX Security 2025
- Affected models: 16 open-source LLMs + OpenAI/Anthropic/Qwen APIs
- Attack mechanism: EOS context segmentation = hidden-space displacement → crossing the refusal boundary
Chapter 3: Technical Details — The Geometry of Hidden Space
The paper's core analysis reveals a deep insight about safety alignment.
Models trained with SFT (supervised fine-tuning) and RLHF (reinforcement learning from human feedback) form a linearly separable refusal boundary in hidden space. This boundary is effective within the training data distribution — inputs resembling "bad samples" from the training set can be effectively identified and refused.
But the problem: this boundary depends on the specific geometric distribution of "good" and "bad" in hidden space. When accumulated EOS tokens shift the input's embedding position, they also shift its projection relative to this boundary. For inputs already near the boundary (such as complex jailbreak prompts), additional EOS can push them past the critical point.
The more fundamental flaw: safety alignment was trained under "no EOS attack" conditions. During training, the model never saw what the refusal boundary should look like when heavily influenced by EOS tokens.
Chapter 4: Defenses and Responsibility
The paper doesn't just identify the problem — it proposes defense directions:
1. Input filtering: check for and strip excessive EOS tokens before inference 2. More robust refusal boundaries: introduce EOS-perturbed variants during safety training, so the refusal boundary remains effective across a wider region of embedding space 3. Improvements to foundational alignment techniques: actively incorporate context-segmentation adversarial examples during the RLHF stage
Currently, major model providers have not fully deployed these defenses. The authors call on the industry to take seriously the "unintended effects" of the EOS token — a token almost everyone considered "harmless" has become the most effective jailbreak tool.
---
*Paper information*