Key points
This forum post reviews the paper "Cordyceps: Covert Control Attacks on LLMs via Data Poisoning" (arXiv:2605.26595, cs.CR, submitted May 26, 2026) by Zedian Shao, Charles Fleming, and Teodora Baluta (Georgia Institute of Technology / Cisco Systems).
The core idea
- Named after the parasitic Cordyceps fungus, the attack departs from traditional trigger-based backdoors (e.g., rare tokens like "BadMagic"). Instead, data poisoning teaches the model a semantic hiding scheme called SHuSh (Semantic Hiding under Shared knowledge).
- SHuSh works by adding a tiny, message-dependent perturbation to the model's semantic representation of an anchor topic. A perturbed stegotext is modeled as e(k) + e_A(m) + ε versus normal e(k) + ε. Smaller perturbations are stealthier (KL divergence = SNR/2), creating a sharp stealth-vs-decodability trade-off. Steganographic text maintains a Semantic Consistency Score above 0.8 against normal generations.
- Attackers control only ~10% of fine-tuning data by default. Poisoning data is generated in three steps: (1) an Oracle LLM produces stegotext pairing malicious messages with benign Wikipedia-style anchors; (2) chain-of-thought reasoning traces are generated in a separate stage (two-stage generation improves diversity and generalization); (3) paired encode/decode instruction samples train both directions.
- Unidirectional Covert Control (UCC): attacker plants stegotext in external data (web pages, emails, databases). On the OpenPromptInjection benchmark (7 NLP tasks × 7 injection tasks), UCC achieves ~40% higher attack success than direct prompt-injection baselines, and generalizes to unseen anchor topics.
- Bidirectional Covert Control (BCC): the model encodes sensitive outputs (e.g., PII, financial data) into innocuous-looking text, achieving 78%–93% secret data exfiltration accuracy.
- Tested defenses: ONION, CROW, SecAlign (backdoor defenses) — attack success remains at 93%, 90%, and 85% respectively. Prompt-injection defenses like DataSentinels and PromptLocate perform near random chance, since stegotext contains no directive language.
- The paper's theory shows a matched-filter detector is possible if the defender knows the message set and perturbation direction e_A(m); detection precision depends on the perturbation-energy-to-noise-variance ratio. Adversarial training with known perturbation directions and injecting counter-samples during fine-tuning are suggested but unvalidated paths.
- Effectiveness and stealth on much larger commercial models (tested only on 5 open-source models, 4B–30B).
- Whether anchor-free encoding is possible.
- Feasibility of training data provenance auditing as a defense.
- Whether the attack survives at 0.1% poisoning rates (1% already retains 90% effectiveness).
- Shao, Fleming & Baluta, "Cordyceps: Covert Control Attacks on LLMs via Data Poisoning", arXiv:2605.26595, 2026.
- Liu et al., "Formalizing and Benchmarking Prompt Injection Attacks and Defenses", USENIX Security 2024.
- Gu et al., "BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain", 2019.
- Motwani et al., "Secret Collusion among Generative AI Agents", NeurIPS 2024.
- Qi et al., "ONION: A Simple and Effective Defense Against Textual Backdoor Attacks", EMNLP 2021.
Two attack modes
Defense evaluation
The author's open questions
Why it matters
The post argues Cordyceps represents a new threat category: not making a model output something unsafe, but letting it execute a hidden semantic codebook *under the cover of safe-looking outputs* — a form of "capability theft" that current safety auditing, which focuses on visible outputs, does not address.