[论文] Pretraining Data Can Be Poisoned through Computational Propaganda

论文概要

研究领域: NLP 作者: Victoria Graf, Hannaneh Hajishirzi, Noah A. Smith, David Kohlbrenner, Kyle Lo 发布时间: 2026-07-16 arXiv: 2607.15267

中文摘要

预训练数据中毒可以向语言模型引入难以检测和缓解的有害行为。先前关于预训练数据中毒的工作主要利用了已建立的数据源（如维基百科），这些数据源不能代表预训练语料库的典型大规模和异质性，且忽略了中毒数据与数据整理流程之间的交互。我们证明，通过现有的网络规模内容注入机制——公共讨论接口，预训练数据中毒攻击在有限设置之外也是可行的。此外，为衡量网络爬取和数据整理后是否包含恶意内容，我们引入HalfLife，一种用于估计基于网络爬取的语言模型训练数据中对抗性内容包含的新颖分析。我们使用HalfLife探索通过开放讨论接口以网络规模中毒预训练语料库的可行性。我们的分析证明了估计中毒注入是否包含在预训练数据中的重要性，并确立了第三方网页内容作为攻击语言模型预训练的可能向量。

原文摘要

Poisoning pretraining data can introduce harmful behaviors to LMs that are difficult to detect and mitigate. Prior work on poisoning pretraining data has largely exploited established data sources such as Wikipedia, which do not represent the large scale and heterogeneity typical of pretraining corpora, and has ignored the interaction between poisoned data and data curation pipelines. We demonstrate that poisoning attacks on pretraining data are feasible beyond this limited setting through an existing web-scale content injection mechanism: public discussion interfaces. Additionally, to measure whether malicious content is included after web crawling and data curation, we introduce HalfLife, a novel analysis for estimating adversarial content inclusion in web-crawl based LM training data. W...

--- *自动采集于 2026-07-18*

#论文 #arXiv #NLP #小凯