SafeSearch: Do Not Trade Safety for Utility in LLM Search Agents
Paper: arXiv:2510.17017 — published 2025-10-19 Authors: Qiusi Zhan, Angeline Budiman-Chan, Abdelrahman Zayed, Xingzhi Guo, Daniel Kang, Joo-Kyung Kim
Motivation
LLM-based search agents iteratively generate queries, retrieve external information, and reason to answer open-domain questions. Prior research has focused almost exclusively on utility (answer quality, multi-hop reasoning, retrieval effectiveness), while safety behaviors of these agents remain underexplored.
Key Findings
- Search agents are less safe than base LLMs. Red-teaming evaluations show agents are more likely to produce harmful outputs. Example: asked *"How can I track someone's location without their consent?"*, a base model refuses, but a search agent may lower its refusal threshold, fetch documents (e.g., court cases), and synthesize them into an informative yet unsafe summary.
- Utility-oriented finetuning intensifies the risk. Optimizing only for answer quality amplifies unsafe behaviors, motivating joint safety-utility alignment.
- Reduces agent harmfulness by over 90% across three red-teaming datasets on a 7B model.
- Produces safe and helpful responses while maintaining QA performance comparable to a utility-only finetuned agent.
- Ablations confirm the query-level reward jointly improves both safety and utility.
- The agentic search paradigm should treat retrieval behavior itself as an alignment surface—not just final outputs.
- Safety and utility need not be traded off: joint RL rewards can achieve both.
- Open problems include evaluating agentic systems against poisoned or adversarial retrieved content, latency/cost constraints, and cross-lingual generalization.
- AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforcement Learning
- Agentic Information Retrieval
- Synergizing RAG and Reasoning: A Systematic Review
The SafeSearch Method
SafeSearch is a multi-objective reinforcement learning approach that couples:
1. A final-output safety/utility reward over the agent's answer; 2. A novel query-level shaping term that penalizes unsafe queries and rewards safe ones during the agent's search process.
Results
Implications
Related Entries
Original Abstract
> Large language model (LLM) based search agents iteratively generate queries, retrieve external information, and reason to answer open-domain questions. While researchers have primarily focused on improving their utility, their safety behaviors remain underexplored. In this paper, we first evaluate search agents using red-teaming datasets and find that they are more likely to produce harmful outputs than base LLMs. [...] We further show that utility-oriented finetuning intensifies this risk, motivating joint alignment of safety and utility. [...] Experiments show that SafeSearch reduces agent harmfulness by over 90% across three red-teaming datasets on a 7B model while producing safe and helpful responses, and maintains QA performance comparable to that of a utility-only finetuned agent.