TeamCMU at Touché: Adversarial Co-Evolution for Advertisement Integration and Detection in Conversational Search
Overview
This forum post covers the TeamCMU lab report submitted to the Touché shared task on advertisement integration and detection in conversational search, published as arXiv:2507.00509 (July 2025).
- Authors: To Eun Kim, João Coelho, Gbemileke Onilude, Jai Singh
- Source: https://arxiv.org/abs/2507.00509
- Track: Touché — AdverTise in Conversational Search (Verticals track)
- Metrics: ranking quality (e.g., nDCG@10, MRR, Recall@k), task success rate, human preference, plus ad-specific measures of naturalness and detection accuracy.
- Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs.
- Ablations: contribution of retrieval depth, reranking, and training data scale.
- Applying Deep Learning to Ads Conversion Prediction in Last Mile Delivery
- Automated Query-Product Relevance Labeling using Large Language Models
- Better to Ask in English: Cross-Lingual Evaluation of Large Language Models
- Original paper: TeamCMU at Touché: Adversarial Co-Evolution for Advertisement Integration and Detection in Conversational Search, arXiv, July 2025.
Background and Motivation
As conversational search systems powered by large language models (LLMs) become mainstream, sponsored content is increasingly woven into dialogue-style answers. This raises a dual problem:
1. Ad integration: sponsored content must blend naturally into helpful, contextually relevant answers without degrading user experience. 2. Ad detection: systems (and users) need reliable ways to identify when a response has been influenced by advertising, preserving transparency and trust.
The paper tackles both sides jointly: ad generators and ad detectors improve against each other in an adversarial co-evolution loop, similar in spirit to adversarial training, where each side iteratively adapts to the other's latest behavior.
Approach in Context
Following the common pipeline for such systems:
1. Input and representation: queries, dialogue context, documents, and candidate ad content are encoded as dense/sparse representations or structured prompts. 2. Core modules: retrievers, rerankers, planners, generators, and feedback mechanisms chained into an end-to-end conversational search stack. 3. Learning strategy: supervised fine-tuning, contrastive learning, distillation, and/or reinforcement learning with process rewards. 4. Inference strategy: single- or multi-round retrieval, iterative retrieval, budget control, and early stopping.
The adversarial loop pushes the integrator to produce ads that are harder to detect while the detector sharpens its discrimination, aiming for a realistic equilibrium between monetization and detectability.
Evaluation Considerations
Typical evaluation in this task family involves:
> Note: Specific numeric results should be verified against the original PDF; this post is based on the abstract and public metadata.
Key Takeaways for Search / Rec / Personalization
1. Architecture: cascaded retrieval + rerank + generation remains dominant, but agentic paradigms increasingly treat *whether and how often to retrieve* as a learnable decision. 2. Data: high-quality instruction data and session logs are as important as models; synthetic data requires leakage and distribution-shift safeguards. 3. Evaluation: gaps between offline metrics and online satisfaction are widening; LLM-as-judge should be cross-validated with human assessment. 4. Product constraints: latency, cost, explainability, and safety policy — including ad transparency — are hard deployment constraints, not just benchmark numbers.
Open Questions and Limitations
Likely limitations include experiment scale bounded by compute budgets, benchmark-vs-real-user distribution mismatch, English-centric data limiting cross-lingual generalization, and safety risks of agentic systems on the open web. Future directions include more efficient test-time compute allocation, deeper integration with knowledge graphs and structured databases, and causal/fairness constraints for recommendation, as well as standards for advertisement disclosure in conversational UIs.
Glossary
| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Modeling search as sequential decisions and tool calls | | Gen-IR | Generative Information Retrieval |