TeamCMU at Touché: Adversarial Co-Evolution for Advertisement Integration and Detection in Conversational Search
Paper: arXiv:2507.00509 — July 2025 Authors: To Eun Kim, João Coelho, Gbemileke Onilude, Jai Singh (CMU) Venue: Touché shared task (advertisement integration and detection in conversational search)
> Note: The original forum post is largely a metadata stub built from the paper's abstract; the substantive summary below follows the listed framework. Quantitative results should be verified against the official PDF.
Background and Motivation
Conversational search systems powered by LLMs increasingly handle retrieval, ranking, and generation end to end. This creates a new surface for native advertising: sponsored content can be woven directly into generated conversational answers, and conversely, systems must detect when such integration occurs. The paper addresses both sides — ad integration and ad detection — through adversarial co-evolution.
Core Contributions
- A unified framing of advertisement integration and detection as an adversarial co-evolution problem, where generation and detection components iteratively improve against each other.
- A decomposition of the pipeline (representation, retriever, reranker/generator, detection/feedback mechanisms) suitable for engineering reproducibility.
- Participation and evaluation within the Touché shared task, providing a reproducible benchmark setup for follow-up work.
- Discussion of interfaces with adjacent paradigms: LLM tool calling, reinforcement learning, and multi-agent collaboration.
- Datasets: shared-task corpora (Touché) plus standard IR benchmarks.
- Metrics: nDCG@10, MRR, Recall@k, task success rate, human preference, latency and token cost.
- Baselines: BM25, dense retrieval, cross-encoder rerankers, retrieval-free LLMs.
- Ablations: contribution of retrieval steps, reranking depth, and training data scale.
- Applying Deep Learning to Ads Conversion Prediction in Last Mile Delivery
- Automated Query-Product Relevance Labeling using LLMs
- Behavior Modeling Space Reconstruction for E-Commerce Search
- Source: https://arxiv.org/abs/2507.00509
Method Sketch
Following the typical four-step structure (problem formalization → system design → training/construction → inference pipeline):
1. Input and representation: queries, documents, and conversational context encoded into dense/sparse representations or structured prompts. 2. Core modules: retriever, generator, and ad-detection components chained or run in parallel. 3. Learning strategy: supervised fine-tuning, contrastive learning, distillation, or bootstrapped data synthesis (per the standard recipe for this problem class). 4. Inference: iterative retrieval with budget control and early stopping.
Evaluation Context
Typical evaluation for this line of work uses:
Specific numeric results are not reproduced here; consult the paper tables.
Takeaways for Search / Ads / Personalization
1. Architecture: cascade retrieval + rerank + generation remains dominant, but agentic approaches make retrieval strategy itself learnable. 2. Data: high-quality instruction data and session logs matter as much as model design; synthetic data risks leakage and distribution shift. 3. Evaluation: offline metrics diverge from online satisfaction; LLM-as-judge should be cross-validated with human evaluation. 4. Production: latency, cost, explainability, and ad-disclosure safety are hard deployment constraints, not just benchmark numbers.
Limitations and Open Problems
Experimental scale may be bounded by compute budgets; benchmarks may not match real user distributions; cross-lingual generalization is unclear; adversarial ad content in open-web retrieval raises poisoning and safety risks. Future directions include better test-time compute allocation, deeper integration with structured knowledge, and fairness-aware ad placement.