An Awkward Scenario
Imagine you are an evolutionary algorithm using an LLM as the mutation operator, trying to solve a scientific optimization problem — say, designing more efficient GPU kernels or denoising RNA sequencing data. You have the LLM generate candidate solutions, the evaluator scores them, and high-scoring solutions move to the next round.
But you quickly get stuck. The LLM's knowledge has limits: it doesn't know about tricks just published in a specific field, or that a library's API was renamed last year. You add a web search tool, and it ends up searching the same keyword every time, returning the same set of pages — because your solutions are evolving, but your search queries are not.
This is the problem EvoDuet addresses: search and solving are two objectives that need to co-evolve, not a subordinate feature of one tool.
The Bi-Level Loop: Outer Evolves Solutions, Inner Evolves Queries
EvoDuet's core architecture is a bi-level optimization framework:
Outer loop (Solution Optimization): the classic evolutionary search flow — select parent solutions, LLM generates offspring, evaluator scores them, update the population. Same as frameworks like OpenEvolve.
Inner loop (Query Optimization): EvoDuet's innovation. Instead of simply "doing a search," it treats the search query itself as an object that needs optimization. The inner loop constructs queries based on the current state of the solution population, searches the web, scores returned documents, and keeps useful ones. A document's "usefulness" is defined by the outer loop's solution performance — if a document helps produce a high-scoring solution, it's a good document.
The bridge connecting the two loops: the Knowledge-Gap-based Retrieval Gate. This is the most critical design. At the start of each outer iteration, the LLM first assesses itself: "Is my current knowledge sufficient to generate a better solution?" It has three choices:
1. retrieve: knowledge is insufficient, search for new documents 2. look-up: knowledge is sufficient, and previously retrieved documents haven't been fully used — reuse them 3. no-op: knowledge is sufficient, no external information needed, generate directly
This gating mechanism avoids two extremes: searching every iteration (wasting API calls and retrieving duplicate content) and never searching (missing new knowledge).
Why "Just Add Search" Doesn't Work
The paper ran a key experiment: attaching web search directly to the outer loop (searching every iteration) actually performed worse. The reason is a mismatch between search results and solution state.
Think of cooking. In round one you search "how to make scrambled eggs with tomatoes" and get a batch of recipes. In round two you've improved your method and want to search "how to remove gaminess from scrambled eggs with tomatoes" — but if the query doesn't evolve with the solution, you keep searching the original phrase and keep getting the same recipes.
EvoDuet's inner loop solves this. It constructs query descriptors from the current population state, letting search terms evolve alongside solutions. The paper describes this as "search placement changes how evidence enters the solution loop" — how search results enter the solution loop determines whether they are useful.
The Numbers
Tested on 21 optimization tasks:
- GPT-5.6-Luna: OpenEvolve's normalized discovery gain improved from 74.1% to 78.0%
- Gemini-3.8-Flash: from 61.3% to 82.3% (+21 percentage points)
- Qwen3.5-9B: no improvement — showing that model capability itself is a prerequisite
Behavioral Analysis: Search Isn't Searching, It's Knowledge Management
The paper's behavioral analysis reveals several counterintuitive findings:
1. Multilingual queries are rare, appearing only in competition-style tasks. Models default to English search, using other languages only for specific tasks like Rosetta.
2. Source preferences vary by task domain. Different tasks need different document types — algorithm tasks search papers, engineering tasks search documentation.
3. Models balance retrieval and reuse differently. Some models tend to search for new documents frequently; others lean toward reusing existing ones.
4. Late-stage searches can reuse early documents. Search isn't linear — a document retrieved in round 5 may be found again in round 20, having gone unused the first time.
5. Stored documents are used continuously across iterations. This validates the look-up mechanism: one search's output can be reused many times.
Failure Case Analysis: Not All Search Helps
The paper honestly analyzes search failures. A typical case is the Galileo task: the model found a promising-looking document, but the offspring solution generated from it was worse than the parent. This shows that retrieval quality ≠ solution quality — even with relevant information retrieved, the LLM may misuse it.
Another failure mode is "evidence-inspired over-reach": after finding relevant documents, the model overconfidently extends the methods beyond their applicable scope.
My Take: Search as a First-Class Citizen
EvoDuet points to a broader trend: search is shifting from "tool call" to "first-class citizen."
In early agent frameworks, search was just a function call — invoke it when information is needed. But EvoDuet elevates search into an independently optimized objective, on equal footing with solution optimization. This mirrors Hindsight (memory as learning, not storage) and treg (a tool-API routing layer): upgrade what was previously considered an "auxiliary function" into an independent optimization target.
Another notable point is knowledge-gap assessment. The retrieval gate lets the LLM judge "do I know this or not," which is more efficient than fixed-frequency search. This matches human researcher behavior — you don't check the literature after every paragraph you write; you check when you "feel the information gap." EvoDuet formalizes this intuition.
That said, the Qwen3.5-9B null result deserves reflection. Bi-level optimization requires the model to have some self-assessment ability — it must judge "what knowledge am I missing." If the model isn't capable enough, the retrieval gate's judgments become unreliable and the whole mechanism fails. This may be a capability threshold: co-evolution needs two sufficiently smart "brains," otherwise the inner loop is just noise.
---
Paper link: https://arxiv.org/abs/2609.40340
HTML full text: https://arxiv.org/html/2609.40340v1