Imagine a brilliant but impatient writer: give him a complex task — planning a trip under dozens of constraints, scheduling a day of meetings, or hiding a numeric code in a playful Shel Silverstein-style poem — and he'll blurt out a first answer, then get stuck. The problem isn't a lack of intelligence; it's the very human instinct to "write something roughly right first." In these tasks, "roughly right" usually means "wrong."
Kuang-Huei Lee et al. (Google DeepMind, 2025) address this in "Evolving Deeper LLM Thinking" (arXiv:2501.09891v1) with a very biological add-on: let answers evolve like species. They call it Mind Evolution.
From "thinking" to "thinking deeper": how should inference-time compute be spent?
Prior inference-time strategies include:
- Chain-of-Thought: writing out reasoning steps
- Self-consistency: sampling multiple reasoning chains and voting
- Sequential revision / Reflexion: iteratively revising drafts, with or without feedback
- Best-of-N: generating N complete answers and picking the best
- Tree search / verifier-guided search
- vs. Best-of-N: Best-of-N is wide but shallow — candidates are independent. Mind Evolution is wide *and* deep, repeatedly polishing promising candidates.
- vs. tree search / stepwise reasoning: those usually need process-level rewards. Mind Evolution performs global revision of whole solutions, requiring only a global solution evaluator.
- Critic: reads the candidate solution plus evaluator feedback and analyzes what's wrong and how to fix it
- Author: produces a new complete solution based on the Critic's advice
- Max generations: \(N_{gens}=10\); islands: \(N_{island}=4\); conversations per island: \(N_{convs}=5\); turns per conversation: \(N_{seq}=4\)
- \(10 \times 4 \times 5 \times 4 = 800\) candidates max — a budget matched to Best-of-N for fair comparison
- Reset interval 3 (reset 2 islands each time); 5 emigrants; up to 5 parents per recombination; 1/6 probability of no parents (preserving random exploration); 5 retries per turn
The common theme: trade more inference-time compute for higher success rates. But in many tasks the hard part isn't "thinking more steps" — constraints live in natural language, per-step scoring may be unknown, yet you can often write a program that checks whether the final answer is valid. Mind Evolution exploits exactly this: evaluation is easier than generation, echoing the classic computational-theory insight that verification is often easier than solving (as with many NP problems).
Mind Evolution: genetic search over natural-language solutions
> Mind Evolution is a genetic search strategy over the natural-language solution space: given only a "solution evaluator," it uses an LLM to generate, recombine, and iteratively improve candidate solutions — without formalizing the problem.
It combines divergent thinking (parallel exploration) with convergent thinking (evaluate, select, refine).
Key contrasts:
In the language-based genetic algorithm, "genes" are human-readable plans; the LLM acts as both gene editor and reproduction mechanism, while the evaluator supplies selection pressure.
RCC: Refinement through Critical Conversation
Each refinement round splits the model into two roles:
Separating criticism from writing improves the quality of critical thinking — supported by ablations.
Island model: preserving diversity
To avoid premature convergence, the population is split into islands evolving independently, with migration of strong solutions between islands and island resets replacing weak islands — the same techniques used successfully by FunSearch.
Key hyperparameters
Benchmark results
TravelPlanner
Single-pass performance is dire: Gemini 1.5 Flash scores 5.6% (val, 1-pass); o1-preview 11.7%. Best-of-N at 800 samples reaches only 55.6%. Mind Evolution achieves 95.6% (val) / 95.2% (test) — and more cheaply than Best-of-N (174 LLM calls, $0.29 vs. 472 calls, $0.47 on val). A two-stage strategy (Flash first, Pro for unsolved cases) reaches 100% (val) / 99.9% (test) — without needing a formal solver, unlike prior near-parity work.
Natural Plan – Trip Planning
Mind Evolution: 96.2% (val) / 94.1% (test) vs. Best-of-N 77.2% and Sequential-Revision+ 74.4% (val). Two-stage: 100% / 99.6%. A striking example shows different strategies failing differently: 1-pass and Best-of-N err on the number of days; sequential revision gets days right but misses a Madrid show and books a nonexistent flight — Mind Evolution satisfies all constraints. Evolution's diversity-plus-feedback corrects for single strategies' error biases.
Natural Plan – Meeting Planning
An optimization task: maximize friends met under travel-time constraints. Mind Evolution: 85.0% (val) / 83.8% (test) vs. Best-of-N 69.4%, Sequential-Revision+ 62.0%. Two-stage: 98.4% / 98.2%. The evaluator doubles as an objective function — e.g., judging a plan meeting Sandra, Mark, Kevin, and Amanda (but not Michelle) as optimal given time windows.
StegPoet (new benchmark)
A new task hiding numeric messages in poems: build a digit→word cipher, embed cipher words in order in a stylized text, with at least \(B\) filler words between cipher words (\(3 \le B \le 7\)). Results: 1-pass 0%, Best-of-N 1%, Sequential-Revision+ 19.8%, Mind Evolution (Flash) 46.5%, two-stage 87.1% (val) / 79.2% (test) — proof the method handles hard-to-formalize but verifiable tasks.
Does evolution really "think deeper"?
Success rates rise steadily from generation 1 to 10 across all three planning tasks, showing genuine scaling with inference-time compute. At equal candidate counts, Mind Evolution consistently outperforms Best-of-N and sequential revision. The authors speculate Best-of-N struggles on TravelPlanner because many constraints (return to origin, no repeated restaurants) are implicit and must be learned from evaluator feedback — which Best-of-N ignores.
Ablations
On TravelPlanner val: removing the critic hurts significantly; removing textual evaluator feedback hurts significantly; task-specific strategy/question prompts help; LLM-based diversity selection for island resets beats pure score-based elitism. The two most important ingredients: critic + textual feedback.
Evaluation and cost
All tasks use programmatic evaluators scoring "0 as perfect" with deductions per violated constraint plus textual feedback. TravelPlanner's textual test-set constraints were extracted to JSON with Gemini 1.5 Flash (validated to match official JSON exactly on val).
On cost: Mind Evolution is cheaper than Best-of-N on TravelPlanner; Sequential-Revision+ is very expensive due to 80-turn dialogues consuming massive input tokens. The two-stage Flash→Pro strategy maximizes cost-effectiveness.
Conclusion and limitations
Mind Evolution significantly outperforms Best-of-N and sequential revision via inference-time evolutionary search, achieving near-perfect results on TravelPlanner and Natural Plan without formal solvers, and handles harder-to-formalize tasks via StegPoet. Limitations: it currently requires programmatically evaluable tasks; extending to LLM-based evaluators risks noisy, unreliable feedback.
The real magic isn't making models inherently smarter — it's reframing generation as search and optimization under verifiable feedback. When evaluators are reliable, search is strong; when they're vague or misleading, evolution can sprint in the wrong direction.
References
1. Lee, K.-H., Fischer, I., Wu, Y.-H., Marwood, D., Baluja, S., Schuurmans, D., & Chen, X. (2025). Evolving Deeper LLM Thinking. arXiv:2501.09891v1. 2. Xie et al., TravelPlanner benchmark. 3. Natural Plan benchmark. 4. Shinn et al., Reflexion. 5. FunSearch (island model precedent).