LATS (Language Agent Tree Search): A Unified Framework for Reasoning, Acting, and Planning

Introduction

As large language models (LLMs) are increasingly used in decision-making tasks, expectations for them as autonomous agents have grown. However, traditional LLM agents often follow simple linear execution patterns and struggle with the deliberative, multi-step planning required by complex tasks. LATS (Language Agent Tree Search) is a novel, general framework that elegantly integrates the reasoning, acting, and planning capabilities of LLMs. Borrowing ideas from Monte Carlo Tree Search (MCTS) in model-based reinforcement learning, LATS systematically searches over a latent action space, using the language model as the decision agent, value function, and optimizer, thereby significantly improving decision quality. A key innovation is the introduction of external environment feedback, enabling self-reflection and adjustment based on signals from the environment. LATS is the first framework to unify reasoning, acting, and planning to enhance LLM performance.

LATS Methodology

Unlike linear Chain-of-Thought (CoT) prompting, LATS constructs a language agent tree, where each node represents a partial decision sequence (task input plus a series of reasoning or action steps). Key components include:

• Node and state representation: Each node stores current context, intermediate reasoning steps, and observations from the environment.
• Action and search strategy: The LLM acts as a policy network, sampling multiple candidate next actions (both reasoning steps and executable actions) at each node. MCTS performs selection, expansion, simulation, and backpropagation, balancing exploration and exploitation.
• Value function and heuristics: The LLM is prompted to score each node's state with a scalar value reflecting its promise toward the goal, optionally combined with environment feedback (e.g., code execution results).
• Self-reflection and learning: When a search reaches a terminal node with a poor outcome (e.g., failed tests), the model generates a self-reflection text summarizing errors and possible improvements. These reflections, stored with failed trajectories in external memory, are reused in later iterations—enabling learning from experience without gradient updates.
• Environment interaction and feedback: After each action, the model receives observations or feedback (webpage content, compile errors, test results). This feedback serves both as a reward signal for value updates and as input to self-reflection, making decisions adaptive to the external world.

In summary, LATS uses tree search to coordinate reasoning, acting, and planning: the LLM serves as the core agent, MCTS explores multiple decision paths, the LLM itself provides value estimates, and environment feedback plus self-reflection continuously refine the process.

Comparison with Mainstream Methods

LATS differs from prior approaches across five dimensions: reasoning, acting, planning, self-reflection, and external memory.

| Method | Reasoning | Acting | Planning | Self-reflection | External memory | |---|---|---|---|---|---| | CoT | ✓ | ✗ | ✗ | ✗ | ✗ | | ReAct | ✓ | ✓ | ✗ | ✗ | ✗ | | ToT | ✓ | ✗ | ✓ | ✓ | ✓ | | RAP | ✓ | ✗ | ✓ | ✗ | ✓ | | Self-Refine | ✓ | ✗ | ✗ | ✓ | ✗ | | Beam Search | ✓ | ✗ | ✗ | ✓ | ✗ | | Reflexion | ✓ | ✓ | ✗ | ✓ | ✓ | | LATS | ✓ | ✓ | ✓ | ✓ | ✓ |

• Chain-of-Thought (CoT): Generates a single linear reasoning chain; errors accumulate with no path to recover. LATS explores multiple paths in parallel and corrects errors via feedback and reflection.
• ReAct: Alternates "thinking" and "acting" for interactive problem solving, but decisions are reflexive with a single candidate per step. LATS extends ReAct's loop into a tree search over multiple thought-action candidates.
• Tree-of-Thought (ToT): Searches over tree-structured reasoning using simple DFS/BFS with LLM-generated heuristics; limited to pure reasoning. LATS extends the search space to reasoning + acting, uses the more powerful MCTS (better exploration–exploitation balance via UCB), and adds self-reflection.
• RAP: Also applies MCTS with the LLM as a simulated environment and heuristic, but lacks real environment feedback and reflection. LATS interacts with a genuine environment and learns from failures.
• Self-Refine: Iteratively critiques and improves a single solution. LATS applies reflection across parallel search paths and uses it to guide tree pruning and expansion.
• Beam Search: Maintains multiple candidates ranked by a static score (e.g., LM probability). LATS's MCTS dynamically updates node values via backpropagated feedback and adaptively balances exploration/exploitation.
• Reflexion: Adds post-hoc reflection to ReAct-style interaction, stored in memory. LATS integrates reflection into multi-path tree search, extending it from single-thread to multi-thread search and adding explicit multi-step planning.

LATS is not a simple stack of these techniques—experiments show it achieves more than additive gains, e.g., doubling ReAct performance on HotPotQA and improving WebShop average score by 22.1%.

Experiments and Results

Programming (HumanEval)

LATS was evaluated on HumanEval (164 programming problems), combining the LLM with a Python execution environment that returns compile/test results. With GPT-4, LATS achieved 94.4% Pass@1 accuracy, a state-of-the-art result at the time—achieved without any fine-tuning, purely via prompting and search. This highlights LATS's potential as a gradient-free optimization method, with external test feedback proving crucial for program synthesis.

Interactive Question Answering (HotPotQA)

On HotPotQA multi-hop QA, where models must issue retrieval queries and reason across documents, LATS doubled GPT-3.5's accuracy relative to ReAct. Tree search explored alternative query combinations and reasoning orders, while environment feedback (answer correctness) and self-reflection allowed the model to avoid repeating failed strategies.

Web Navigation (WebShop)

In WebShop, a simulated e-commerce environment requiring browsing, searching, and purchasing, LATS with GPT-3.5 achieved an average score of 75.9—a substantial improvement over baselines and close to the performance of gradient-finetuned agents, again without any parameter updates. The search explored multiple browsing strategies (direct search, category browsing, recommendation links), and reflection identified ineffective keyword strategies.

Additional explorations (e.g., the Game of 24 math task) also yielded positive results, demonstrating the framework's generality across reasoning-heavy and interaction-heavy tasks.

Performance Evaluation and Metrics

• Reasoning tasks: Accuracy/F1 for QA (HotPotQA); Pass@k for code generation (HumanEval, where LATS reached 94.4% Pass@1).
• Decision tasks: Average score / cumulative reward on WebShop, reflecting task completion quality and efficiency; success rate and steps-per-episode are also relevant.
• Search efficiency and cost: Number of searches/generations, wall-clock time. LATS's intelligent pruning and feedback-guided evaluation keep search complexity controllable; search depth and candidate counts can be tuned to trade off performance and cost.
• Baselines: Comparisons against ReAct-only, ToT-only, and Reflexion-only configurations show clear gains (e.g., +22.1 points on WebShop over ReAct), and LATS matches finetuned-agent performance on WebShop without gradient updates.

Conclusion

LATS is the first framework to unify reasoning, acting, and planning for LLM agents. By organizing decision-making explicitly with Monte Carlo Tree Search and incorporating environment feedback and self-reflection, LATS overcomes the flexibility, perception, and adaptability limitations of prior methods. Experiments across programming, QA, and web navigation show record-setting or baseline-surpassing performance without gradient-based training. Academically, LATS demonstrates that (1) explicit planning and search are critical for complex-task performance, and (2) environment feedback enables continuous calibration of the model's internal knowledge—pointing toward combining classical planning algorithms with modern language models to build more capable agents.