Is Monte Carlo Tree Search Just Every-Visit Monte Carlo Control?

arXiv:2608.27985v1 Announce Type: new Abstract: Monte Carlo Tree Search (MCTS) and every-visit Monte Carlo (MC) control are usually presented as different methods. MCTS is described in the language of search (selection, expansion, simulation, and backup), whereas MC control is described in the language of reinforcement learning (trajectory sampling, return estimation, action-value updating, and policy improvement). This note argues that, at the level of trajectory generation and action-value updating, the distinction is largely terminological. The tree policy and rollout policy can be viewed as the learned and not-yet-learned parts of a single evolving policy; expansion corresponds to first visit and initialization; and backup is the ordinary every-visit Monte Carlo update. Under this interpretation, the four stages of MCTS reduce to two basic operations: trajectory sampling under the current policy and every-visit Monte Carlo updating. In this sense, MCTS is simply every-visit Monte Carlo control expressed in the language and data structure of search. The purpose of this note is expository: to make this equivalence explicit and easier to recognize.

Related

• PMCTS: Particle Monte Carlo Tree Search for Principled Parallelized Inference Time Scaling
• Twice Sequential Monte Carlo for Tree Search
• Decoupling Exploration and Policy Optimization: Uncertainty Guided Tree Search for Hard Exploration

Source: arXiv cs.LG | 2026-08-31