Demystifying Hidden-State Recurrence: Switchable Latent Reasoning with On-Policy Reinforcement Learning

arXiv:2606.13106v2 Announce Type: replace Abstract: Latent chain-of-thought compresses reasoning through continuous hidden-state recurrence, but existing approaches are difficult to optimize with standard on-policy reinforcement learning (RL) and hard to analyze mechanistically. We introduce SWITCH, a switchable latent reasoning framework that addresses both challenges with two discrete boundary tokens: enters latent reasoning and exits it. These discrete anchors make policy ratios well-defined for standard on-policy RL while providing natural intervention points for mechanistic analysis. We train SWITCH with a visible-to-latent curriculum and Switch-GRPO, which propagates gradients through recurrent latent computation. SWITCH consistently outperforms prior hidden-state-recurrence approaches at similar model scales. Mechanistic analysis reveals that (i) implements a localized, learned switching policy rather than a stylistic artifact; (ii) the latent step performs problem-specific, causally important computation; and (iii) this computation is concentrated in a single hidden-state transition upon entering latent mode. Together, these results show that hidden-state-recurrence latent reasoning can be both effectively optimized with on-policy RL and directly analyzed mechanistically, enabling us to study how RL reshapes internal reasoning computation. Code is available at https://github.com/LARK-AI-Lab/SWITCH.

Source: arXiv cs.LG | 2026-10-14