Continuous-time reinforcement learning for optimal switching over multiple regimes

arXiv:2512.04697v3 Announce Type: replace-cross Abstract: This paper studies the continuous-time reinforcement learning (RL) for optimal switching problems across multiple regimes. We consider a type of exploratory formulation under entropy regularization where the agent randomizes both the timing of switches and the selection of regimes through the generator matrix of an associated continuous-time finite-state Markov chain. We establish the well-posedness of the associated system of Hamilton-Jacobi-Bellman (HJB) equations and provide a characterization of the optimal policy. The policy improvement and the convergence of the policy iterations are rigorously established by analyzing the system of equations. We also show that the value function in the exploratory formulation converges to the one in the classical formulation as the temperature parameter vanishes. Finally, a model-free reinforcement learning algorithm is devised and implemented by invoking the policy evaluation based on the martingale characterization. Our numerical examples with financial applications illustrate the effectiveness and efficiency of the proposed RL algorithm.

Related

• Mean-Field PhiBE: Continuous-Time Mean-Field Reinforcement Learning from Discrete-Time Data
• Optimal control of the future via prospective learning with control
• A Nonlinear Separation Principle: Applications to Neural Networks, Control and Learning

Source: arXiv cs.LG | 2026-07-31