Safety
Lyapunov-Certified Direct Switching Theory for Q-Learning
arXiv:2604.19569v1 Announce Type: cross Abstract: Q-learning is one of the most fundamental algorithms in reinforcement learning. We analyze constant-stepsize Q-learning through a direct stochastic sw
arXiv:2604.19569v1 Announce Type: cross Abstract: Q-learning is one of the most fundamental algorithms in reinforcement learning. We analyze constant-stepsize Q-learning through a direct stochastic switching system representation. The key observation is that the Bellman maximization error can be represented exactly by a stochastic policy. Therefore, the Q-learning error admits a switched linear conditional-mean recursion with martingale-difference noise. The intrinsic drift rate is the joint spectral radius (JSR) of the direct switching family, which can be strictly smaller than the standard row-sum rate. Using this representation, we derive a finite-time final-iterate bound via a JSR-induced Lyapunov function and then give a computable quadratic-certificate version.
Related
- FASTER: Value-Guided Sampling for Fast RL
- StructRL: Recovering Dynamic Programming Structure from Learning Dynamics in Distributional Reinforcement Learning
- CROP: Conservative Reward for Model-based Offline Policy Optimization
- Low-Rank Adaptation for Critic Learning in Off-Policy Reinforcement Learning
Source: arXiv cs.AI | 2026-04-22