Research
Central Limit Theorems for Asynchronous Averaged Q-Learning
arXiv:2509.18964v3 Announce Type: replace Abstract: This paper establishes central limit theorems for Polyak-Ruppert averaged Q-learning under asynchronous updates. We prove a non-asymptotic central l
arXiv:2509.18964v3 Announce Type: replace Abstract: This paper establishes central limit theorems for Polyak-Ruppert averaged Q-learning under asynchronous updates. We prove a non-asymptotic central limit theorem, where the convergence rate in Wasserstein distance explicitly reflects the dependence on the number of iterations, state-action space size, the discount factor, and the quality of exploration. In addition, we derive a functional central limit theorem, showing that the partial-sum process converges weakly to a Brownian motion.
Related
- Gaussian Approximation for Asynchronous Q-learning
- Wasserstein-p Central Limit Theorem Rates: From Local Dependence to Markov Chains
- A Tale of Two Learning Algorithms: Multiple Stream Random Walk and Asynchronous Gossip
- Choose Your Battles: Distributed Learning Over Multiple Tug of War Games
Source: arXiv cs.LG | 2026-04-21