Research
DARLING: Detection Augmented Reinforcement Learning with Non-Stationary Guarantees
arXiv:2604.16684v1 Announce Type: new Abstract: We study model-free reinforcement learning (RL) in non-stationary finite-horizon episodic Markov decision processes (MDPs) without prior knowledge of th
arXiv:2604.16684v1 Announce Type: new Abstract: We study model-free reinforcement learning (RL) in non-stationary finite-horizon episodic Markov decision processes (MDPs) without prior knowledge of the non-stationarity. We focus on the piecewise-stationary (PS) setting, where both the reward and transition dynamics can change an arbitrary number of times. We propose Detection Augmented Reinforcement Learning (DARLING), a modular wrapper for PS-RL that applies to both tabular and linear MDPs, without knowledge of the changes. Under certain change-point separation and reachability conditions, DARLING improves the best available dynamic regret bounds in both settings and yields strong empirical performance. We further establish the first minimax lower bounds for PS-RL in tabular and linear MDPs, showing that DARLING is the first nearly optimal algorithm. Experiments on standard benchmarks demonstrate that DARLING consistently surpasses the state-of-the-art methods across diverse non-stationary scenarios.
Related
- Contraction-Aligned Analysis of Soft Bellman Residual Minimization with Weighted Lp-Norm for Markov Decision Problem
- Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning
- Provably Efficient Offline-to-Online Value Adaptation with General Function Approximation
- A Modularized Framework for Piecewise-Stationary Restless Bandits
- Offline-Online Reinforcement Learning for Linear Mixture MDPs
Source: arXiv cs.LG | 2026-04-21