Safety
Offline-Online Reinforcement Learning for Linear Mixture MDPs
arXiv:2604.11994v1 Announce Type: new Abstract: We study offline-online reinforcement learning in linear mixture Markov decision processes (MDPs) under environment shift. In the offline phase, data ar
arXiv:2604.11994v1 Announce Type: new Abstract: We study offline-online reinforcement learning in linear mixture Markov decision processes (MDPs) under environment shift. In the offline phase, data are collected by an unknown behavior policy and may come from a mismatched environment, while in the online phase the learner interacts with the target environment. We propose an algorithm that adaptively leverages offline data. When the offline data are informative, either due to sufficient coverage or small environment shift, the algorithm provably improves over purely online learning. When the offline data are uninformative, it safely ignores them and matches the online-only performance. We establish regret upper bounds that explicitly characterize when offline data are beneficial, together with nearly matching lower bounds. Numerical experiments further corroborate our theoretical findings.
Related
- When Can You Poison Rewards? A Tight Characterization of Reward Poisoning in Linear MDPs
- Beyond Pessimism: Offline Learning in KL-regularized Games
- CROP: Conservative Reward for Model-based Offline Policy Optimization
- MDP modeling for multi-stage stochastic programs
- MDP Planning as Policy Inference
Source: arXiv cs.LG | 2026-04-15