Safety
Sample Complexity Bounds for Stochastic Shortest Path with a Generative Model
arXiv:2604.16111v1 Announce Type: new Abstract: We study the sample complexity of learning an epsilon-optimal policy in the Stochastic Shortest Path (SSP) problem. We first derive sample complexity bo
arXiv:2604.16111v1 Announce Type: new Abstract: We study the sample complexity of learning an epsilon-optimal policy in the Stochastic Shortest Path (SSP) problem. We first derive sample complexity bounds when the learner has access to a generative model. We show that there exists a worst-case SSP instance with S states, A actions, minimum cost c_{min}, and maximum expected cost of the optimal policy over all states B_{star}, where any algorithm requires at least Omega(SAB_{star}^3/(c_{min}epsilon^2)) samples to return an epsilon-optimal policy with high probability. Surprisingly, this implies that whenever c_{min} = 0 an SSP problem may not be learnable, thus revealing that learning in SSPs is strictly harder than in the finite-horizon and discounted settings. We complement this lower bound with an algorithm that matches it, up to logarithmic factors, in the general case, and an algorithm that matches it up to logarithmic factors even when c_{min} = 0, but only under the condition that the optimal policy has a bounded hitting time to the goal state.
Related
- Restless Bandits with Individual Penalty Constraints: A New Near-Optimal Index Policy and How to Learn It
- Offline-Online Reinforcement Learning for Linear Mixture MDPs
- A Two-Timescale Primal-Dual Framework for Reinforcement Learning via Online Dual Variable Guidance
- Continuous-time reinforcement learning: ellipticity enables model-free value function approximation
Source: arXiv cs.LG | 2026-04-20