Second-Order Smooth Planning with Optimal-Transport Bellman Smoothing

arXiv:2609.06484v1 Announce Type: new Abstract: Planning with a generative model aims to estimate the value of a state using as few simulator calls as possible. SmoothCruiser achieves problem-independent complexity widetilde O(arepsilon^{-4}) by exploiting the smoothness of the entropy-regularized Bellman backup, but its estimator is only first-order. We show that the sample-complexity exponent of SmoothCruiser-type planners is governed by the order eta of the local Taylor remainder, giving oracle complexity widetilde O(arepsilon^{-(2+2/(eta-1))}): the first-order case eta=2 recovers SmoothCruiser, while a second-order/cubic remainder eta=3 yields widetilde O(arepsilon^{-3}). We reach this regime with an optimal-transport-smoothed Bellman backup over action distributions, which has a closed form, a policy gradient, and a Lipschitz Hessian, and whose quadratic correction admits an unbiased cross-product estimator. The resulting SecondOrderSmoothCruiser achieves widetilde O(arepsilon^{-3}) oracle complexity for fixed OT parameters, and we relate the OT, entropy-regularized, and unregularized objectives through explicit regularization-bias bounds.

Source: arXiv cs.LG | 2026-09-09