Same Flow, Different Paths: Variance Reduction in Flow Matching

arXiv:2609.17287v1 Announce Type: new Abstract: In flow matching (FM), a velocity model v_{heta} is trained using a predefined path g_t that connects data and noise samples (e.g., g_t(x_0, x_1) = (1 - t) x_0 + t x_1). In this work, we study the choice of this path from an optimization perspective by analyzing the variance of stochastic gradients. We consider the class G(p_t,v^star_t) of paths that induce the same marginal distributions p_t and marginal velocity field v^star_t, and therefore the same FM objective. Our main finding is that the choice of path g_t can fundamentally change the convergence rate of SGD, even when the FM objective remains exactly the same. (i) For a linear velocity model and one-dimensional Gaussian data, we derive a tight bound on the SGD iteration complexity up to logarithmic factors and find an analytically optimal path that minimizes this bound among linear paths inducing the same FM problem. (ii) We then extend the variance analysis to general FM problems and formulate path selection at a fixed heta as the variance-minimization problem PathOpt_heta, constrained to g_tin G(p_t,v^star_t). We show that this constraint is essential: reducing variance without it can lead to slower convergence. (iii) Since the constraint g_t in G(p_t,v^star_t) cannot generally be verified directly, we derive an equivalent formulation with constraints that can be estimated from samples, allowing paths to be found numerically. Our theoretical results are supported by experiments with Gaussian data, Gaussian mixture models, and real datasets.

Source: arXiv cs.LG | 2026-09-16