A convergence result of a continuous model of deep learning via a L{}ojasiewicz--Simon inequality

arXiv:2311.15365v3 Announce Type: replace Abstract: We study an idealized training process for deep neural networks in a continuous-depth, mean-field model in which each layer is parameterized by a probability measure on a Euclidean parameter space. The training dynamics are formulated as a Wasserstein-type gradient flow of an objective with a fixed L^2-regularization. Under suitable analyticity and growth assumptions, together with a coercivity assumption and sufficient regularity of the initial data, we prove that every curve of maximal slope converges to a single critical point of the objective as the training time tends to infinity. The proof combines compactness of the curve with a L{}ojasiewicz--Simon inequality for the metric slope. To establish the inequality, we lift the objective to a Hilbert space of random variables and use the analyticity of the lifted gradient in a stronger L^infty topology to overcome its lack of continuous differentiability in the Hilbert-space topology. Our convergence result does not require global displacement convexity, a Polyak--L{}ojasiewicz-type condition, or initialization near a minimizer; the objective may remain genuinely nonconvex.

Source: arXiv cs.LG | 2026-07-23