Research
Continuous-time Online Learning via Mean-Field Neural Networks: Regret Analysis in Diffusion Environments
arXiv:2604.10958v1 Announce Type: cross Abstract: We study continuous-time online learning where data are generated by a diffusion process with unknown coefficients. The learner employs a two-layer ne
arXiv:2604.10958v1 Announce Type: cross Abstract: We study continuous-time online learning where data are generated by a diffusion process with unknown coefficients. The learner employs a two-layer neural network, continuously updating its parameters in a non-anticipative manner. The mean-field limit of the learning dynamics corresponds to a stochastic Wasserstein gradient flow adapted to the data filtration. We establish regret bounds for both the mean-field limit and finite-particle system. Our analysis leverages the logarithmic Sobolev inequality, Polyak-Lojasiewicz condition, Malliavin calculus, and uniform-in-time propagation of chaos. Under displacement convexity, we obtain a constant static regret bound. In the general non-convex setting, we derive explicit linear regret bounds characterizing the effects of data variation, entropic exploration, and quadratic regularization. Finally, our simulations demonstrate the outperformance of the online approach and the impact of network width and regularization parameters.
Related
- Detecting Invariant Manifolds in ReLU-Based RNNs
- NetworkNet: A Deep Neural Network Approach for Random Networks with Sparse Nodal Attributes and Complex Nodal Heterogeneity
- Wolkowicz-Styan Upper Bound on the Hessian Eigenspectrum for Cross-Entropy Loss in Nonlinear Smooth Neural Networks
Source: arXiv cs.AI | 2026-04-14