Diffeomorphic Optimization

arXiv:2607.00947v1 Announce Type: new Abstract: Generative models learn data distributions that reside on a low-dimensional manifold within a higher-dimensional ambient space. Optimizing differentiable objectives on this manifold is challenging: the ambient loss landscape is high-dimensional, rugged, and non-convex. Direct gradient descent, blind to the manifold's geometry, quickly drifts off it. Diffeomorphic optimization starts from the observation that diffusion and flow models provide a map from the data manifold to a much simpler base space in which we perform gradient descent. Using differential geometry, we show this is equivalent to Riemannian gradient descent on the data manifold up to O(lambda^2) corrections, keeping trajectories on-manifold by construction and yielding a smoother optimization surface. For protein design, we extend diffeomorphic optimization to the matrix Lie groups SO(3) and SE(3), deriving an autograd-compatible SO(3) gradient and a generalized adjoint-state method for backpropagation through Lie-group ODE solvers. Diffeomorphic optimization improves over tuned guidance on secondary-structure targeting with FrameFlow (91.3% vs. 63.3% of residues in the Ramachandran target), outperforms OC-Flow on peptide binding affinity at 2imes the speed, and reduces Rosetta energies by thousands of units across the PDB test set for structures with hundreds of residues.

Source: arXiv cs.LG | 2026-07-02