A Variational Analysis of Kernel Learning with Learnable Linear Transformations

arXiv:2502.11665v3 Announce Type: replace-cross Abstract: The classical kernel ridge regression problem aims to find the best fit for the output Y as a function of the input data Xin R^d, with a fixed choice of regularization term imposed by a given choice of a reproducing kernel Hilbert space, such as a Sobolev space. Here we consider a generalization of the kernel ridge regression problem, by introducing an extra matrix parameter U, which aims to detect the scale parameters and the feature variables in the data, and thereby improve the efficiency of kernel ridge regression. This naturally leads to a nonlinear variational problem to optimize the choice of U. We study various foundational mathematical aspects of this variational problem, including its Euler-Lagrange equation, continuity and first variation, limiting behavior under degenerate or diverging transformations, and the structure of its local minimizers. Particular attention is given to two data-distribution settings, namely multi-scale and multi-index models, where the learned transformation U encodes intrinsic scale parameters and the essential low-dimensional feature variables, respectively.

Source: arXiv cs.LG | 2026-08-13