KerJEPA: Kernel Discrepancies for Euclidean Self-Supervised Learning

arXiv:2512.19605v2 Announce Type: replace-cross Abstract: Breakthroughs in self-supervised Joint-Embedding Predictive Architectures (JEPAs) have established that regularizing Euclidean representations toward isotropic Gaussian priors yields provable gains in training stability and downstream generalization. We introduce a new, flexible family of Euclidean joint embedding self-supervised learning algorithms with kernel-based regularizers. One instance of this family corresponds to the SIGReg Epps-Pulley regularizer which approximates a sliced Maximum Mean Discrepancy (MMD) with a Gaussian prior and Gaussian kernel. By expanding the class of viable kernels and priors and computing the closed-form high-dimensional limit of sliced discrepancies, we develop alternative methods with improved training stability and design flexibility and leverage those techniques to train strong vision learners at scale.

DSeq-JEPA: Discriminative Sequential Joint-Embedding Predictive Architecture
lambda-JEPA Spectral Anti-Collapse Regularization for Self-Supervised Learning
Qantara: Bridge-Flow Training for Multi-Paradigm JEPA Control
The JEPA Predictor: A Transferable Operator for Occluded Feature Completion

Source: arXiv cs.CV | 2026-10-14

Related