Research
Which Tasks Survive Self-Supervised Learning?
arXiv:2609.38393v1 Announce Type: cross Abstract: Same-instance self-supervised learning (SSL) learns representations by enforcing consistency across two views of the same underlying instance. This pr
arXiv:2609.38393v1 Announce Type: cross Abstract: Same-instance self-supervised learning (SSL) learns representations by enforcing consistency across two views of the same underlying instance. This principle alone, however, does not determine which downstream tasks remain recoverable from the learned representation. We study this question through semantic recoverability, defined as the amount of a task's posterior score captured by the represented function space. We show that, for centered and whitened representations, recoverability exactly determines directional class-distance-normalized variance (CDNV), controls few-shot nearest-centroid classification, and governs the strength of task-relevant semantic directions. The population linear probe and centroid axis coincide, and multiple well-recovered tasks approach a factorial centroid geometry. We then analyze a canonical two-view SSL objective and show that its population optimum spans the leading cross-view-stable modes of the associated two-view operator. This yields a closed-form spectral characterization of semantic recoverability: a downstream task is preserved to the extent that its posterior lies in the selected spectral subspace. We validate these predictions on synthetic and real datasets across several SSL methods, testing the predicted relationships among recoverability, directional geometry, spectral structure, and few-shot transfer. Together, these results give a task-level account of what information survives same-instance SSL and how the retained information appears in downstream geometry and transfer.
Related
- The Impact of Semantic Pairs on Self-Supervised Representation Learning
- BEST-RQ-2: Contextualize-Then-Predict, a Two-Step Approach for Self-Supervised Audio Representations
- Consistency-Driven Co-Evolution for Self-Supervised Cross-Representation Learning
- Predictive Self-Supervised Learning Provably Identifies Stochastic Signals under Nuisance
- A Minimal Model of Representation Collapse: Frustration, Stop-Gradient, and Dynamics
Source: arXiv cs.AI | 2026-10-01