Why Subliminal Learning Needs So Much Data: A Noisy Inverse View through Steering Vector Recovery

arXiv:2610.04907v1 Announce Type: new Abstract: Subliminal learning lets a student inherit a teacher's behavioral trait from semantically unrelated data, yet published demonstrations typically require tens of thousands of carrier examples. We ask where this data requirement comes from. Our testbed is subliminal steering: the teacher trait is a known residual-stream vector Delta_T, so transfer can be measured directly as parameter recovery. On identical carrier prefixes, we compare token-level (hard) NLL supervision with full-distribution (soft) KL supervision. At initialization the two objectives give nearly collinear gradients, and both align poorly with Delta_T. Under iterative optimization, however, they diverge: soft supervision recovers Delta_T almost exactly from a few hundred carriers, while hard supervision stays well below it even with tens of thousands. We explain this gap by casting steering-vector distillation as a noisy linear inverse problem. Locally, the carrier task maps the trait through its Fisher matrix F, so gradients point toward FDelta_T rather than Delta_T. Gradient descent then acts as a progressively less-damped inverse of F. With soft targets, this inverse restores low-curvature directions. With hard labels, it also amplifies the sampling noise in those same directions. The result is an optimal inversion depth that grows with the number of independent carriers. Experiments on Qwen2.5-7B and Gemma-2-9B confirm four predictions: the Fisher distortion of the initial gradient, recovery ordered from steep to flat directions, an optimal depth that shifts with data scale, and the finding that resampling completions from a fixed prompt pool works as well as adding new prompts. In this setting, large carrier datasets are needed less to reveal the trait than to suppress label noise amplified by Fisher inversion. Code is available at rl{https://github.com/luoyuchenmlcv/subliminal-data}.

Related

• Stored in Optimizer State, Valued by Later Training: A Causal Account of Subliminal Trait Transfer
• On Mitigation of Subliminal Learning in Large Language Models
• Subliminal Learning as Trait-Direction Drift: A Mechanism and Targeted Control under SFT Distillation
• Channel Location Constrains the Auditability of Subliminal Learning
• Cross-Trait Transfer in Subliminal Learning

Source: arXiv cs.LG | 2026-10-06