Research
Kernelized Activation Steering
arXiv:2610.01062v1 Announce Type: new Abstract: Activation steering provides a simple, training-free mechanism for controlling attributes of generative models such as sentiment, style, and helpfulness
arXiv:2610.01062v1 Announce Type: new Abstract: Activation steering provides a simple, training-free mechanism for controlling attributes of generative models such as sentiment, style, and helpfulness. However, standard approaches such as Difference-in-Means apply a single input-independent steering vector across all activations, limiting expressivity and ignoring the local geometry of the activation space. We propose Kernelized Activation Steering (KAS), a unifying framework that lifts activation steering into a reproducing kernel Hilbert space. KAS formulates steering as an optimization problem expressed purely via kernel evaluations, yielding an implicit, activation-dependent steering score without constructing explicit feature maps. Unlike DiM, KAS induces locally adaptive steering: each activation is modified according to its relative position with respect to source and target reference sets, producing a nonlinear steering field over the representation space. Importantly, DiM is recovered as a special case under a linear kernel, while richer kernels enable geometry-aware interventions. Across standard activation steering tasks, including jailbreaking LLMs and image style control, KAS outperforms or is on par with the existing methods.
Related
- Zero-shot rib design: merging training-free generative prior with topology optimization
- GeoEdit: Local Frames for Fast, Training-Free On-Manifold Editing in Diffusion Models
- Tempered Guided Diffusion
- Discrete Diffusion Language Models Are Training-Free Multi-Label Classifiers
Source: arXiv cs.LG | 2026-10-02