Signatures of Steerability in Activation Space of Language Models

arXiv:2609.14151v1 Announce Type: cross Abstract: Steering language models using a set of contrastive representations has been a canonical and computationally efficient method for controlling model behavior. Despite this success in controlling certain model behaviors, the effectiveness of activation steering varies markedly across concepts; the generalization properties of steering vectors are often considered a function of the dataset used to construct them. We make this dataset-dependence claim more rigorous and show that simple separation metrics strongly correlate with the downstream steerability of language models across diverse settings, even after controlling for layers and dataset effects. Beyond prediction, we provide evidence from a synthetic superposition experiment that separation metrics are strongly correlated with alignment between the empirical and true feature direction. Our results suggest that simple separability statistics can serve as practical diagnostics for when steering vectors are likely to work.

Related

• Contrastive Conceptor Activation Steering (COAST): Unlocking Vision-Language-Action Models through Hidden States
• Deployable Per-Instance Multi-Layer Activation Steering for Large Language Models
• Where Steering Signals Come From: Activation Source Selection in Activation Steering
• Adaptive Multi-Subspace Representation Steering for Attribute Alignment in Large Language Models

Source: arXiv cs.AI | 2026-09-15