Safety
First-Order Steering: Translating Weight Adaptation into Activation Steering
arXiv:2610.04283v1 Announce Type: new Abstract: Activation steering exploits interpretable directions in the residual stream to enable inference-time manipulation of model behavior. Composing steering
arXiv:2610.04283v1 Announce Type: new Abstract: Activation steering exploits interpretable directions in the residual stream to enable inference-time manipulation of model behavior. Composing steering vectors to apply multiple target behaviors simultaneously is important in various fields-including AI alignment and safety-but remains a challenge for existing activation steering methods. In contrast, prior work in model merging shows that target behaviors represented by learned weight adaptations can be combined with high accuracy. A method that translates weight adaptations into activation steering vectors could therefore extend prior work in model merging to generate composable steering vectors that better enable simultaneous inference-time behavioral control. For this, we introduce First-Order Steering, a formulation of activation steering as a first-order approximation of weight update matrices parameterized by a vector of steering strengths, and establish theoretical bounds on the approximation error of first-order steering. We then develop a novel model merging procedure, HeRD-Merging, which minimizes the first-order approximation error terms to enable higher first-order steering accuracy. Together, our method produces steering vectors that control both individual and composed behaviors more accurately than existing activation steering methods. Furthermore, HeRD-Merging matches the performance of conventional model-merging baselines, while producing weight adaptations that admit more accurate first-order steering vectors.
Related
- Policy Gradient Steering: Interventions from Behavioral Objectives
- Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior
- Minimizing Collateral Damage in Activation Steering
- Does Fine-Tuning Undo Activation Steering? Behavioural Recovery Without Weight-Edit Reversal
- Spillover-Aware Multi-Value Steering for Pluralistic LLM Alignment
Source: arXiv cs.LG | 2026-10-06