Tutorials
Orthogonal and Parametrized Multi-Attribute Steering of LLMs
arXiv:2610.37517v1 Announce Type: cross Abstract: Activation steering offers an effective and efficient way to modify an LLM's behavior at inference time. However, it remains unclear how to apply stee
arXiv:2610.37517v1 Announce Type: cross Abstract: Activation steering offers an effective and efficient way to modify an LLM's behavior at inference time. However, it remains unclear how to apply steering such that the signal is consistently strong and the steering strength has a universally interpretable meaning. We address this first by identifying the most steerable interval for each steering vector and rescaling the steering range accordingly. Building on this, we introduce a more interpretable method for steering, called parameterized steering which instead of pushing activations in a direction, it sets them at the desired trait strength. Next, with multi-trait steering, we find that naively combining vectors causes them to interfere with one another. In response, we develop a method to orthogonalize the vectors; it decouples these traits, allowing a model to exhibit one behavior without inducing the others, thus enabling multiple traits to be steered simultaneously.
Related
- UNBIND: UNlearning By INference-time Directional Steering for Code LLMs
- Beyond Visual Enhancement: Adaptive Multi-Context Steering to Mitigate LVLM Hallucinations
- Correcting Influence: Unboxing LLM Outputs with Orthogonal Latent Spaces
Source: arXiv cs.AI | 2026-10-28