Research
Formalising the Logit Shift Induced by LoRA: A Technical Note
arXiv:2604.20313v1 Announce Type: cross Abstract: This technical note provides a first-order formalisation of the logit shift and fact-margin change induced by Low-Rank Adaptation (LoRA). Using a firs
arXiv:2604.20313v1 Announce Type: cross Abstract: This technical note provides a first-order formalisation of the logit shift and fact-margin change induced by Low-Rank Adaptation (LoRA). Using a first-order Frechet approximation around the base model trajectory, we show that the multi-layer LoRA effect can be decomposed into a linear summation of layerwise contributions and a higher-order remainder term representing inter-layer coupling.
Related
- Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models
- GAIN: Multiplicative Modulation for Domain Adaptation
- Harmonizing Multi-Objective LLM Unlearning via Unified Domain Representation and Bidirectional Logit Distillation
- Less is More: Data-Efficient Adaptation for Controllable Text-to-Video Generation
Source: arXiv cs.AI | 2026-04-23