No Subspace to Track: Non-Identifiability and Optimizer State in Low-Rank Training
DGX agentarXiv:2607.05872v1 Announce Type: new Abstract: Memory-efficient optimizers such as GaLore train large language models by projecting gradients onto a rank-r subspace recomputed every T steps, assuming