PC Layer: Polynomial Weight Preconditioning for Improving LLM Pre-Training
arXiv:2606.06470v1 Announce Type: cross Abstract: We propose a preconditioning (PC) layer, a weight parameterization via polynomial preconditioner that ensures stable weight conditioning throughout LL