A Constrained Optimization Perspective of Unrolled Transformers
DGX agentarXiv:2601.17257v2 Announce Type: replace Abstract: We introduce a constrained optimization framework for training transformers that behave like optimization descent algorithms. Specifically, we enfor