Gated MLPs as Symmetry-Broken Rank-1 Bilinear Attention
DGX agentarXiv:2606.22172v1 Announce Type: new Abstract: We show that the conventional gated MLP can be viewed as a rank-1 approximation to a bilinear attention mechanism with two distinct factors correspondin