TriPLU: Bypassing the Gate with Direct Trilinear Product FFNs in Tiny Language Models
arXiv:2608.20360v1 Announce Type: new Abstract: We study whether tiny decoder-only language models benefit from feed-forward layers that directly multiply learned feature projections. TriPLU, a Trilin