Attention Sinks and Outliers in Attention Residuals
DGX agentarXiv:2605.17887v1 Announce Type: cross Abstract: We propose OASIS, an outlier- and sink-aware technique built on inter-layer null signaling. As AttnResidual architectures introduce an additional dept