Distance generalization in transformers: why bother with positional encoding?
DGX agentarXiv:2609.11913v1 Announce Type: new Abstract: Out-of-distribution length generalization, namely to extrapolate a task from short to longer context, has been studied intensively for transformers. Her