Discrete Cosine Transform Based Decorrelated Attention for Vision Transformers
DGX agentarXiv:2405.13901v4 Announce Type: replace Abstract: Self-attention is central to the success of Transformer architectures; however, learning the query, key, and value projections from random initializ