Fingerprint, Not Blueprint: How Positional Schemes Set the Default Spectral Algebra of Attention
arXiv:2607.06621v1 Announce Type: new Abstract: The pre-softmax score of an attention head is a bilinear form score(i,j) = x_i^T M x_j in a learned operator M = W_q^T W_k. Because M is generally non-s