Paper: http://arxiv.org/abs/2605.06546 HF: http://huggingface.co/papers/2605.06546 Blog: http://nousresearch.com/token-superposition
This paper investigates token superposition, a phenomenon where language models can encode multiple token representations simultaneously in a single position, enabling more efficient use of model capa