Context Staircase: Signature-Aligned Dynamics of Token Embeddings under Small Initialization
arXiv:2608.30315v1 Announce Type: cross Abstract: Token embeddings are the basic representational units that connect discrete tokens with continuous computation in language models. Although modern lan