When Tokenizers Fail: Byte-Level Chunking for Zero-Shot Transfer to Low-Resource Languages
DGX agentarXiv:2608.27658v1 Announce Type: new Abstract: Subword tokenization hinders low-resource language processing by imposing frequency patterns from dominant languages onto script-sharing variants. Byte-