Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior
DGX agentarXiv:2609.39827v1 Announce Type: cross Abstract: Pre-pretraining (PPT) on synthetic non-natural language data improves token efficiency during language model pre-training (PT). Prior work attributes