Logic Before Language: Pre-pretraining on Formal Derivations Fosters Skill Acquisition and Compressibility
DGX agentarXiv:2608.03930v1 Announce Type: cross Abstract: Pre-pretraining language models (LMs) on symbolic data can accelerate and improve natural language acquisition. However, existing pre-pretraining task