Scaling Laws for Mixture Pretraining Under Data Constraints
HumanApple derives a repetition-aware mixture scaling law showing scarce target data can be reused ~15–20× when mixed with generic corpora.
arXiv:2604.14198v1 Announce Type: cross Abstract: Domain reweighting can improve sample efficiency and downstream generalization, but data-mixture optimization for multimodal midtraining remains large