Scaling Laws for Mixture Pretraining Under Data Constraints
Apple derives a repetition-aware mixture scaling law showing scarce target data can be reused ~15–20× when mixed with generic corpora.
Knowledge catalogue
Apple derives a repetition-aware mixture scaling law showing scarce target data can be reused ~15–20× when mixed with generic corpora.
arXiv:2604.14198v1 Announce Type: cross Abstract: Domain reweighting can improve sample efficiency and downstream generalization, but data-mixture optimization for multimodal midtraining remains large
arXiv:2605.10288v1 Announce Type: new Abstract: Stochastic bilevel optimization (SBO) has become a standard framework for hyperparameter learning, data reweighting, representation learning, and data-m
arXiv:2610.01238v1 Announce Type: new Abstract: Unified language models are increasingly expected to combine heterogeneous capabilities, such as mathematics, code, instruction following, and controlla
arXiv:2606.11740v1 Announce Type: cross Abstract: We study whether grounded reasoning supervision from abundant 2D medical images can improve 3D medical VQA when both input types are aligned through a
This paper was accepted at the Workshop on Navigating and Addressing Data Problems for Foundation Models (NADPFM) at ICLR 2026. Principled domain reweighting can substantially improve sample efficienc