Predict and Reconstruct: Joint Objectives for Self-Supervised Language Representation Learning
DGX agentarXiv:2606.05173v1 Announce Type: new Abstract: Masked language modelling (MLM) has been the dominant pre-training objective for text encoders since BERT, yet it encourages representations that are st