Parameter Alignment Mitigates Catastrophic Forgetting in Multilingual Expert Language Models
DGX agentarXiv:2606.00284v1 Announce Type: new Abstract: While continual pretraining~(CPT) is a practical way to extend large language models to new languages, naive finetuning on targeted data erodes existing