Bi-Level Optimization for Single Domain Generalization

arXiv:2604.06349v1 Announce Type: cross Abstract: Generalizing from a single labeled source domain to unseen target domains, without access to any target data during training, remains a fundamental challenge in robust machine learning. We address this underexplored setting, known as Single Domain Generalization (SDG), by proposing BiSDG, a bi-level optimization framework that explicitly decouples task learning from domain modeling. BiSDG simulates distribution shifts through surrogate domains constructed via label-preserving transformations of the source data. To capture domain-specific context, we propose a domain prompt encoder that generates lightweight modulation signals to produce augmenting features via feature-wise linear modulation. The learning process is formulated as a bi-level optimization problem: the inner objective optimizes task performance under fixed prompts, while the outer objective maximizes generalization across the surrogate domains by updating the domain prompt encoder. We further develop a practical gradient approximation scheme that enables efficient bi-level training without second-order derivatives. Extensive experiments on various SGD benchmarks demonstrate that BiSDG consistently outperforms prior methods, setting new state-of-the-art performance in the SDG setting.

Related

• EviSnap: Faithful Evidence-Cited Explanations for Cold-Start Cross-Domain Recommendation
• PlaneCycle: Training-Free 2D-to-3D Lifting of Foundation Models Without Adapters
• Stabilizing Unsupervised Self-Evolution of MLLMs via Continuous Softened Retracing reSampling

Source: arXiv cs.AI | 2026-04-10