Model Releases
Robust CurveMoE: Multi-Norm Adversarial Defense for Mixture-of-Experts Models via Mode Connectivity
arXiv:2608.26043v1 Announce Type: new Abstract: Multi-norm adversarial defense aims to protect neural networks against perturbations defined by different norm constraints, but existing methods typical
arXiv:2608.26043v1 Announce Type: new Abstract: Multi-norm adversarial defense aims to protect neural networks against perturbations defined by different norm constraints, but existing methods typically optimize competing robustness objectives within a single parameter configuration, leading to substantial training cost and unfavorable robustness trade-offs. We propose Robust CurveMoE, an efficient mixture-of-experts framework that connects models specialized for different perturbation norms through a low-loss path and exploits the complementary robustness profiles of models along this path. Robust CurveMoE derives clean and norm-specialized experts from robustness-constrained curve locations and selectively expertizes only influential layers, while sharing the remaining parameters across routing paths. To further reduce curve-construction cost, we introduce contribution-guided partial updating, which selects influential curve parameters using initialization-based gradient scores. We also theoretically bound the objective gap between partial and full curve optimization. Experiments on CIFAR-100 and ImageNet-100 with WideResNet and Vision Transformer architectures show that Robust CurveMoE consistently improves clean, norm-specific, and Union accuracy over MSD and ERMC. In particular, it improves Union accuracy by 2.37 and 2.13 percentage points over the strongest baseline on CIFAR-100 and ImageNet-100, respectively. Extensive ablations further validate the effectiveness of partial updating, selective expertization, and robustness-constrained expert selection.
Related
- Scaling Linear Mode Connectivity and Merging to Billion Parameter Pretrained Transformers
- Adversarial Fine-tuning of Compressed Neural Networks for Joint Improvement of Robustness and Efficiency
- On Bayesian Softmax-Gated Mixture-of-Experts Models
- Beyond ell_2-norm and ell_infty-norm: A Curvature-Inspired ell_p-Norm Scheme for Deep Neural Networks
Source: arXiv cs.LG | 2026-08-27