A Lightweight Foundation Model for Collider Physics with Multi-Domain Adaptation

arXiv:2607.27501v1 Announce Type: new Abstract: We present a lightweight approach to foundation modeling (extbf{NEXUS}) that leverages pre-trained learning from collider physics data towards out-of-domain tasks in other scientific datasets, using a fully connected autoencoder model with approximately 3 million parameters. The model pre-trains with no supervision over a large-scale collision dataset from the Large Hadron Collider modeled by charged particle track features. Downstream tasks for collider analyses, such as kinematic regression and event classification, are developed on pre-trained model weights and achieve improved accuracy with only small labeled datasets when compared to equivalent architectures trained from scratch. The benefits of pre-training are additionally investigated through latent space interpretation and application to other domains, including gravitational waves, flood forecasting, and neural activity. Furthermore, the relative computational simplicity of NEXUS is demonstrated compared to transformer approaches at comparable scale, opening the door to power-efficient inference and real-time or edge applications of foundation models in scientific experiments.

Related

• Foundation Model Guided Dual-Branch Co-Adaptation for Source-Free EEG Decoding
• A Foundation Model for Instruction-Conditioned In-Context Time Series Tasks
• PLAID: A Unified Data Model for Machine Learning on Heterogeneous Physics Simulations
• Zeus: Towards Tuning-Free Foundation Model for Time Series Analysis

Source: arXiv cs.LG | 2026-07-31