BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models

arXiv:2505.01912v3 Announce Type: replace-cross Abstract: Data-driven molecular discovery leverages artificial intelligence/machine learning (AI/ML) and generative modeling to filter and design novel molecules. Discovering novel molecules requires accurate out-of-distribution (OOD) predictions, but ML models struggle to generalize OOD. Currently, no systematic benchmarks exist for molecular OOD prediction tasks. We present mathbf{BOOM}, mathbf{b}enchmarks for mathbf{o}ut-mathbf{o}f-distribution mathbf{m}olecular property predictions: a chemically-informed benchmark for OOD performance on common molecular property prediction tasks. We evaluate over 150 model-task combinations to benchmark deep learning models on OOD performance. Overall, we find that no existing model achieves strong generalization across all tasks: even the top-performing model exhibited an average OOD error 3x higher than in-distribution. Current chemical foundation models do not show strong OOD extrapolation, while models with high inductive bias can perform well on OOD tasks with simple, specific properties. We perform extensive ablation experiments, highlighting how data generation, pre-training, hyperparameter optimization, model architecture, and molecular representation impact OOD performance. Developing models with strong OOD generalization is a new frontier challenge in chemical ML. This open-source benchmark is available at https://github.com/FLASK-LLNL/BOOM

Related

• GraphBench: Next-generation graph learning benchmarking
• Symbolic Neural Generation with Applications to Lead Discovery in Drug Design
• Beyond Drug Discovery: The Nanotechnology Molecular Optimization (NMO) Benchmark
• A Systematic Evaluation of Co-folding Model Representations for Small-Molecule Learning

Source: arXiv cs.AI | 2026-09-17