CIDER-FM: Foundation Models for Causal Inference from Diverse Experimental Regimes

arXiv:2609.39523v1 Announce Type: new Abstract: Causal foundation models (CFMs) amortise causal inference over priors of synthetic structural causal models (SCMs), predicting the effect of an experiment on a specific variable. However, observational data alone may leave multiple causal models compatible with available evidence, while experimental data with interventions on exactly the variable of interest might be unavailable. This work studies CFMs as a method to combine finite observational and surrogate-interventional datasets in order to predict a target conditional interventional distribution (CID) more accurately than with observational data alone. We first formalise the conceptual benefits of surrogate experiments. Building on this analysis, we introduce extsc{Foundation Models for Causal Inference from Diverse Experimental Regimes} (CIDER-FM), a causal foundation model that uses an intervention-aware representation and hierarchical three-axis attention to exchange information across variables, samples, and experimental regimes. We evaluate CIDER-FM against a wide range of baselines across diverse synthetic graph and mechanism families, as well as on both simulated and real-world data from Causal Chambers. Our results demonstrate strong CID prediction performance and show that incorporating experimental context can improve predictions over observational data alone.

Related

• MOCA: A Transformer-based Modular Causal Inference Framework with One-way Cross-attention and Cutting Feedback
• Bayesian Sensitivity of Causal Inference Estimators under Evidence-Based Priors
• Differentiable Causal Discovery for Singular Linear Models under Confounding
• Optimizing Likelihoods via Mutual Information: Bridging Simulation-Based Inference and Bayesian Optimal Experimental Design

Source: arXiv cs.LG | 2026-10-01