Relational In-Context Learning via Synthetic Pre-training with Structural Prior

arXiv:2603.03805v2 Announce Type: replace Abstract: Relational Databases (RDBs) are the backbone of modern business, yet they lack foundation models comparable to those in text or vision. A key obstacle is that high-quality RDBs are private, scarce and structurally heterogeneous, making internet-scale pre-training infeasible. To overcome this data scarcity, We introduce extbf{RDB-PFN}, the first relational foundation model trained purely via extbf{synthetic data}. Inspired by Prior-Data Fitted Networks (PFNs) where synthetic data generated from Structural Causal Models (SCMs) enables reasoning on single tables, we design a extbf{Relational Prior Generator} to create an infinite stream of diverse RDBs from scratch. Pre-training on extbf{over 2 million} synthetic single-table and relational tasks, RDB-PFN learns to adapt to any new database instantly via genuine extbf{in-context learning}. Experiments verify RDB-PFN achieves strong few-shot performance on 19 real-world relational prediction tasks, outperforming graph-based and single-table foundation-model baselines (given the same DFS-linearized inputs), while using a lightweight architecture and fast inference. The code is available at https://github.com/MuLabPKU/RDBPFN

Related

• Graph In-Context Operator Networks for Generalizable Spatiotemporal Prediction
• In-Context Learning Under Regime Change
• Tabular foundation models for in-context prediction of molecular properties
• GraphWalker: Graph-Guided In-Context Learning for Clinical Reasoning on Electronic Health Records
• Learning Posterior Predictive Distributions for Node Classification from Synthetic Graph Priors

Source: arXiv cs.LG | 2026-04-29