Model Releases
Context Window Failures in Relational Foundation Models
arXiv:2609.00460v1 Announce Type: new Abstract: Recent Relational Deep Learning architectures have been proposed as foundation models for multi-table relational data, yet they impose constrained neigh
arXiv:2609.00460v1 Announce Type: new Abstract: Recent Relational Deep Learning architectures have been proposed as foundation models for multi-table relational data, yet they impose constrained neighborhood budgets that force row truncation when an entity has many related records. We introduce Animus, a synthetic financial dataset in which predicting customer income requires aggregating up to tens of thousands of transactions. On the raw representation, three recently proposed models (RT, Griffin, RelGT) achieve R^2 le 0.18; a single, routine, temporal pre-aggregation step recovers R^2 up to 0.65. This questions whether current relational foundation models are ready for high-cardinality real-world data.
Related
- Relational Task Generation Language: A Declarative Specification Framework for Relational Deep Learning
- MetaSieve: Faster Relational Deep Learning through SQL-Based Metapath Selection
- TabH2O: A Unified Foundation Model for Tabular Prediction
Source: arXiv cs.LG | 2026-09-02