Research
Tydra: An Efficient Hybrid Model for Tabular Data
arXiv:2608.21199v1 Announce Type: new Abstract: Transformer-based tabular foundation models such as TabPFN achieve strong predictive performance but incur quadratic computational cost with context len
arXiv:2608.21199v1 Announce Type: new Abstract: Transformer-based tabular foundation models such as TabPFN achieve strong predictive performance but incur quadratic computational cost with context length. On the other hand, subquadratic SSM-based alternatives such as Hydra trade away accuracy for efficiency. To balance both, we introduce Tydra, a hybrid Transformer-State Space Model (SSM) architecture for tabular in-context learning that interleaves attention and SSM layers. Across 30 OpenML datasets, Tydra reduces inference time by 30% relative to TabPFN while retaining much of its predictive performance. Tydra also outperforms an approximately ten-times-larger Hydra model while providing faster inference. The results indicate that hybrid architectures are a promising direction for tabular foundation models.
Related
- Memory Efficient Tabular Foundation Models
- TabSwift: An Efficient Tabular Foundation Model with Row-Wise Attention
- GEAR: Generative Expansion and Real Anchoring for Two-Stage Distillation of Tabular Foundation Models
- Rethinking Stepwise Model Routing: A Cost-Efficient Table Reasoning Perspective
Source: arXiv cs.LG | 2026-08-24