Latent Matters: Learning Deep State-Space Models

arXiv:2602.23050v2 Announce Type: replace Abstract: Deep state-space models (DSSMs) enable temporal predictions by learning the underlying dynamics of observed sequence data. They are often trained by maximising the evidence lower bound. However, as we show, this does not ensure the model actually learns the underlying dynamics. We therefore propose a constrained optimisation framework as a general approach for training DSSMs. Building upon this, we introduce the extended Kalman VAE (EKVAE), which combines amortised variational inference with classic Bayesian filtering/smoothing to model dynamics more accurately than RNN-based DSSMs. Our results show that the constrained optimisation framework significantly improves system identification and prediction accuracy on the example of established state-of-the-art DSSMs. The EKVAE outperforms previous models w.r.t. prediction accuracy, achieves remarkable results in identifying dynamical systems, and can furthermore successfully learn state-space representations where static and dynamic features are disentangled.

Related

• Time-series forecasting through the lens of dynamics
• Learning to Defer in Non-Stationary Time Series via Switching State-Space Models
• Diffusion-Driven State Space Models
• Observable Neural ODEs for Identifiable Causal Forecasting in Continuous Time

Source: arXiv cs.LG | 2026-07-31