Real-world dynamic systems often exhibit time-varying behavior. While state space models (SSMs) have shown great potential for sequence modeling, especially in capturing long-range dependencies, most previous studies have been limited to time-invariant dynamics. To overcome this limitation, we propose a neural network architecture based on time-varying SSMs with dynamics that evolve over time, called dynamic SSMs. To enhance scalability and efficiency, several techniques are introduced, including sparsification strategy via diagonalization and fast tensor convolution with quasi-linear complexity in sequence length. Extensive experiments on both synthetic and real-world datasets show that the proposed model consistently outperforms existing state-of-the-art methods. Moreover, the model achieves significantly lower time and space complexity compared to architectures such as Transformer and LSTM. This work advances the theoretical foundation of SSMs-based neural networks in deep learning and promotes their further development. Code is available at:https://github.com/leonty1/dssmhttps://github.com/leonty1/dssm.

Although deep learning-based methods have shown great success in spatiotemporal predictive learning, the frameworks of those models are mainly designed by intuition. How to make spatiotemporal forecasting with theoretical guarantees is still a challenging issue. In this work, we tackle this problem by applying domain knowledge from the dynamical system to the framework design of deep learning models. An observer theory-guided deep learning architecture, called Spatiotemporal Observer, is designed for predictive learning of high dimensional data. The characteristics of the proposed framework are twofold: first, it provides the generalization error bound and convergence guarantee for spatiotemporal prediction; second, dynamical regularization is introduced to enable the model to learn system dynamics better during training. Further experimental results demonstrate that this framework could effectively model the spatiotemporal dynamics and make accurate predictions in both one-step-ahead and multi-step-ahead forecasting scenarios.