Neural Ordinary Differential Equations (NODEs) serve as continuous-time analogs of residual networks. They provide a system-theoretic perspective on neural network architecture design and offer natural solutions for time series modeling, forecasting, and applications where invertible neural networks are essential. However, these models suffer from slow performance due to heavy numerical solver overhead. For instance, a popular solution for training and inference of NODEs consists in using adaptive step size solvers such as the popular Dormand-Prince 5(4) (DOPRI). These solvers dynamically adjust the Number of Function Evaluations (NFE) as the equation fits the training data and becomes more complex. However, this comes at the cost of an increased number of function evaluations, which reduces computational efficiency. In this work, we propose a novel approach: making the parameters of the numerical integration scheme trainable. By doing so, the numerical scheme dynamically adapts to the dynamics of the NODE, resulting in a model that operates with a fixed NFE. We compare the proposed trainable solvers with state-of-the-art approaches, including DOPRI, for different benchmarks, including classification, density estimation, and dynamical system modeling. Overall, we report a state-of-the-art performance for all benchmarks in terms of accuracy metrics, while enhancing the computational efficiency through trainable fixed-step-size solvers. This work opens up new possibilities for practical and efficient modeling applications with NODEs.

When considering the data-driven identification of non-linear differential equations, the choice of the integration scheme to use is far from being trivial and may dramatically impact the identification problem. In this work, we discuss this aspect and propose a novel architecture that jointly learns Neural Ordinary Differential Equations (NODEs) as well as the corresponding integration schemes that would minimize the forecast of a given sequence of observations. We demonstrate its relevance with numerical experiments on non-linear dynamics, including chaotic systems.

The reconstruction of sea surface geophysical variables typically relies either on Optimal Interpolation (OI) or on model-based approaches which explicitly exploit a dynamical model. While the optimal interpolation suffers from smoothing issues making it unreliable in retrieving fine scale variability, the selection and parametrization of a dynamical model, when considering model based data assimilation strategies, remains a complex issue since several trade-offs between the models complexity and its applicability in sea surface data assimilation need to be carefully addressed. In this work, and motivated by the success of artificial intelligence algorithms in various signal processing fields as well as the increasing amounts of observations and simulation datasets, we explore a data driven Koopman model and formulate the reconstruction problem as solution of the classical Kalman filter in a space of observables. The proposed architecture although linear, is shown to outperform state-of-the-art non linear data-driven filtering schemes such as the Analog Data Assimilation (AnDA).

Despite some promising results under ideal conditions (i.e. noise-free and complete observation), learning chaotic dynamical systems from real life data is still a very challenging task. We propose a novel framework, which combines data assimilation schemes and neural network representation, namely Auto-Encoders and Ensemble Kalman Smoother, to learn the governing equations of dynamical systems. By treating the learning as a Bayesian estimation problem, our framework can deal with noisy and partial observations. Experiments on the chaotic Lorenz-63 dynamics with different noise settings demonstrate the advantages of our method over the state-of-the-art.

The forecasting and assimilation of sea surface dynamics from satellite-derived data is a challenging issue. Data-driven approaches have arisen as promising schemes to fully exploit satellite observations. A key feature of sea surface dynamics is that they relate to partially-observed dynamics. Here, guided by physical and mathematical considerations of the underlying dynamics of a given system, we propose a novel neural networks architecture for the identification of ordinary differential equations (ODE) of partially observed systems. Numerical experiments for toy models and a sea level anomaly dynamics illustrate the relevance of the proposed scheme for forecasting and assimilation issues w.r.t. other state-of-the-art data-driven models.

In the last years, Neural Networks have enriched the state-of-the-art in probabilistic modeling. This is principally due to the advances in deep learning which allow a better understanding of complex systems. However, the stochastic representation of spatio-temporal fields is still an open challenge that may benefit from the recent advances in probabilistic modelization. In this work, we explore neural network to derive a stochastic representation of spatio-temporal dynamical systems based on ensemble forecasting. Trough the implementation of our stochastic model in a classical Kalman filtering scheme, we demonstrate the relevance of the proposed architecture in the reconstruction of geophysical fields with respect to the state-of-the-art approaches.

In this work, we investigate residual neural network representations for the identification and forecasting of dynamical systems. We propose a novel architecture that jointly learns the dynamical model and the associated Runge-Kutta integration scheme. We demonstrate the relevance of the proposed architecture with respect to learning-based state-of-the-art approaches in the identification and forecasting of chaotic dynamics when provided with training data with low temporal sampling rates.

In this work, we propose an alternative to the Ensemble Kalman filter through the implementation of a neural networks filtering scheme based on a parametric stochastic model. From our numerical experiment, we prove the relevance of the proposed architecture in the reconstruction of geophysical fields with respect to the state-of-the-art schemes.

Recent studies have investigated the identification of governing equations of geophysical systems from data. Here, we investigate such identification issues for ocean surface dy-namcis from ocean remote sensing data. From a methodological point of view, we address the learning of data-driven dynamical models when only provided with a noisy training dataset. We propose a novel architecture that relies on data assimilation schemes to learn the underlying dynamical model through the minimization of a reconstruction cost. We demonstrate the relevance of the proposed architecture with respect to the state-of-the-art approaches in the identification and forecasting of synthetic and real case-studies.

The forecasting and reconstruction of ocean and atmosphere dynamics from satellite observation time series are key challenges. While model-driven representations remain the classic approaches, data-driven representations become more and more appealing to benefit from available large-scale observation and simulation datasets. In this work we investigate the relevance of recently introduced bilinear residual neural network representations, which mimic numerical integration schemes such as Runge-Kutta, for the forecasting and assimilation of geophysical fields from satellite-derived remote sensing data. As a case-study, we consider satellite-derived Sea Surface Temperature time series off South Africa, which involves intense and complex upper ocean dynamics. Our numerical experiments demonstrate that the proposed patch-level neural-network-based representations outperform other data-driven models, including analog schemes, both in terms of forecasting and missing data interpolation.