Reservoir computing networks (RCNs) have been recognized as a popular machine learning tool for modeling the temporal evolution of dynamic data due to their close relationship with dynamical systems. A defining characteristic of RCNs is their fixed (training parameter-free) hidden layers, which offers significant computational benefits. However, this feature also introduces adverse impacts, such as increased warm-up time, limited long-term memory, and sensitivity to hyperparameters. To balance these advantages and disadvantages and expand the application domains of RCNs, we develop a novel deep reservoir computing network (DRCN) architecture that integrates control-theoretic concepts and techniques into RCNs. This architecture is designed as a cascade of shallow RCNs and is represented as a piecewise time-invariant control system. We further propose a layer-by-layer training strategy for the DRCN, resulting in an iterative deep learning algorithm for modeling dynamical systems. This enables us to exploit the DRCN as a generative model to generate output-of-sample data using the learned dynamical systems. The performance and efficiency of the DRCN-based dynamic generative model are demonstrated through various learning problems arising from time-series analysis and control systems, using both synthetic and real-world datasets.

Interest in reinforcement learning (RL) for large-scale systems, comprising extensive populations of intelligent agents interacting with heterogeneous environments, has surged significantly across diverse scientific domains in recent years. However, the large-scale nature of these systems often leads to high computational costs or reduced performance for most state-of-the-art RL techniques. To address these challenges, we propose a novel RL architecture and derive effective algorithms to learn optimal policies for arbitrarily large systems of agents. In our formulation, we model such systems as parameterized control systems defined on an infinite-dimensional function space. We then develop a moment kernel transform that maps the parameterized system and the value function into a reproducing kernel Hilbert space. This transformation generates a sequence of finite-dimensional moment representations for the RL problem, organized into a filtrated structure. Leveraging this RL filtration, we develop a hierarchical algorithm for learning optimal policies for the infinite-dimensional parameterized system. To enhance the algorithm's efficiency, we incorporate early stopping at each hierarchy, demonstrating the fast convergence property of the algorithm through the construction of a convergent spectral sequence. The performance and efficiency of the proposed algorithm are validated using practical examples in engineering and quantum systems.

The ability to promptly respond to environmental changes is crucial for the perception system of autonomous driving. Recently, a new task called streaming perception was proposed. It jointly evaluate the latency and accuracy into a single metric for video online perception. In this work, we introduce StreamDSGN, the first real-time stereo-based 3D object detection framework designed for streaming perception. StreamDSGN is an end-to-end framework that directly predicts the 3D properties of objects in the next moment by leveraging historical information, thereby alleviating the accuracy degradation of streaming perception. Further, StreamDSGN applies three strategies to enhance the perception accuracy: (1) A feature-flow-based fusion method, which generates a pseudo-next feature at the current moment to address the misalignment issue between feature and ground truth. (2) An extra regression loss for explicit supervision of object motion consistency in consecutive frames. (3) A large kernel backbone with a large receptive field for effectively capturing long-range spatial contextual features caused by changes in object positions. Experiments on the KITTI Tracking dataset show that, compared with the strong baseline, StreamDSGN significantly improves the streaming average precision by up to 4.33%. Our code is available at https://github.com/weiyangdaren/streamDSGN-pytorch.