This article develops a model-based reinforcement learning (RL) approach to the closed-loop control of nonlinear dynamical systems with a partial nonlinear observation model. We propose an "information-state"-based approach to rigorously transform the partially observed problem into a fully observed problem where the information state consists of the past several observations and control inputs. We further show the equivalence of the transformed and the initial partially observed optimal control problems and provide the conditions to solve for the deterministic optimal solution. We develop a data-based generalization of the iterative linear quadratic regulator (ILQR) for the RL of partially observed systems using a local linear time-varying model of the information-state dynamics approximated by an autoregressive-moving-average (ARMA) model that is generated using only the input-output data. This approach allows us to design a local perturbation feedback control law that provides an optimum solution to the partially observed feedback design problem locally. The efficacy of the developed method is shown by controlling complex high-dimensional nonlinear dynamical systems in the presence of model and sensing uncertainty.

This article presents and studies a recursive information consensus filter for decentralized dynamic state estimation under circumstances in which the communication network is unreliable. Local estimators are assumed to have access only to local information, and no structure is assumed about the topology of the communication network, which need not be connected at all times. The filter is a hybrid approach: it uses iterative covariance intersection to reach consensus over priors, which might become correlated, while consensus over new information is handled using weights based on a Metropolis–Hastings Markov chain. We establish bounds for estimation performance and show that this hybrid method produces unbiased conservative estimates that are better than covariance intersection. The performance of the hybrid method is evaluated extensively, including comparisons with competing algorithms, with a hypothetical “full history” yardstick, and centralized performance. We conduct an assessment on a realistic atmospheric dispersion problem and also on more carefully crafted settings to help characterize particular aspects of the performance.