This article develops a model-based reinforcement learning (RL) approach to the closed-loop control of nonlinear dynamical systems with a partial nonlinear observation model. We propose an "information-state"-based approach to rigorously transform the partially observed problem into a fully observed problem where the information state consists of the past several observations and control inputs. We further show the equivalence of the transformed and the initial partially observed optimal control problems and provide the conditions to solve for the deterministic optimal solution. We develop a data-based generalization of the iterative linear quadratic regulator (ILQR) for the RL of partially observed systems using a local linear time-varying model of the information-state dynamics approximated by an autoregressive-moving-average (ARMA) model that is generated using only the input-output data. This approach allows us to design a local perturbation feedback control law that provides an optimum solution to the partially observed feedback design problem locally. The efficacy of the developed method is shown by controlling complex high-dimensional nonlinear dynamical systems in the presence of model and sensing uncertainty.

This paper presents a data-based approach to control robotic systems with partially-observed feedback. First, an open-loop optimization problem is solved to generate the nominal trajectory and then a linear time-varying Autoregressive–Moving-Average (ARMA) model of the system is calculated along the trajectory from the output measurement data. The system is then described in information state, which contains input-output information of the past few steps. Finally, a feedback gain which is calculated by solving a specific LQG problem along the nominal trajectory. The separate design of the open-loop and the closed-loop problem is used following the Decoupled Data-based Control (D2C) approach. Simulation results are also shown for complex models with fluid-structure interaction in the presence of both process and measurement noise.

Abstract Transparent environments and social‐coding platforms as GitHub help developers to stay abreast of changes during the development and maintenance phase of a project. Especially, notification feeds can help developers to learn about relevant changes in other projects. Unfortunately, transparent environments can quickly overwhelm developers with too many notifications, such that they lose the important ones in a sea of noise. Complementing existing prioritization and filtering strategies based on binary compatibility and code ownership, we develop an anomaly detection mechanism to identify unusual commits in a repository, which stand out with respect to other changes in the same repository or by the same developer. Among others, we detect exceptionally large commits, commits at unusual times, and commits touching rarely changed file types given the characteristics of a particular repository or developer. We automatically flag unusual commits on GitHub through a browser plug‐in. In an interactive survey with 173 active GitHub users, rating commits in a project of their interest, we found that, although our unusual score is only a weak predictor of whether developers want to be notified about a commit, information about unusual characteristics of a commit changes how developers regard commits. Our anomaly detection mechanism is a building block for scaling transparent environments.

Feedback control problems pertaining to the control of tensegrity robotic systems are detailed in this paper. The unique problems that arise due to the positivity of the string tensions required to maintain the static stability and desirable stiffness of the structural system are shown to bring about interesting opportunities to optimize for the redundancy in the actuation process. The static stability consideration, coupled with the nonlinear dynamics and the sensor models introduce additional algebraic constraints in implementation of both kinematic and model based dynamic controllers for tensegrity systems. Approaches to develop kinematic and dynamic control techniques are detailed in this paper. A bench top experimental setup consisting of a simple tensegrity system is utilized to demonstrate the efficacy of the output feedback control approach developed in the paper. Near real time image measurements are utilized to drive the output error used in the control scheme.