In this paper, we present an extension to the recursive Gaussian Process (RGP) regression that enables the satisfaction of inequality constraints and is well suited for a real-time execution in control applications. The soft inequality constraints are integrated by introducing an additional extended Kalman Filter (EKF) update step using pseudo-measurements. The sequential formulation of the algorithm and several developed heuristics ensure both the performance and a low computational effort of the algorithm. A special focus lies on an efficient consideration of monotonicity assumptions for GPs in the form of inequality constraints. The algorithm is statistically validated in simulations, where the possible advantages in comparison with the standard RGP algorithm become obvious. The paper is concluded with a successful experimental validation of the developed algorithm for the monotonicity-preserving learning of heat transfer values for the control of a vapor compression cycle evaporator, leveraging a previously published partial input output linearization (IOL).

Parallelization and vectorization strategies play a key role in accelerating the training of Deep Reinforcement Learning agents, for example by utilizing several agents simultaneously or by operating multiple independent training environments in parallel. Focusing on the latter approach, this work investigates the impact of Vectorized Environments on the training speed and performance for a collision-free path planning task with the seven-degree-of-freedom Franka Research 3 robotic manipulator. Using the model-free, off-policy Reinforcement Learning algorithm Twin Delayed Deep Deterministic Policy Gradient, the results show a tremendous potential for improving the overall performance and reducing the wall-clock training time when collecting experiences from multiple different environments at the same time. As it can be seen, a simultaneous increase of the mini-batch size can lead to more generalized agents and an improved training stability. Finally, the effect of true parallelization, i.e., distributing the individual environments onto separate workers on the work station, is illustrated, proving an option for an even further training speed-up.

In many applications of robotic path planning, several obstacles must be avoided simultaneously while their relative importance for collision avoidance may vary over time. By pure intuition, obstacles that are of major importance in the current time step shall be respected more strictly to avoid potential collisions. Using the model-free, actor-critic Reinforcement Learning algorithm Twin Delayed Deep Deterministic Policy Gradient, the impact of a dynamically sorted environmental state description for collision-free path planning tasks incorporating multiple obstacles with the 7 degrees of freedom Franka Research 3 robotic manipulator is investigated. The influence of this approach is analyzed in a sensitivity analysis of the trained actor network, where the determined allocation of the network’s input state vector is found to have significant impact onto the action selection. The training results further indicate a tremendous benefit when consistently sorting the state vector during operation according to the minimum Euclidean distances of the obstacles to the robot.

This paper proposes a general control approach for first-order input-affine systems of arbitrary finite system order. The control scheme, which is based on an inversion of the system dynamics in combination with an adaptation of the reference values, guarantees close-to-optimal tracking of systems with input constraints. After the presentation of the control scheme for the SISO case, stability, performance and robustness properties are investigated. Then, a first application in the form of a cooling cycle is presented in which the proposed approach shows a nearly identical performance to an alternative offline optimisation. The control scheme is extended to MIMO systems by means of a prioritization of outputs and an online solution of linear programming problems. For this scenario, a nonlinear real-world example is discussed in the form of a vapor compression cycle. The achieved performance of the control scheme is demonstrated by simulation results.

This paper presents an innovative model-based control approach of a vapor compression cycle, which relies on a partial input-output linearisation (IOL) for a nonlinear feedback control of the evaporator outlet enthalpy. First, the test rig and the corresponding control-oriented system model are presented. For the inner control loop, the IOL is derived and used for the design of a nonlinear feedback control. Moreover, different extensions are discussed, among them an integral error feedback by means of the heat transfer value and a nonlinear stabilizing feedback control. For the control of the coolant outlet temperature in the outer loop, a PI-controller in combination with an inversion-based feedforward control is employed. The partial IOL-control is tested separately, and the effects of the adaptations are compared in experiments on the test rig. Here, the integral feedback by means of the heat-transfer value improves the control performance significantly, whereas the nonlinear stabilizing feedback contributes to a reduction of the necessary control effort. The overall control approach is validated experimentally on the test rig. Especially the partial IOL-control in the inner loop provides very promising results.

In robotic path planning tasks, Reinforcement Learning agents typically receive global or relative Euclidean coordinates, e.g., with respect to a target reference point as direct state information. Nevertheless, a more egocentric view of the environment seems to be favorable – based on information in polar or spherical coordinates about objects surrounding the robot. Using the model-free, actor-critic algorithm Twin Delayed Deep Deterministic Policy Gradient in combination with Prioritized Experience Replay, the advantages of an alternative definition of states using egocentric TCP-coordinates is evaluated and compared in simulations to classical approaches within two typical environments. The training results indicate a tremendous potential of the egocentric state space definition that not only offers faster learning but also more successful trainings.

This paper presents the design and a comparison of two cascaded control schemes based on nonlinear model predictive controllers (NMPC) for the position control of a pneumatic actuator. Both control schemes consist of a NMPC for the pneumatic subsystem while the outer mechanical subsystem is in one case controlled by a NMPC as well, and in the other case by a state-feedback controller (LQR). To cope with the high impact of nonlinear friction, a feedforward friction compensation is employed, and the mechanical subsystem is extended by an integrator. The control approaches are implemented on a test rig consisting of a double-acting pneumatic cylinder actuated by a single 5/3-proportional valve using the GRAMPC-toolbox. Based on experimental results on the test rig it can be shown that both controllers achieve a precise position tracking for different desired trajectories.