Mobile robots are being used on a large scale in various crowded situations and become part of our society. The socially acceptable navigation behavior of a mobile robot with individual human consideration is an essential requirement for scalable applications and human acceptance. Deep Reinforcement Learning (DRL) approaches are recently used to learn a robot’s navigation policy and to model the complex interactions between robots and humans. We propose to divide existing DRL-based navigation approaches based on the robot’s exhibited social behavior and distinguish between social collision avoidance with a lack of social behavior and socially aware approaches with explicit predefined social behavior. In addition, we propose a novel socially integrated navigation approach where the robot’s social behavior is adaptive and emerges from the interaction with humans. The formulation of our approach is derived from a sociological definition, which states that social acting is oriented toward the acting of others. The DRL policy is trained in an environment where other agents interact socially integrated and reward the robot’s behavior individually. The simulation results indicate that the proposed socially integrated navigation approach outperforms a socially aware approach in terms of ego navigation performance while significantly reducing the negative impact on all agents within the environment.

Robust and performant controllers are essential for industrial applications. However, deriving controller parameters for complex and nonlinear systems is challenging and time-consuming. To facilitate automatic controller parametrization, this work presents a novel approach using deep reinforcement learning (DRL) with N-dimensional B-spline geometries (BSGs). We focus on the control of parameter-variant systems, a class of systems with complex behavior which depends on the operating conditions. For this system class, gain-scheduling control structures are widely used in applications across industries due to well-known design principles. Facilitating the expensive controller parametrization task regarding these control structures, we deploy an DRL agent. Based on control system observations, the agent autonomously decides how to adapt the controller parameters. We make the adaptation process more efficient by introducing BSGs to map the controller parameters which may depend on numerous operating conditions. To preprocess time-series data and extract a fixed-length feature vector, we use a long short-term memory (LSTM) neural networks. Furthermore, this work contributes actor regularizations that are relevant to real-world environments which differ from training. Accordingly, we apply dropout layer normalization to the actor and critic networks of the truncated quantile critic (TQC) algorithm. To show our approach's working principle and effectiveness, we train and evaluate the DRL agent on the parametrization task of an industrial control structure with parameter lookup tables.

This work presents an approach for parameter estimation of nonlinear systems by means of robust maximum entropy offline reinforcement learning (RL). The identification of parameter variant systems is a challenging problem in industrial applications. We address the parameter estimation on disturbed measurements with an actor-critic RL agent that is extended by a Lyapunov neural network. Accordingly, a robust soft actor-critic algorithm (RSAC) is applied to the parametrization problem. The policy is learned on test trajectories and can be applied to identify nonlinear system parameter maps for comparable system dynamics across an operating range. In a simulative study, the performance of the proposed concept is shown for a permanent magnet synchronous machine model in the d/q-frame formulation with current dependent and nonlinear flux linkages. The trained RL agent is evaluated on a different nonlinear machine parameter set under noisy measurements. The results indicate applicability to real-world tasks such as system parameter tracking.

The increasing need to slow down climate change for environmental protection demands further advancements toward regenerative energy and sustainable mobility. While individual mobility applications are assumed to be satisfied with improving battery electric vehicles (BEVs), the growing sector of freight transport and heavy-duty applications requires alternative solutions to meet the requirements of long ranges and high payloads. Fuel cell hybrid electric vehicles (FCHEVs) emerge as a capable technology for high-energy applications. This technology comprises a fuel cell system (FCS) for energy supply combined with buffering energy storages, such as batteries or ultracapacitors. In this article, recent successful developments regarding FCHEVs in various heavy-duty applications are presented. Subsequently, an overview of the FCHEV drivetrain, its main components, and different topologies with an emphasis on heavy-duty trucks is given. In order to enable system layout optimization and energy management strategy (EMS) design, functionality and modeling approaches for the FCS, battery, ultracapacitor, and further relevant subsystems are briefly described. Afterward, common methodologies for EMSs are structured, presenting a new taxonomy for dynamic optimization-based EMSs from a control engineering perspective. Finally, the findings lead to a guideline toward holistic EMSs, encouraging the co-optimization of system design, and EMS development for FCHEVs. For the EMS, we propose a layered model predictive control (MPC) approach, which takes velocity planning, the mitigation of degradation effects, and the auxiliaries into account simultaneously.