In this paper, we propose a multispectral deep fusion network with distance estimation to ensure the safety of persons. Autonomous mobile robots (AMRs) have to detect persons in all weather and lighting conditions. By fusing sensors in the thermal and visible spectrum, fewer persons are missed. We introduce a detail module, which dynamically adjusts the weights of the modalities depending on the information density of the RGB and thermal images. We evaluate the proposed multispectral fusion method on the FLIR dataset. If the distance to the detected person is unknown, the AMR cannot judge whether it needs to brake or adjust its path. We therefore propose a distance estimation, which we evaluate on the KITTI dataset. To evaluate the multispectral fusion method as well as the distance estimation, we recorded and labelled a dataset from the perspective of an AMR in the visible and thermal spectrum with distance information for each object. On this dataset, we evaluate the performance of our network and discuss the number of dangerous failures for functional safety applications, and thereby, contribute to safe outdoor applications.

Autonomous mobile robots are increasingly used in pedestrian-rich environments where safe navigation and appropriate human interaction are crucial. While Deep Reinforcement Learning (DRL) enables socially integrated robot behavior, challenges persist in novel or perturbed scenarios to indicate when and why the policy is uncertain. Unknown uncertainty in decision-making can lead to collisions or human discomfort and is one reason why safe and risk-aware navigation is still an open problem. This work introduces a novel approach that integrates aleatoric, epistemic, and predictive uncertainty estimation into a DRL navigation framework for policy distribution uncertainty estimates. We, therefore, incorporate Observation-Dependent Variance (ODV) and dropout into the Proximal Policy Optimization (PPO) algorithm. For different types of perturbations, we compare the ability of deep ensembles and Monte-Carlo dropout (MC-dropout) to estimate the uncertainties of the policy. In uncertain decision-making situations, we propose to change the robot’s social behavior to conservative collision avoidance. The results show improved training performance with ODV and dropout in PPO and reveal that the training scenario has an impact on the generalization. In addition, MC-dropout is more sensitive to perturbations and correlates the uncertainty type to the perturbation better. With the safe action selection, the robot can navigate in perturbed environments with fewer collisions.

The application of radar sensors for motion estimation has recently been discussed in the research community. However, challenging environments such as garages and tunnels can lead to erroneous motion estimation results. Since the estimate serves as an input for trajectory control in automated driving functions, it can pose a safety hazard, e.g., if undesired acceleration is applied. This work presents a framework to address unsafe controller actions caused by motion estimation failures. A monitoring module continuously observes controller actions and feeds back critically evaluated measures to the motion estimation module. We propose three additional algorithms that adapt both the estimation module and its input data-namely, the filtered radar point cloud. First, a method that incorporates previous motion states into the point cloud filtering process. Second, a ridge regression algorithm that generates alternative estimates and compensates for erroneous conclusions arising from an unfavorably selected point set. Third, an adapted cluster selection approach that increases the number of true positive detection points. Experiments show that critical estimates can be identified in most cases, with a success rate of 97.7%. Additionally, we found that the estimation process becomes significantly more robust. In summary, this work emphasizes the importance of safe motion state measurement for the operation of automated vehicles. It introduces a method for determining the criticality of motion states and presents approaches to mitigate erroneous estimations.

Radar sensors are superior at measuring distances and velocities, even in adverse weather conditions. However, the noisiness of radar point clouds requires a filtering cascade to make these sensors usable for applications in automated driving. This filtering may cause hazardous situations if points on existing objects are removed by a filter. To serve as a perception component in fully automated driving systems, radar’s error rate needs to be reduced significantly. To achieve this goal, we advance the view that points should be filtered adaptively according to their presumable relevance for the current driving task. In this work, we present a real-world study of our radar point criticality estimation algorithm. In addition, we introduce methods for acting on critical points. While the Posterior method recalls points in critical regions, we demonstrate how the creation of clusters and tracks can be facilitated for critical radar points. Our methods are evaluated on a novel dataset including 92 critical scenes with pedestrians in a parking garage. We demonstrate that all methods significantly increase detection rates at both the cluster and track levels and enable earlier detection. However, utilizing every radar point inevitably leads to several false positives. Future work should investigate how to invalidate these points by additional perception sensors.

Task performance in terms of task completion time in teleoperation is still far behind compared to humans conducting tasks directly. One large identified impact on this is the human capability to perform transformations and alignments, which is directly influenced by the point of view and the motion retargeting strategy. In modern teleoperation systems, motion retargeting is usually implemented through a one time calibration or switching modes. Complex tasks, like concatenated screwing, might be difficult, because the operator has to align (e.g. mirror) rotational and translational input commands. Recent research has shown, that the separation of translation and rotation leads to increased task performance. This work proposes a formal motion retargeting method, which separates translational and rotational input commands. This method is then included in a optimal control based trajectory planner and shown to work on a UR5e manipulator.

Autonomous mobile robots (AMRs) are used in industrial indoor applications for heavy loads. However, due to the lack of safety, only lightweight AMRs can be used for outdoor applications. In indoor applications, the binary information whether an object is in the protective field is sufficient and is often determined through use of a safety certified 2D scanner. AMRs intended for outdoor use must be able to traverse obstacles such as curbs and continue driving even if greenery, rainfall or dust enters the protective field. Therefore, a safe perception system capable of classifying objects is necessary. This perception system must function reliably under all weather and lighting conditions, be cost- and space-efficient, and can be computed in real time on processing resources with limited capabilities. To achieve this, we propose a method for localising and classifying multiple objects using an ultrasonic array mounted on an AMR.

This paper presents a framework for odometry estimation in automotive application using six asynchronously operating millimeter wave radar sensors and a combination of gyroscope and accelerometer. Two different motion models are combined to estimate motion with three degrees of freedom. For this purpose, we propose a novel three-part radar filtering method for outlier detection: By analyzing uncertainties and system limits, sensor-specific outliers are detected and removed in the first filter. We introduce knowledge about the previous motion state by a status-quo-ante filter and hereby identify further false positive raw targets in the current measure which are not accessible from the previous state. Moreover, we suggest employing a downstream, resampling-based algorithm for additional outlier detection. Based on the filtered data, radar motion state estimation is performed by use of curve fitting methods. To fuse the radar odometry estimation with the acceleration and yaw rate measurements handling non-linearities, an Unscented Kalman Filter is used. The developed framework is evaluated with reference data in various scenarios. The results demonstrate that it accurately and robustly determines motion and position states even in radar-challenging scenes, such as environments with few radar targets or with heavy metal structures. Our method keeps up with common approaches such as wheel speed sensor odometry while outperforming it in terms of drift-impairment.

Mobile robots are being used on a large scale in various crowded situations and become part of our society. The socially acceptable navigation behavior of a mobile robot with individual human consideration is an essential requirement for scalable applications and human acceptance. Deep Reinforcement Learning (DRL) approaches are recently used to learn a robot’s navigation policy and to model the complex interactions between robots and humans. We propose to divide existing DRL-based navigation approaches based on the robot’s exhibited social behavior and distinguish between social collision avoidance with a lack of social behavior and socially aware approaches with explicit predefined social behavior. In addition, we propose a novel socially integrated navigation approach where the robot’s social behavior is adaptive and emerges from the interaction with humans. The formulation of our approach is derived from a sociological definition, which states that social acting is oriented toward the acting of others. The DRL policy is trained in an environment where other agents interact socially integrated and reward the robot’s behavior individually. The simulation results indicate that the proposed socially integrated navigation approach outperforms a socially aware approach in terms of ego navigation performance while significantly reducing the negative impact on all agents within the environment.

The physical coupling between robots has the potential to improve the capabilities of multi-robot systems in challenging manufacturing processes. However, the path tracking accuracy of physically coupled robots is not studied adequately, especially considering the uncertain kinematic parameters, the mechanical elasticity, and the built-in controllers of off-the-shelf robots. This paper addresses these issues with a novel differential-algebraic system model which is verified against measurement data from real execution. The uncertain kinematic parameters are estimated online to adapt the model. Consequently, an adaptive model predictive controller is designed as a coordinator between the robots. The controller achieves a path tracking error reduction of 88.6% compared to the state-of-the-art benchmark in the simulation.

Simultaneous localization and mapping is a prevalent method for localization in automated parking applications. However, it requires to access an area for exploring purposes before the automated parking function can be completely applied. As this is inconvenient for parking functions, where often unknown areas are entered, our approach proposes a radar-based localization method primarily for applications inside of buildings which have a ground plan available. This ground plan contains the walls, pillars and parking lots of the building in a two-dimensional, bird’s eye view perspective. Based on the ground plan, a synthesized point cloud is generated to be matched with the filtered radar point cloud via a Normal Distributions Transform algorithm. The measurements generated hereby are fused with odometry measurements in a factor graph. This architecture is capable of processing independent, asynchronous incoming data in parallel and can easily be extended, e.g., by camera data. We outline our pipeline and show in experiments that it serves as a solid basis which competes with other state-of-the-art localization algorithms. Some drawbacks, e.g., the noisiness of the radar data in slow-speed or standstill situations, are discussed. Future work could incorporate camera data to further improve the robustness of this approach.

During the use of Advanced Driver Assistance Systems (ADAS), drivers can intervene in the active function and take back control due to various reasons. However, the specific reasons for driver-initiated takeovers in naturalistic driving are still not well understood. In order to get more information on the reasons behind these takeovers, a test group study was conducted. There, 17 participants used a predictive longitudinal driving function for their daily commutes and annotated the reasons for their takeovers during active function use. In this paper, the recorded takeovers are analyzed and the different reasons for them are highlighted. The results show that the reasons can be divided into three main categories. The most common category consists of takeovers which aim to adjust the behavior of the ADAS within its Operational Design Domain (ODD) in order to better match the drivers’ personal preferences. Other reasons include takeovers due to leaving the ADAS’s ODD and corrections of incorrect sensing state information. Using the questionnaire results of the test group study, it was found that the number and frequency of takeovers especially within the ADAS’s ODD have a significant negative impact on driver satisfaction. Therefore, the driver satisfaction with the ADAS could be increased by adapting its behavior to the drivers’ wishes and thereby lowering the number of takeovers within the ODD. The information contained in the takeover behavior of the drivers could be used as feedback for the ADAS. Finally, it is shown that there are considerable differences in the takeover behavior of different drivers, which shows a need for ADAS individualization.

Physical Human-Machine Interaction plays a pivotal role in facilitating collaboration across various domains. When designing appropriate model-based controllers to assist a human in the interaction, the accuracy of the human model is crucial for the resulting overall behavior of the coupled system. When looking at state-of-the-art control approaches, most methods rely on a deterministic model or no model at all of the human behavior. This poses a gap to the current neuroscientific standard regarding human movement modeling, which uses stochastic optimal control models that include signal-dependent noise processes and therefore describe the human behavior much more accurate than the deterministic counterparts. To close this gap by including these stochastic human models in the control design, we introduce a novel design methodology resulting in a Human-Variability-Respecting Optimal Control that explicitly incorporates the human noise processes and their influence on the mean and variability behavior of a physically coupled human-machine system. Our approach results in an improved overall system performance, i.e. higher accuracy and lower variability in target point reaching, while allowing to shape the joint variability, for example to preserve human natural variability patterns.

In this paper, a method for a cooperative trajectory planning between a human and an automation is extended by a behavioral model of the human. This model can characterize the stubbornness of the human, which measures how strong the human adheres to his preferred trajectory. Accordingly, a static model is introduced indicating a link between the force in haptically coupled human-robot interactions and humans's stubbornness. The introduced stubbornness parameter enables an application-independent reaction of the automation for the cooperative trajectory planning. Simulation results in the context of human-machine cooperation in a care application show that the proposed behavioral model can quantitatively estimate the stubbornness of the interacting human, enabling a more targeted adaptation of the automation to the human behavior.

Abstract Providing high quality solutions is crucial when solving NP‐hard time‐extended multi‐robot task allocation (MRTA) problems. Reoptimization, that is, the concept of making use of a known solution to an optimization problem instance when the solution to a similar problem instance is sought, is a promising and rather new research field in this application domain. However, so far no approximative time‐extended MRTA solution approaches exist for which guarantees on the resulting solution's quality can be given. We investigate the reoptimization problems of inserting as well as deleting a task to/from a time‐extended MRTA problem instance. For both problems, we can give performance guarantees in the form of an upper bound of 2 on the resulting approximation ratio for all heuristics fulfilling a mild assumption. We furthermore introduce specific solution heuristics and prove that smaller and tight upper bounds on the approximation ratio can be given for these heuristics if only temporal unconstrained tasks and homogeneous groups of robots are considered. A conclusory evaluation of the reoptimization heuristic demonstrates a near‐to‐optimal performance in application.

While robotic arms have been used in a vast range of application areas, so far no extensive reports on the utilization as human-machine interface exist. Compared to HMI devices from literature, the robotic arm used in this work (KUKA LBR iiwa 14 R820) features a relatively large workspace and is able to generate force and torque feedback that surpasses the capabilities of literature devices. We describe the setup allowing to use the robotic arm as HMI and analytically determine the optimal initial pose of it based on the manipulability measure of Yoshikawa. To demonstrate that the robotic arm is able to serve as HMI, we report on a comparative study with a state of the art haptic HMI featuring 20 participants. Additionally, two applications from the context of planetary exploration are presented: The first considers the teleoperation of the pan-tilt unit of a lightweight rover unit and illustrates how the large workspace of the HMI benefits the precision of the teleoperation compared to a setup with a smaller workspace. The second experiment showcases the use of the force feedback of the HMI to enable a cooperation between the operator and a supporting path-following automation in a shared control of a simulated ground robot. Both the study and the applications highlight the performance, precision and reliability of our proposed system.

Telemanipulation has become a promising technology that combines human intelligence with robotic capabilities to perform tasks remotely. However, it faces several challenges such as insufficient transparency, low immersion, and limited feedback to the human operator. Moreover, the high cost of haptic interfaces is a major limitation for the application of telemanipulation in various fields, including elder care, where our research is focused. To address these challenges, this paper proposes the usage of nonlinear model predictive control for telemanipulation using low-cost virtual reality controllers, including multiple control goals in the objective function. The framework utilizes models for human input prediction and task-related models of the robot and the environment. The proposed framework is validated on an UR5e robot arm in the scenario of handling liquid without spilling. Further extensions of the framework such as pouring assistance and collision avoidance can easily be included.

With human-machine interaction continuously developing, the consideration of human experience in the design of a machine becomes increasingly relevant. An experience measure that rates how well a work or interaction state is perceived by a user is psychological flow. In this paper we introduce a novel approach to assess flow in human-machine interaction by game-theoretically modeling the interaction. To validate our approach, we perform a user study with 30 participant with a study design that allows for the manipulation of the user's experience through three experience modes. The results both validate our study design and provide indications that our approach to assess flow is promising to be developed further.

Adjusting the level of autonomy in human-machine systems (e.g., human-robot systems) holds great potential for achieving high system performance while maintaining operator involvement. To support operators with the task of setting the proper level of autonomy, we present a novel approach to realise a Model Predictive Controller that determines the optimal LoA for each tessellation in the robot's path plan based on the estimated performance degradation due environmental adversities. We also report on an experimental evaluation of a mixed-initiative system where both the operator and the Model Predictive Controller are in charge of dynamically adjusting the level of autonomy cooperatively while performing a challenging navigational task with a mobile ground robot in a high-fidelity simulation. To this end, we conducted a user study with 15 participants comparing the performance and user experience of the model predictive system with a state-of-the-art system. The results show significant benefits of the model predictive system in terms of a reduction of conflicts for control and an improved user experience. Additionally, there are indications of benefits in terms of robot health and, consequently, performance for the model predictive system.

Robust and performant controllers are essential for industrial applications. However, deriving controller parameters for complex and nonlinear systems is challenging and time-consuming. To facilitate automatic controller parametrization, this work presents a novel approach using deep reinforcement learning (DRL) with N-dimensional B-spline geometries (BSGs). We focus on the control of parameter-variant systems, a class of systems with complex behavior which depends on the operating conditions. For this system class, gain-scheduling control structures are widely used in applications across industries due to well-known design principles. Facilitating the expensive controller parametrization task regarding these control structures, we deploy an DRL agent. Based on control system observations, the agent autonomously decides how to adapt the controller parameters. We make the adaptation process more efficient by introducing BSGs to map the controller parameters which may depend on numerous operating conditions. To preprocess time-series data and extract a fixed-length feature vector, we use a long short-term memory (LSTM) neural networks. Furthermore, this work contributes actor regularizations that are relevant to real-world environments which differ from training. Accordingly, we apply dropout layer normalization to the actor and critic networks of the truncated quantile critic (TQC) algorithm. To show our approach's working principle and effectiveness, we train and evaluate the DRL agent on the parametrization task of an industrial control structure with parameter lookup tables.

The experiment reported in this article provides a first experimental evaluation of human–machine cooperation on decision level: It explicitly focuses on the interaction of human and machine in cooperative decision-making situations for which a suitable experimental design is introduced. Furthermore, it challenges conventional leader–follower approaches by comparing them to newly proposed automation designs based on cooperative decision-making models. These models originate from negotiation theory and game theory and allow for an investigation of cooperative decision making between equal partners. This equality is motivated by similar approaches on the action level of human–machine cooperation. The experiment's results indicate an added value of the proposed automation designs in terms of objective cooperative performance as well as human trust in and satisfaction with the cooperation. Hence, the experiment yields the same insight on decision level as already observed on action level: It may be beneficial to design machines as equal cooperation partners and in accordance to models of emancipated human–machine cooperation.

This article presents a systematic method for the design of a limited information shared control (LISC). LISC is used in applications where not all system states or references trajectories are measurable by the automation. Typical examples are partially human controlled systems, in which some subsystems are fully controlled by the automation, whereas others are controlled by a human. The proposed systematic design uses a novel class of games to model human–machine interaction: the near potential differential games (NPDG). We provide a necessary and sufficient condition for the existence of an NPDG and derive an algorithm for finding a NPDG, which completely describes a given differential game. The proposed design method is applied to the control of a large vehicle manipulator system, in which the manipulator is controlled by the human operator and the vehicle is fully automated. The suitability of the NPDG modeling differential games is verified in simulations leading to a faster and more accurate controller design compared with manual tuning. Furthermore, the overall design process is validated in a study with 16 test subjects indicating the applicability of the proposed concept in real applications.

This work presents an approach for parameter estimation of nonlinear systems by means of robust maximum entropy offline reinforcement learning (RL). The identification of parameter variant systems is a challenging problem in industrial applications. We address the parameter estimation on disturbed measurements with an actor-critic RL agent that is extended by a Lyapunov neural network. Accordingly, a robust soft actor-critic algorithm (RSAC) is applied to the parametrization problem. The policy is learned on test trajectories and can be applied to identify nonlinear system parameter maps for comparable system dynamics across an operating range. In a simulative study, the performance of the proposed concept is shown for a permanent magnet synchronous machine model in the d/q-frame formulation with current dependent and nonlinear flux linkages. The trained RL agent is evaluated on a different nonlinear machine parameter set under noisy measurements. The results indicate applicability to real-world tasks such as system parameter tracking.

The cooperation between humans and robots is of great importance e.g. in medical, industrial or service applications. Here, the task to be pursued by the robot often depends on the current goal of the human. In cases where a direct communication of the human’s goal is impractical or even impossible, an estimation of the human’s goal is necessary. This estimation as well as potential process or measurement noise introduces uncertainty that needs to be taken into consideration during the planning of the robot’s actions. To this end, we propose an automation comprising an Unscented Kalman filter as state estimator, a model based intention recognition algorithm to estimate the human’s goal and a model predictive belief space controller based on Belief i-LQG explicitly considering the estimation uncertainty. We report on a simulated scenario featuring a mobile robot platform cooperating with a human. It demonstrates the ability of the proposed automation to actively reduce uncertainty about the system states and the human’s goal while successfully pursuing the overall cooperative task.

This paper addresses the problem of transfer of control authority between a robot’s AI and a remote human operator, when controlling a Mixed-Initiative (MI) robotic system. We propose a negotiation-theoretic method that enables the robot’s AI and the human operator to cooperatively and dynamically determine (i. e. negotiate) the transfer of control authority between these two agents. An experimental study is presented in which a state-of-the-art Expert-guided Mixed-Initiative Control Switcher (EMICS) method is compared with our proposed Negotiation-Enabled Mixed-Initiative Control Switcher (NEMICS) algorithm. Results suggest that the NEMICS framework is able to successfully avoid conflicts for control, which is a fundamental challenge encountered with previous MI control methods. Comparing NEMICS with the EMICS, we provide evidence of improved navigational safety (i. e. fewer collisions). Additionally, our usability study suggests that human operators perceived their interactions with NEMICS as less intrusive than with EMICS.

This paper presents the validation and the comparative study of a shared control concept for a large vehicle manipulator (LVM). The state-of-the-art controlling a LVM is manual control: The operator controls the manipulator to carry out a specific task and keeps the vehicle on the road. Easing the work for the operator, an automatic lane-keeping of the vehicle can be taken into account: An automation of the vehicle which keeps it on its reference, but without taking into consideration of the manipulator’s specific task. However, the operator has his specific task with the manipulator, and therefore, such automation may not be satisfying. Therefore, this paper presents the validation and compares the Limited Information Shared Controller (LISC) proposed previously with the manual control mode. This step is crucial, showing the concept’s applicability and benefits compared to the state-of-the-art solution. Thus, the LISC is compared with a non-cooperative controller (NCC) and the manual mode on a real-time simulator with test subjects. It has a more realistic experimental setup than in other studies because there is no predefined manipulator reference. The study results indicate that the NCC can lead to undesired motions of the overall system because the test subjects cannot carry out their specific task. On the other hand, the proposed the LISC of the vehicle can reduce the working load while supporting the operator in carrying out the manipulator’s specific task.

Reinforcement Learning is a promising method for automated tuning of controllers, but is yet rarely applied to real systems like longitudinal vehicle control, since it struggles in the face of real-time tasks, noise, partially observed dynamics and delays. We propose a model-based reinforcement learning algorithm for the task of speed tracking control on constrained hardware. In order to cope with partially observed dynamics, delay and noise our algorithm relies on an autoregressive model with external inputs (ARX model) that is learned using a decaying step size. The output controller is updated by policy search on the learned model. Multiple experiments show that the proposed algorithm is capable of learning a controller in a real vehicle in different speed ranges and with a variety of exploration noise distribution and amplitudes. The results show that the proposed approach yields similar results to a recently published model-free reinforcement learning method in most conditions, e.g, when adapting the controller to very low speeds, but succeeds to learn with a wider variety of exploration noise types.

The coordination of heterogeneous robotic teams demands suitable planning algorithms based on an appropriate model of the problem instance. While there exists a great variety of automated planning algorithms, the modeling of a problem instance often requires expertise in problem recognition and faculty of abstraction—a task which can be done best by humans. In order to exploit the synergy potential inherent in human-machine cooperative planning, we propose a new reoptimization framework based on a genetic algorithm (GA) for heterogeneous multi-robot task allocation problems including cooperative tasks and precedence constraints. The main idea of the reoptimization framework is to reuse insights from previous solutions of similar problem instances. In particular, a modified problem instance, resulting for example from adding or deleting individual tasks, is solved based on the solution of the unmodified problem instance. To this end, we introduce suitable heuristics for the adaption of the initial solution based on the considered problem modification. The simulative investigation of the proposed approach shows great positive effects compared to the application of a standardized GA that does not make use of the solution of the unmodified problem instance.

In this contribution, an approach for modeling driving behavior in intersection scenarios, based on a hybrid dynamic game framework, is presented. Using the hybrid system model, the movement of the traffic agent is divided into maneuvers. Therefore, the decision-making process is a maneuver selection problem having reduced complexity compared to trajectory planning. While previous models used maneuver description with constant acceleration values, the presented approach models the maneuvers on a more macroscopic level using the well known Intelligent Driver Model. This has the advantage of creating more realistic acceleration profiles without increasing the computational complexity of the model. The maneuver selection process is modeled using the nash equilibrium concept of game theory. The resulting coupled optimization problem for each player is solved using an iterated best response algorithm determining the nash equilibrium. Finally, using simulation examples, it is shown that the presented model is capable of creating a variety of scenarios using different parameter sets as well as simulating scenarios with a high number of traffic participants.

In recent years, the amount of electric and electronic components in the automotive power network (APN) has increased significantly. With regard to autonomous driving, telecommunication, infotainment, and other comfort functionalities, a further growing complexity in the APN is emerging. Additionally, the still requested customization of cars leads to more variants, models, and decoupled development cycles in terms of software and hardware. To ensure a flexible APN design, this paper demonstrates a modular approach for the power management combining auction theory and service-oriented architecture (SOA) for APNs with multiple voltage levels. The presented algorithm facilitates distributed decision-making by the growing number of components in the network regarding their power consumption. Hence, the individual components adapt their behavior in terms of component states and the APN condition. Furthermore, the SOA paradigm ensures the scalability and fault tolerance of the proposed algorithm.

Large vehicle-manipulators are systems consisting of a medium-sized heavy-duty vehicle and a hydraulic manipulator. They operate in an unstructured environment and are therefore not fully automated. However, semi-automation of such a system is possible, where the automation controls the vehicle and a human operator controls the manipulator. Since in the unstructured environment not all system trajectories are measurable for the automation, a so-called Limited Information Cooperative Shared-Controller (LICSC) has been proposed in previous work. However, the design of the LICSC is done through tuning which is sensitive to the operator controlling the manipulator. The first contribution of this work is to reduce this sensitivity. A novel personalized design of the LICSC is presented that provides control tailored to the human operator. Our proposed approach uses state-of-the-art design methods of a cooperative controller to obtain the LICSC. The second contribution is the experimental validation of the LICSC, which proves the importance of the personalization to the human operator and demonstrates the advantage of the method.

The increasing need to slow down climate change for environmental protection demands further advancements toward regenerative energy and sustainable mobility. While individual mobility applications are assumed to be satisfied with improving battery electric vehicles (BEVs), the growing sector of freight transport and heavy-duty applications requires alternative solutions to meet the requirements of long ranges and high payloads. Fuel cell hybrid electric vehicles (FCHEVs) emerge as a capable technology for high-energy applications. This technology comprises a fuel cell system (FCS) for energy supply combined with buffering energy storages, such as batteries or ultracapacitors. In this article, recent successful developments regarding FCHEVs in various heavy-duty applications are presented. Subsequently, an overview of the FCHEV drivetrain, its main components, and different topologies with an emphasis on heavy-duty trucks is given. In order to enable system layout optimization and energy management strategy (EMS) design, functionality and modeling approaches for the FCS, battery, ultracapacitor, and further relevant subsystems are briefly described. Afterward, common methodologies for EMSs are structured, presenting a new taxonomy for dynamic optimization-based EMSs from a control engineering perspective. Finally, the findings lead to a guideline toward holistic EMSs, encouraging the co-optimization of system design, and EMS development for FCHEVs. For the EMS, we propose a layered model predictive control (MPC) approach, which takes velocity planning, the mitigation of degradation effects, and the auxiliaries into account simultaneously.

In order to fully exploit the advantages inherent to cooperating heterogeneous multi-robot teams, sophisticated coordination algorithms are essential. Time-extended multi-robot task allocation approaches assign and schedule a set of tasks to a group of robots such that certain objectives are optimized and operational constraints are met. This is particularly challenging if cooperative tasks, i.e. tasks that require two or more robots to work directly together, are considered. In this paper, we present an easy-to-implement criterion to validate the feasibility, i.e. executability, of solutions to time-extended multi-robot task allocation problems with cross schedule dependencies arising from the consideration of cooperative tasks and precedence constraints. Using the introduced feasibility criterion, we propose a local improvement heuristic based on a neighborhood operator for the problem class under consideration. The initial solution is obtained by a greedy constructive heuristic. Both methods use a generalized cost structure and are therefore able to handle various objective function instances. We evaluate the proposed approach using test scenarios of different problem sizes, all comprising the complexity aspects of the regarded problem. The simulation results illustrate the improvement potential arising from the application of the local improvement heuristic.

In order to support a human in a human-machine system, the cooperating automation requires information about the goal pursued by the human. We model human-machine systems as a Partially Observable Markov Decision Process to develop an assistant system operating on maneuver or navigation level featuring an automatic detection of the humans goal which is initially unknown to it. Predicting the next actions of the human allows for computing corresponding assistant actions by employing Partially Observable Monte-Carlo Planning With Observation Widening. These supporting actions are generated based on continuous action-, observation- and state spaces and are executed synchronously to the actions of the human. New information about the humans goal is gathered through observations to further enhance future supporting actions. With a progressing certainty of the goal recognition, the assistant system is increasingly able to assist the human by completing the task both cooperatively or even fully autonomously, thus reducing workload, while always allowing for the human to smoothly in-or decrease their involvement. The assistant system demonstrates its ability to recognize the goal pursued by the human as well as to execute appropriate supporting actions while being robust to the human changing their goal in multiple experiments investigating the interaction of a real human with a simulated cooperative positioning scenario.

A new concept for modeling the behavior and the interaction between drivers at intersections is presented. For this purpose a hybrid dynamic game is introduced using the concepts of game theory. The proposed hybrid game is used to model the interactive behavior in a junction scenario. The presented hybrid approach divides the modeling of motion into individual maneuvers. With this partition of motion, the decision process can be reduced to a simple maneuver selection that takes into account the motion of the vehicle without the need for solving complex coupled differential equations. In order to make computation of a solution feasible we propose a rule based adaption mechanism. Simulation is used to show the applicability of the developed hybrid dynamic game approach.

This paper presents a new robust fault detection and isolation (FDI) strategy for distributed and decentralized systems. Such a system consists of several interconnected subsystems. A novel FDI system architecture is designed for these systems. Each subsystem is assigned to a local fault detector performing a part of the overall fault detection algorithm. Fault isolation is performed in a separate global fault isolator. Consistency-based methods based on state-set observation are used for robust FDI. Moreover, this paper introduces a faster fault isolation. For this, the search-space containing all fault candidates is split. Possible network effects in the communication between the computing units are also considered. A simulation example of a three-tank system is used to illustrate the effectiveness of this FDI strategy.

In the past decade, remarkable research has been done on human-machine cooperation to generate synergies and mutual benefits. However, most research so far only considers the control level of interaction with concepts like haptic shared control. This paper focuses on the emerging research on human-machine cooperation on higher levels of interaction to tackle more complex challenges. Therefore, we first introduce a generalized level model based on established models to define our research emphasis on emancipated human-machine cooperation on all levels. Second, the design and results of a study on human-machine cooperation on decision level are presented. We examine the negotiation behavior of humans in a scenario with discrete decision options and a deadline. The results indicate the validity of a previously proposed model based on negotiation theory to describe the observed human behavior. Additionally, the observed influencing factors on the negotiation behavior are crucial for a proper automation design: adaptation and identification methods are required to enable the automation to take part in an emancipated negotiation with a human.

This paper presents a novel cooperative control algorithm for vehicle-manipulators (VMs) with a human operator. VMs usually operate in unstructured environments, which means that a full automation of the overall system, combing a vehicle and a robotic manipulator is currently very challenging. Therefore, human operator controlled VMs are state-of-the-art. With current developments in autonomous driving, the automation of the vehicle platform is within reach. A cooperative shared control between the autonomous platform and the human controlled manipulator can happen through the coupling motion between the vehicle platform and the manipulator. An autonomous vehicle platform can furthermore be used to support the human operator with the control of the manipulator. However, the future trajectory of the manipulator intended by the human operator is in general not known to the autonomous vehicle. The main question is thus how the autonomous vehicle should act in order to support the human controlled manipulator in following its unknown trajectory. To solve this problem, we propose an approach that characterizes the cooperation and the unknown errors with an algebraic equation. The novel approach is compared to cooperative control methods with known errors of the manipulator, based on the theory of differential games. The benefits of the proposed method are that no sensors for the environment perception and for the state measurements of the manipulator are necessary, which are demonstrated in simulations.

Model-free control based on the idea of Reinforcement Learning is a promising approach that has recently gained extensive attention. However, Reinforcement-Learning-based control methods solely focus on the regulation problem or learn to track a reference that is generated by a time-invariant exo-system. In the latter case, controllers are only able to track the time-invariant reference dynamics which they have been trained on and need to be re-trained each time the reference dynamics change. Consequently, these methods fail in a number of applications which obviously rely on a trajectory not being generated by an exo-system. One prominent example is autonomous driving . This paper provides for the first time an adaptive optimal control method capable to track arbitrary reference trajectories that are provided on a moving horizon. The main innovation is a novel Q-function that directly incorporates the given reference trajectory. This new Q-function exhibits a particular structure which allows the design of an efficient, iterative, provably convergent Reinforcement Learning algorithm that enables optimal tracking. Two real-world examples demonstrate the effectiveness of our new method.

Haptic shared control systems represent a useful approach for a safer and more intuitive cooperation between human and machines. For an adequate controller design, it is essential to understand the control behavior of a human during the interaction with a cooperation partner. A considerable amount of studies has shown that human behavior in a cooperative scenario is different than in a manual control task, indicating the need for modeling approaches which take the interaction into account. In this paper, we validate an approach to model haptic cooperative behavior of humans based on differential games, where the motion trajectories of each partner arise from the minimization of an individual cost function. We evaluate the applicability of the model for goal-oriented movements by means of an experiment consisting of 26 pairs of subjects moving a virtual marker cooperatively to a goal position. The interaction takes place through haptically coupled steering wheels. Moreover, we compare the differential game approach with another shared control model which considers the action of the cooperating partner as a system disturbance. The results show that the differential game model slightly outperforms the disturbance model in the approximation of observed motion trajectories with statistical significance.

In this paper a new negotiation model is introduced for the design of Advanced Driver Assistance Systems (ADAS). The objective is to establish an emancipated ADAS capable of cooperative decision making. This can be seen as a step from Shared Control to Cooperative Control. Due to the descriptive nature of negotiation theory of cooperative human decision processes, a high human user acceptance is expected if the ADAS is designed accordingly. However, conventional negotiation theory requires adaptation towards the ADAS context. The first extension enables the necessary consideration of a dynamical environment. The second extension is a new asynchronous negotiation protocol, allowing a more realistic human-machine interaction model. Furthermore a new opponent model is introduced to identify human negotiation behavior. This enables the ADAS to adapt itself during the interaction with the human driver, leading to potentially faster negotiations. Our simulation results show that a successful negotiation is possible with the proposed model, motivating further investigations in real applications.

Shared control systems have a great potential to contribute to a safer human-machine interaction. A great body of literature has been concerned with the design of the automation the human is sharing control with. At the same time, an adequate design is connected to the availability of reliable models of human behavior. A promising modeling approach is given by optimal control theory, where human behavior arises from the minimization of a cost function. However, most of the work found in literature focus on determining the cost function of the human in a situation without any haptic interaction with a partner, i.e. in manual control tasks. Motivated by several studies which indicate that human behavior changes when completing a task cooperatively, this paper proposes an optimal control approach for human behavior modeling in a shared control scenario. We further hypothesize that the human cost functions in a shared control scenario change significantly when compared to the ones which arise from a human performing a control task alone. We apply an inverse optimization approach in order to identify the cost function in both scenarios. In order to evaluate our hypothesis, a study was conducted where 42 participants performed a tracking task in a manual mode and then sharing control with an assistance system. The findings show that the model is able to describe human behavior in both shared and manual control. Furthermore, the results confirm that the human cost function changes considerably between both scenarios.

There is a trend towards interconnected and complex dynamical systems that are controlled by more than one controller. Due to the coupling of the controllers by means of the system, these interacting controllers need to consider not only the system dynamics but also the influence of each other. However, in realistic scenarios, they usually do not exchange all the information concerning their parameters and control laws and an exact model of the system dynamics is often hard to obtain. This is why we consider the challenging setting where the controllers have no access neither to the parameters of each other nor to the system dynamics. The controller design is quite difficult in this scenario, as the final system configuration is not known during the design process. In this complex scenario, we propose algorithms where each controller uses Adaptive Dynamic Programming to adapt its control law. Here, each controller strives for reaching its individual control objectives, a setting which can be formulated as a coupled optimization problem, respectively a dynamic game. As an example, we consider a vehicle model with two lateral controllers. With our proposed algorithms, the controllers converge successfully to a solution of the coupled optimization problem without knowing the parameters of each other and the system dynamics.

Until upcoming autonomous vehicles can handle every scenario in all operation domains there will be situations, in which the human driver needs to retrieve control of the vehicle. This can lead to safety issues as literature shows impaired driving performance after taking over from highly automated driving. A discrete switch of control might be a valid option to hand over to an attentive human driver. However, it is questionable if this is a safe way for a transition of control to somebody who is distracted at the time of the takeover request. Instead, gradually shifting control via a period of haptic shared control can provide guidance during the process of establishing situation awareness. Here, we present different design concepts for handing over control from a controller to a human partner. To analyze these concepts, we designed an experiment, in which the participants need to take over a simplified steering task when requested by the automation. We conducted a study comparing two discrete switching transitions and three concepts in which the authority of the controller fades out slowly. The results show that in situations, needing a continuation of the control action of the automation, an early switch-off underperformed compared to the other concepts. If the situation demanded action during the transition the shared control approaches achieved the lowest tracking error. A survey of the participants revealed that gradually shifting transitions lead to a sense of better performance and are also regarded as more comfortable. The best overall results were obtained by a concept which models human behavior and considers it within the takeover process. This approach provides a systematic method for the design of authority transitions by finding the best trade-off between guidance and handover.

This paper contributes to research investigating if the concept of shared control is extendable to higher levels of human-machine cooperation. For this purpose, an experiment is presented analyzing the guidance level in cooperation among humans to transfer these findings in future assistance system design. Within the experiment, pairs of participants were facing a virtual obstacle avoidance course. They had to decide cooperatively on the maneuvers to avoid the obstacles. The only interaction possibility between team members was via haptically coupled steering wheels. The teams were classified based on the decisiveness of their members. Afterwards the time to reach a cooperative decision was measured. The results show a significant link between the distribution of decisiveness in a team and the time to reach a cooperative decision in ambiguous situations. Furthermore, the experiment reveals a high ability of humans to cooperate on the guidance level in general. This motivates human-imitating design of future assistance systems on guidance level in human-machine cooperation with the intention of high user satisfaction and trust in these systems.

Determining the current intentions of other drivers is essential for correctly predicting or simulating their future actions. Especially unpredicted lane changes can result in very uncomfortable or even dangerous braking maneuvers for succeeding vehicles. Bayesian Networks (BN) allow for a physically motivated probabilistic representation of features influencing driver intentions. While features often take continuous values, e.g. velocity and distance, maneuver intentions are discrete, which results in hybrid BN. For efficient and exact inference, we implement an approach for hybrid nets into the original Bayes Net Toolbox. Furthermore, we extend the approach with a learning component to train a BN with simulated traffic data. Finally, we compare the classification performance for lane changes with a Deep Neural Network (DNN) classifier.

Shared control techniques have a great potential to create synergies in human-machine interaction for efficient and safe applications. However, an optimal interaction requires the machine to consider the individual behavior of the human partner. A widespread approach for modeling human behavior is given by optimal control theory, where the movement trajectories of a human arise from an optimized cost function. The aim of the identification is thus to determine parameters of a cost function which explains observed human motion. The central thesis of this paper is that individual cost function parameters which describe specific behavior can be determined by means of Inverse Reinforcement Learning. We show the applicability of the approach with a tracking control task example. The experiment consists in following a reference trajectory by means of a steering wheel. The study confirms that optimal control is suitable for modeling individual human behavior and demonstrates the suitability of Inverse Reinforcement Learning in order to determine the cost function parameters which explain measured data.

At higher automation levels, drivers do not need to permanently monitor the surrounding traffic environment and are allowed to focus on other tasks, while an advanced driver assistance system controls the vehicle. However, there is evidence that human drivers are prone to driving mistakes when they have to take over control from the ADAS due to their lack of situation awareness. Smoothly shifting control authority during the takeover period by a shared control approach promises to remedy this issue. We present a framework which models the human-machine interaction and makes it possible to adapt the shift of control to the needs of the human driver. Driver and ADAS are modeled as optimal controllers interfering with the same system (differential game). The handover/takeover-interaction is realized by time-variant objective functions. In consideration of the course of the takeover and the actions of the other partner this framework enables to obtain the allocation of the control effort by calculating the Nash equilibrium between driver and ADAS through solving a set of coupled Riccati differential equations. The functionality of this framework and the effects of three different takeover concepts (direct, continuous and stepwise shift) are shown via simulation of steering interaction during a lane change scenario.

This paper presents a cooperative shared control driver assistance system which supports the driver in the longitudinal vehicle control. The system architecture design enables the driver and the assistance system to apply a force on the accelerator pedal. Therefore, a cooperative control algorithm has to be developed. This design problem can be described mathematically as a differential game. For this approach, a model of the driver and the longitudinal vehicle dynamics is necessary. On this basis, the controller design can be stated as an optimization problem. Since the assistance system design is traced back to an optimization problem, the system is capable to support the driver in various driving situations. To verify if the assistance system is capable to support the driver with vehicle following and velocity control, a driving study on a driving simulator is performed. The results of the study demonstrate that the system can support the participants and improve the performance for the longitudinal vehicle control.

In this paper, a cooperative shared control driver assistance system that supports the driver in the steering task is proposed. The aim behind this concept is a cooperation between the driver and the assistance system. Thereby, both, driver and assistance system, can apply a torque on the steering wheel. Mathematically, this structure is described as a differential game. As a primary condition to facilitate cooperation, it is essential to explicitly regard the aims and steering actions of the driver for the calculation of the optimal torque, which the assistance system should apply. This requires an appropriate model of the human driver. A model that describes the driver steering motion as a sequence of motion primitives, which can be identified, is proposed for this task. Next, a real-time capable implementation of this concept is proposed. The concept is validated in a driving study. The study indicates that the system improves the lane keeping performance of the participants and leads to a higher user rating compared with noncooperative driver assistance systems.

Shared control is a promising approach for designing an Advanced Driver Assistance System, since it unifies the advantages of both manual control and full automation. However, for a true cooperative shared control ADAS the automation has to understand the human and thus a suitable model which describes the driver in the control loop is essential. Our gray-box approach bases on the biological concept that humans realize motion by combining a finite set of motion primitives (we call movemes). With the assumption that a driver switches between movemes based on perceived information, we propose a Hidden Markov Model which determines the probability of each movement given a certain driving situation. Car turn maneuver experiments show a good approximation of steering trajectories recorded in a driving simulator. A comparison with a black-box model show that the movement-based driver model performs significantly better. In addition, training algorithms are available and the probabilistic approach of the model allows further interpretation of the results.

In a shared control system humans and machines cooperatively interact. From the control theoretic point of view this can be seen as a system which is controlled by several controllers that are either formed by a human or by a machine. Since all controllers influence the system they affect each other. However, the human parts are given and cannot be changed. Therefore, the question is how to design the non-human controller systematically stable and without experiments. In this paper a design concept for these controllers is proposed which is based on game theoretic modeling. We show that adding a controller to the overall system has to lead to a Nash equilibrium. We further show that remaining degrees of freedom may then be used to optimize the designed controller with respect to a certain global objective function that specifies the demands of the system designers. Based on this idea necessary and sufficient conditions for cooperative shared design are stated. Practical approaches to solve the design problem are presented for real world problems. An example shows the applicability of the concept.

Several publications have shown that it is beneficial to design a driver assistance system using a shared control structure. For the steering task this structure can be realized with a setup in which driver and automation can apply a torque on the steering wheel in parallel. Thereby both, driver and assistance system, interact with the vehicle and each other over the haptical channel. In the system the driver is given and cannot be changed. The question is how to design the assistance system controller as an ideal complement to the driver. In this paper a formal design concept is applied to this problem which utilizes the fact that adding a controller to the overall system has to lead to a Nash equilibrium. Remaining degrees of freedom are used to optimize the designed controller with respect to a global objective function that specifies overall system performance. We refer the concept as “cooperative shared control design”. For the concept driver and vehicle are modeled as a differential game. We show systematically that this concept can be used to determine the optimal assistance system if the driver characteristics are known. Simulations prove the applicability of this concept.

To achieve a high acceptance by drivers Advanced Driver Assistance Systems (ADAS) have to consider the individual driver behavior. For example an ADAS should not intervene in a situation where drivers purposely cross road markings in harmless situations. To address this issue, a driver behavior characterization can be used to predict a driver's future trajectory based on his past behavior. In this paper, we present a method for driver behavior classification based on an inverse dynamic optimal approach and show how it can be applied to predict driver specific trajectories. The presented algorithm consists of two phases. In the trainingphase, a description of the driver's behavior is established using a set of generic driver characteristics. Hereby, a specific driver is described by his individual weighting of the characteristics and additional parameters used in the characteristics. In the prediction-phase, the model is applied to a specific track predicting the driver's future behavior. The model is adaptable to different situations and modeling purposes. It is shown by simulations that the approach is suited to model drivers with different driving characteristics and that the driver parameters can be reliably identified from recorded trajectories.

Ongoing research deals with the question how to design a machine for an optimal shared control with a human being. For examinition of different methods of cooperation an experiment was carried out in which two humans solve a simplified driving task together. A pilot study was executed where both the determination of the combined output as well as the feedback from the partners were varied. The results indicate that there is neither in signal fusion nor in the way of feedback a single form of cooperation which is preferable for all participants. Instead the optimal way is dependant on the level and homogeneity of the cooperating partners.

In this paper an Advanced Driver Assistance System (ADAS) supporting the driver to achieve an energy efficient driving strategy is proposed. State of the art driver assistance systems for this purpose either take over full control of the vehicle longitudinal dynamics or only provide information to the driver and do not directly influence the vehicle dynamics. We propose an assistance system which performs the control task in parallel with the driver. The system adapts the driver inputs in an energy optimal manner under the constraint the influence is not perceptible for the driver. A control framework is proposed which allows a real-time capable implementation of the ADAS. The concept is validated in a small experiment on a driving simulator based on a test scenario consisting of a 2.5 km straight road with varying speed limits. The results show the real-time capability of the control framework and indicate a significant reduction of the fuel consumption.

Using switched systems, we model individual driver steering behaviour from a new point of view. This approach allows to incorporate the idea of human motion being built up by an individual and limited repertoire of learned patterns. The identification of the generating subsystem parameters of such individual motion primitives solely on measured output data requires a new identification method. We propose an algorithm using a multi-step model output error criterion and discuss different implementations in detail. We show that this method is capable of tracking real measurement data of driver steering motion trajectories with low model orders and number of switches respectively. The presented method is online-capable. Experimental driving results proof the concept.

A model of the driver's lateral steering behavior is important for the design of a lateral steering assistance system, which performs the control task in cooperation with the driver. It is believed that humans realize their motions by combining elementary motion blocks - so called "movemes". Based on this theory and in contrast to state of the art driver models, the dynamic steering primitives resulting from the movemes are used as a grey box model of the neuromuscular system of a specific driver. Hence the proposed human inspired lateral steering model consists of a set of movemes and a superimposed switching mechanism. The switching mechanism is modeled using model predictive control and determines the optimal sequence of active movemes with respect to the steering task. In this article an experimental validation of the moveme based driver model using reference data obtained from a driving study is given.