Driving simulators are essential tools to guide automotive research and development. Their motion system requires a motion cueing algorithm (MCA) to keep the simulator motion platform within its physical boundaries, while simultaneously aiming to recreate the sensation of real vehicle motion. While traditional, filter-based approaches are still predominant, optimization-based MCAs have been at the center of MCA research for over a decade due to their ability to systematically improve motion cueing quality through explicit cost function design and constraints handling. However, despite their demonstrated advantages, these optimization-based methods have not yet achieved widespread adoption in driving simulation. This paper therefore provides a comprehensive review of optimization-based MCAs for driving simulation, categorizing and comparing algorithms, describing their key developments and core characteristics. The current limited real-time capability, lack of accurate evaluation methods, challenges in cost function design and its tuning, and the current lack of accurate future reference predictions are identified as key barriers to the practical deployment and widespread use of optimization-based MCAs. These theoretical and practical challenges are further reviewed, providing guidelines to advance the theory and application of optimization-based MCAs. Central in these advancements are a better understanding of which motions constitute a realistic motion experience, a framework allowing to compare the achieved motion fidelity of MCAs across papers, the design of the cost function focusing on human motion perception, and techniques for easing up the tuning process to swiftly reach high quality tunings for different simulators, scenarios, and use cases. We identify the need for improving the real-time capability, and providing high quality motion reference predictions using learning-based approaches on diverse datasets, along with techniques to handle existing uncertainties. Following these guidelines, new foundations for optimization-based algorithms in driving simulation can be achieved, which will significantly impact the research and development of automotive systems.

Autonomous robotic grasping of uncertain objects in uncertain environments is an impactful open challenge for the industries of the future. One such industry is the recycling of Waste Electrical and Electronic Equipment (WEEE) materials, in which electric devices are disassembled and readied for the recovery of raw materials. Since devices may contain hazardous materials and their disassembly involves heavy manual labor, robotic disassembly is a promising venue. However, since devices may be damaged, dirty and unidentified, robotic disassembly is challenging since object models are unavailable or cannot be relied upon. This case study explores grasping strategies for industrial robotic disassembly of WEEE devices with uncertain vision data. We propose three grippers and appropriate tactile strategies for force-based manipulation that improves grasping robustness. For each proposed gripper, we develop corresponding strategies that can perform effectively in different grasping tasks and leverage the grippers design and unique strengths. Through experiments conducted in lab and factory settings for four different WEEE devices, we demonstrate how object uncertainty may be overcome by tactile sensing and compliant techniques, significantly increasing grasping success rates.

Grasping fragile objects in the presence of un-certainty is a crucial task for robots, that becomes inherently challenging if the manipulator in use is an industrial robot platform that does not provide compliant control inputs. This requires not only to estimate the alignment error during object contact but also to alter the robot configuration to decrease this error while taking interaction constraints into account. Thus, this work proposes a novel grasping controller tailored to industrial robots by exploiting tactile sensor feedback on the robot gripper fingers in order to estimate and compensate for the alignment error when touching the object. Specifically, we propose two grasping strategies, that allow to either directly compensate for interaction wrenches or to solve a model predictive control-problem to minimize the estimated alignment error. Eventually, we outline how these modalities can be realized as a hybrid Cartesian force-velocity-controller on an industrial manipulator. We evaluate the proposed grasping strategies on a WSG 50 parallel two-finger gripper, that is equipped with a digital sensor array (DSA) per finger, for which we also provide an extended ROS-driver that allows to obtain DSA-data at a communication rate above 5 Hz. Given the collected empirical evidence, the presented grasping controller increases the skill-set of industrial robots in the presence of uncertainty and thus allows to apply stiff robots to handle fragile objects autonomously.

We propose a novel method to tackle the visual-inertial localization problem for constrained camera movements. We use residuals from the different modalities to jointly optimize a global cost function. The residuals emerge from IMU measurements, stereoscopic feature points, and constraints on possible solutions in SE(3). In settings where dynamic disturbances are frequent, the residuals reduce the complexity of the problem and make localization feasible. We verify the advantages of our method in a suitable medical use case and produce a dataset capturing a minimally invasive surgery in the abdomen. Our novel clinical dataset MITI is comparable to state-of-the-art evaluation datasets, contains calibration and synchronization and is available at [1].

Gaussian Process (GP) formulation of continuous-time trajectory offers a fast solution to the motion planning problem via probabilistic inference on factor graph. However, often the solution converges to in-feasible local minima and the planned trajectory is not collision-free. We propose a message passing algorithm that is more sensitive to obstacles with fast convergence time. We leverage the utility of min-sum message passing algorithm that performs local computations at each node to solve the inference problem on factor graph. We first introduce the notion of compound factor node to transform the factor graph to a linearly structured graph. We next develop an algorithm denoted as Min-sum Message Passing algorithm for Motion Planning (MS2MP) that combines numerical optimization with message passing to find collision- free trajectories. MS2MP performs numerical optimization to solve non-linear least square minimization problem at each compound factor node and then exploits the linear structure of factor graph to compute the maximum a posteriori (MAP) estimation of complete graph by passing messages among graph nodes. The decentralized optimization approach of each compound node increases sensitivity towards avoiding obstacles for harder planning problems. We evaluate our algorithm by performing extensive experiments for exemplary motion planning tasks for a robot manipulator. Our evaluation reveals that MS2MP improves existing work in convergence time and success rate.

Redundant robots offer the possibility of improving agility, compared to their non-redundant counterparts, by exploiting the additional kinematic DOFs to increase a measure called manipulability. While it is common to maximize the manipulability measure during redundancy resolution locally, global optimization of a full trajectory is usually computationally too expensive and thus only considered for offline procedures in current literature. However, local maximization is prone to be sub-optimal and at times even fails at preserving agility of a robot that ought to be reactive. In this work we build upon our previous contributions on online trajectory generation on SE(3) and closed-form task space manipulability of a 7-DOF serial robot, and combine it with graph search techniques for global optimization. This enables, for the first time, online trajectory generation with globally optimal redundancy resolution regarding manipulability, to maintain agility in reactive robot behavior.

This article is dedicated to developing an online collision detection and identification (CDI) scheme for human-collaborative robots. The scheme is composed of a signal classifier and an online diagnosor, which monitors the sensory signals of the robot system, detects the occurrence of a physical human–robot interaction, and identifies its type within a short period. In the beginning, we conduct an experiment to construct a data set that contains the segmented physical interaction signals with ground truth. Then, we develop the signal classifier on the data set with the paradigm of supervised learning. To adapt the classifier to the online application with requirements on response time, an auxiliary online diagnosor is designed using the Bayesian decision theory. The diagnosor provides not only a collision identification result but also a confidence index which represents the reliability of the result. Compared to the previous works, the proposed scheme ensures rapid and accurate CDI even in the early stage of a physical interaction. As a result, safety mechanisms can be triggered before further injuries are caused, which is quite valuable and important toward a safe human–robot collaboration. In the end, the proposed scheme is validated on a robot manipulator and applied to a demonstration task with collision reaction strategies. The experimental results reveal that the collisions are detected and classified within 20 ms with an overall accuracy of 99.6%, which confirms the applicability of the scheme to collaborative robots in practice.Note to Practitioners—This article is intended to provide a novel online collision event handling scheme for robots in industrial environments. This scheme is designed to quickly and accurately detect an accidental collision and distinguish it from the intentional human–robot interaction. The method takes the raw signals from external torque sensors and provides a collision diagnosis result with a reliability index. The simple structure makes it easy to be implemented as a regular fault monitoring routine for collaborative robots. Different from the conventional methods, the proposed collision identification scheme in this article especially focuses on overcoming the following two challenges in practice: first, to timely and accurately report a collision within its early stage, and second, to ensure a high identification accuracy in a complicated environment, where ubiquitous disturbance and noise are unneglectable. The experimental validation at the end of this article confirms its promising application value in human–robot collaboration.

Object tracking is crucial for planning safe maneuvers of mobile robots in dynamic environments, in particular for autonomous driving with surrounding traffic participants. Multi-stage processing of sensor measurement data is thereby required to obtain abstracted high-level objects, such as vehicles. This also includes sensor fusion, data association, and temporal filtering. Often, an early-stage object abstraction is performed, which, however, is critical, as it results in information loss regarding the subsequent processing steps. We present a new grid-based object tracking approach that, in contrast, is based on already fused measurement data. The input is thereby pre-processed, without abstracting objects, by the spatial grid cell discretization of a dynamic occupancy grid, which enables a generic multi-sensor detection of moving objects. On the basis of already associated occupied cells, presented in our previous work, this paper investigates the subsequent object state estimation. The object pose and shape estimation thereby benefit from the freespace information contained in the input grid, which is evaluated to determine the current visibility of extracted object parts. An integrated object classification concept further enhances the assumed object size. For a precise dynamic motion state estimation, radar Doppler velocity measurements are integrated into the input data and processed directly on the object-level. Our approach is evaluated with real sensor data in the context of autonomous driving in challenging urban scenarios.

Robots are expected to possess similar capabilities that humans exhibit during close proximity dyadic interaction. Humans can easily adapt to each other in a multitude of scenarios, ensuring safe and natural interaction. Even though there have been attempts to mimic human motions for robot control, understanding the motion patterns emerging during dyadic interaction has been neglected. In this work, we analyze close-proximity human-human interaction and derive an ontology that describes a broad range of possible interaction scenarios by abstracting tasks and using insights from attention theory. This ontology enables us to group interaction behaviors into separate cases, each of which can be represented by a particular graph. Using imitation learning, we train unique interaction policies with recurrent neural networks for each case. The ontology offers a unified and generic approach to categorically analyze and learn close-proximity interaction behaviors that can both be utilized as base models for future studies and enhance natural human-robot collaboration.

With the integration of Human-Robot Collaboration (HRC) in industrial assembly scenarios, robot systems face numerous challenges. In contrast to classic robot systems which follow a pre-programmed and fixed sequence of actions, an interaction scenario with humans in the loop requires mutual adaptation. In this paper a framework based on game theory is presented that allows robots to choose appropriate actions with respect to the action of human coworkers when collaborating in close proximity. The proposed framework models HRC scenarios as iterative games and selects action-strategies for the Human-Robot Team (HRT) by finding the Nash-Equilibria (NEs) of these games. In contrast to most common approaches, our proposed HRC-game treats the decision-making behavior equally for all agents involved. Therefore, the concept of game theory is applied to evaluate the mutual interference of all actions on the HRT to obtain pareto-optimal NEs, i.e. team-optimal action-allocations. The general framework of the proposed HRC-game is realized on an interactive pick-and-place scenario in close proximity. This exemplary HRC-game is tested in a human subject experiment of a KUKA LWR 4+ robot and a human coworker assembling toy-bricks in close proximity. The experimental measurements and statistically significant improvements in the subjective feedback hold as a proof-of-concept of the proposed HRC-game model.

With the increasing demand of humans and robots collaborating in a confined workspace, it is essential that robots can react instantaneously to unforeseen events while respecting comfort of human co-workers. As the human pays especial attention to the end-effector movement rather than individual joints in such tasks, we argue that it is necessary to limit exactly these end-effector dynamics directly on SE(3), regardless of any specific coordinate system. Only then it is possible to constrain the robot end-effector in its absolute dynamic values, to enhance human comfort. Common online trajectory generators (OTGs) are either limited to single degrees of freedom (DOFs) or consider decoupled multi-DOF mainly in joint space. While these strategies can be directly applied to Cartesian coordinate systems by using Euler angles for orientation, constraining the change in Euler angles does not lead to the correct angular velocities. In this work we present a novel OTG with constraints on translational and rotational magnitudes on SE(3). Simulations as well as experiments on a KUKA LWR4+ show the potential contribution towards improved Human-Robot-Collaboration.

Occupancy grid mapping approaches, especially those that additionally estimate the dynamics, enable a robust and consistent modeling of the local environment in a cell-level representation. But a scene understanding of surrounding traffic participants requires a generalized object-level representation. This work presents an object tracking approach based on dynamic occupancy grids. The association of occupied grid cells with existing object tracks is solved individually on the cell-level without clustering or forming object hypotheses. New object tracks are extracted using a clustering strategy and a velocity variance analysis of neighboring occupied cells to reduce false positives. In order to improve the estimates of the position and size, an object boundary extraction is presented that takes the surrounding free space of the selected box representation into account. Experimental results with real sensor data show the effectiveness of the proposed object tracking approach in challenging urban scenarios with dense traffic.

Two-dimensional occupancy grid mapping is a common approach for environment mapping and sensor data fusion but non-planar environments are still a remaining issue. The ground shape has to be considered in such environments, especially when low obstacles on the road need to be recognized. Elevation maps are a suitable model because the height is not discretized. This paper presents an improved lidar-based approach for elevation mapping that uses particle filters to estimate the height indirectly by a fusion of lower and upper height boundaries. These boundaries are extracted from lidar reflections and ray geometry of different time steps. Furthermore, a tailored interpolation algorithm is presented that takes the statistics of the particle population of each cell into account. A conclusive qualitative and quantitative evaluation highlights the performance of the presented approach.

This paper introduces a new approach to optimal online motion planning for human-robot interaction scenarios. For a safe, comfortable, and efficient interaction between human and robot working in close proximity, robot motion has to be agile and perceived as natural by the human partner. The robot has to be aware of its environment, including human motions, in order to proactively take actions while ensuring safety, and task fulfillment. Human motion prediction constitutes the fundamental perception input for the motion planner. The prediction system, which is based on probabilistic movement primitives, generates a prediction of human motion as a trajectory distribution learned in an offline phase. The proposed stochastic optimization-based planning algorithm then progressively finds feasible optimization parameters to replan the motion online that ensures collision avoidance while minimizing the task-related trajectory cost. Our simulation results show that the proposed approach produces collision-free trajectories while still reaching the goal successfully. We also highlight the performance of our planner in comparison to previous methods in stochastic motion planning.

Humans are error-prone in the presence of multiple similar tasks. While Human-Robot Collaboration (HRC) brings the advantage of combining the superiority of both humans and robots in their respective talents, it also requires the robot to communicate the task goal clearly to the human collaborator. We formalize such problems in interactive assembly tasks with hidden goal Markov decision processes (HGMDPs) to enable the symbiosis of human intention recognition and robot intention expression. In order to avoid the prohibitive computational requirements, we provide a myopic heuristic along with a feature-based state abstraction method for assembly tasks to approximate the solution of the resulting HGMDP. A user study with human subjects in round-based LEGO assembly tasks shows that our algorithm improves HRC and helps the human collaborators when the task goal is unclear to them.

Tracking and mapping the local environment form the basis of an autonomous vehicle system. They are often realized separately using occupancy grids, which do not require object or shape assumptions, and model-based object tracking algorithms. Many approaches require a binary classification of the sensor measurements into coming from a static or from a dynamic object, as otherwise inconsistencies between the different representations are likely to occur. This paper presents grid-based tracking and mapping (GTAM), a low-level grid-based approach that simultaneously estimates the static and the dynamic environment, their uncertainties, velocities, as well as information about free space. GTAM works on the level of grid cells, rather than creating object hypotheses. A particle filter is used to obtain continuous cell velocity distributions for all obstacles. Continuous evidences in a Dempster-Shafer model are derived without requiring a binary pre-classification of the sensor measurements. Results and evaluations using a vehicle moving in real dynamic street environments demonstrate the performance of the presented approach.

This article discusses the quadratization of Markov Logic Networks, which enables efficient approximate MAP computation by means of maximum flows. The procedure relies on a pseudo-Boolean representation of the model, and allows handling models of any order. The employed pseudo-Boolean representation can be used to identify problems that are guaranteed to be solvable in low polynomial-time. Results on common benchmark problems show that the proposed approach finds optimal assignments for most variables in excellent computational time and approximate solutions that match the quality of ILP-based solvers.

This paper presents a novel online-capable interaction-aware intention and maneuver prediction framework for dynamic environments. The main contribution is the combination of model-based interaction-aware intention estimation with maneuver-based motion prediction based on supervised learning. The advantages of this framework are twofold. On one hand, expert knowledge in the form of heuristics is integrated, which simplifies the modeling of the interaction. On the other hand, the difficulties associated with the scalability and data sparsity of the algorithm due to the so-called curse of dimensionality can be reduced, as a reduced feature space is sufficient for supervised learning. The proposed algorithm can be used for highly automated driving or as a prediction module for advanced driver assistance systems without the need of intervehicle communication. At the start of the algorithm, the motion intention of each driver in a traffic scene is predicted in an iterative manner using the game-theoretic idea of stochastic multiagent simulation. This approach provides an interpretation of what other drivers intend to do and how they interact with surrounding traffic. By incorporating this information into a Bayesian network classifier, the developed framework achieves a significant improvement in terms of reliable prediction time and precision compared with other state-of-the-art approaches. By means of experimental results in real traffic on highways, the validity of the proposed concept and its online capability is demonstrated. Furthermore, its performance is quantitatively evaluated using appropriate statistical measures.

In robotics applications such as SLAM (Simultaneous Localization and Mapping), loop closure detection is an integral component required to build a consistent topological or metric map. This paper presents an appearance based loop closure detection mechanism titled `IBuILD' (Incremental bag of BInary words for Appearance based Loop closure Detection). The presented approach focuses on an online, incremental formulation of binary vocabulary generation for loop closure detection. The proposed approach does not require a prior vocabulary learning phase and relies purely on the appearance of the scene for loop closure detection without the need of odometry or GPS estimates. The vocabulary generation process is based on feature tracking between consecutive images to incorporate pose invariance. In addition, this process is coupled with a simple likelihood function to generate the most suitable loop closure candidate and a temporal consistency constraint to filter out inconsistent loop closures. Evaluation on different publicly available outdoor urban and indoor datasets shows that the presented approach is capable of generating higher recall at 100% precision in comparison to the state of the art.

This paper presents an extension of the standard occupancy grid for 3D environment mapping. The presented approach adds a fusion process after the occupancy update which modifies the resolution of the grid cells in an incremental manner. Consequently, the proposed approach requires fewer grid cells for 3D representation in comparison to a standard occupancy grid. The resolution adaptation process is based on the occupancy probabilities of the grid cells and leads to the relaxation of the cubic grid cell assumption common to most 3D occupancy grids. The aim of this paper is to show the advantage of the proposed incremental fusion process which leads to the approximation of the 3D environment using rectangular cuboids. Evaluation on a large scale dataset and comparison to the state of the art shows that the proposed approach has faster access time for all occupied grid cells and requires a smaller number of cells for 3D environment representation.

During autonomous driving, in particular conditional or highly automated driving, a critical part of the system is the driver take over request. Little focus has been given to this important aspect in an automated driving journey. A driver take over request, or TOR, can happen for various reasons and under varying circumstances. Once a TOR occurs, as defined in conditional or highly automated driving, the driver has a finite amount of time in order to take over manual control of the vehicle before the automated driving system deactivates. This paper presents a detailed analysis of why a TOR can occur, how the automated driving system should react during the TOR phase and what should happen at the end of a TOR in order to realize a safe and comfortable TOR for the driver. Various driving strategies during a TOR are presented and evaluated for a single-lane highway scenario.

Mapping and tracking in dynamic environments for autonomously-moving robots is still challenging, despite being essential tasks. They are often done separately using occupancy grids and established object tracking algorithms. In this work, an approach is presented that estimates a uniform, low-level, grid-based world model including dynamic and static objects, their uncertainties, as well as their velocities. It does not require existing object tracks to filter out data points not used for creating and updating the map. Nor does it require that measurements can be classified into belonging to a static or to a moving object. Promising results from experiments with an autonomous vehicle equipped with a laser scanner demonstrate the usefulness of the approach.

In this paper the states of inevitable collision for mobile robots are determined using reachable set theory. With this theory the safety for robotic platforms can be guaranteed, still allowing maximum flexibility for navigation. Making use of reachability analysis, limitations due to input sampling as in previous approaches are avoided. Using reachability analysis the obstacles are grown in the state space. The mathematical background is shown in this paper and an exemplary algorithm is given for static environments. This implementation can handle arbitrary environments with multiple obstacles and different high-dimensional linear and non-linear system dynamics including the car-like kinematic model. By means of experimental results in simulated environments, the validity of the proposed concept is shown.

This work presents a trajectory clustering approach that groups trajectories without the need of manually-tuned distance thresholds. Contrary to trajectory clustering approaches that use continuous, often geometrically-motivated similarity measures, path similarity is binary. Similar to homotopy classes, path equivalence is based on the obstacles in the environment. The goal states are, however, not fixed, but the paths have certain length restrictions. The equivalence is efficiently checked by closing the paths with sampled intermediate trajectories and using point-in-polygon tests. The proposed algorithm has linear complexity in the number of paths for non-overlapping clusters and, under certain assumptions, also in the case of overlapping clusters. Experimental results from an integration into a path-planning-based road course estimation system are shown and compared to a traditional distance-similarity cluster analysis to demonstrate the performance.

Humans are more successful in planning collision free, continuous trajectories through populated environments than any motion planning algorithm so far. This is due to the fact that they consider the conditionally cooperative, interactive behavior of the surrounding persons, for example the possibility of mutual avoidance maneuvers. In this paper, interaction during navigation is regarded from a game theoretic perspective and the concept of Nash equilibria is applied to analyze human motion. In contrast to other methods, the game theoretic approach does not necessarily rely on learning the interaction itself and is extendable. Our approach is based on human motion data that is captured during experiments. Two hypotheses are verified: for one thing, interaction exists during human navigation, for another thing, the mutual avoidance behavior of humans can be modeled with the theory of Nash equilibria in non-cooperative games. This knowledge can be used to enhance existing motion planning algorithms for autonomous robots.

Highly automated driving on freeways requires a complex artificial intelligence that makes optimal decisions based on the current measurements and information. The architecture of the decision-making process, hereinafter referred to as driving strategy, should allow diversity in decision-making for various traffic situations and modular expandability of the overall intelligence. Besides a reactive response to changes in the dynamic environment, a deliberative component should also be considered to incorporate the future evolution of the environment. This paper presents a novel driving strategy that meets the above requirements. The complex driving task is discretized by organization into a finite set of “behavioral strategies” through the developed “decision network”. The decision-making process itself is realized by a nonlinear model predictive approach which is solved using combinatorial optimization formulation. Lastly, the capability of the proposed approach is demonstrated in two freeway situations.

Collision checking is the major computational bottleneck for many robot path and motion planning applications, such as for autonomous vehicles, particularly with grid-based environment representations. Apart from collisions, many applications benefit from incorporating costs into planning; cost functions or cost maps are a common tool. Similar to checking a single configuration for collision, evaluating its cost using a grid-based cost map also requires examining every cell under the robot footprint. This work gives theoretical and practical insights on how to efficiently check a large number of configurations for collision and cost. As part of this work, configuration space costs are formulated, which can be seen as generalization of configuration space obstacles allowing a complete configuration check incorporating the robot geometry to be done using a single lookup. Furthermore, this paper presents two efficient algorithms for their calculation: FAMOD, an approximate method based on convolution, which is independent of the size and the shape of the robot mask, and vHGW-360, an exact method based on the van Herk-Gil-Werman morphological dilation algorithm, which can be used if the robot shape is rectangular. Both algorithms were implemented and evaluated on graphics hardware to demonstrate the applicability and benefit to real-time path and motion planning systems.

This paper presents an approach to combine automatic semantic place labeling of robot-generated maps with reasoning on human route descriptions. Enabling robots to understand human route descriptions can simplify HRI situations in household or industrial settings. However, solving this problem requires handling the ambiguity present in route descriptions and the possible unreliability of the semantic perception capabilities of the robot. We address this problem by absorbing these uncertainties in a probability distribution measuring the likelihood of the different interpretations (paths) of a given route description and selecting its MAP solution. The approach is evaluated on a dataset of route descriptions transcribed into a suitable representation using standard information retrieval metrics. These performance measurements indicate that the method can correctly interpret route descriptions even in challenging environments.

In this paper energy optimal solutions for the approach of red traffic lights are derived. As cars waste most of the fuel in city traffic and especially in queuing at traffic lights, the presented framework provides solutions to save fuel and to protect the environment. The solutions are obtained using the definition of spent physical work which has to be minimized. It covers both cases, that the time of switching of the traffic lights is known and that the time of switching can only be modeled as a stochastic process. For a known time of switching a continuous solution is derived using Pontryagin Minimum Principle; in the stochastic case a modified Bellmann equation is formulated. The latter is solved with dynamic programming techniques. The presented solutions can be used for autonomous driving as well as for driving assistant systems. Simulation results show the potential savings using the presented approach.

In this work, a framework for motion prediction of vehicles and safety assessment of traffic scenes is presented. The developed framework can be used for driver assistant systems as well as for autonomous driving applications. In order to assess the safety of the future trajectories of the vehicle, these systems require a prediction of the future motion of all traffic participants. As the traffic participants have a mutual influence on each other, the interaction of them is explicitly considered in this framework, which is inspired by an optimization problem. Taking the mutual influence of traffic participants into account, this framework differs from the existing approaches which consider the interaction only insufficiently, suffering reliability in real traffic scenes. For motion prediction, the collision probability of a vehicle performing a certain maneuver, is computed. Based on the safety evaluation and the assumption that drivers avoid collisions, the prediction is realized. Simulation scenarios and real-world results show the functionality.

The road course is an essential feature for many driver assistance systems and for autonomously-maneuvering vehicles. It is commonly stored in a map and hence assumed to be known a-priori. There are however situations in which the map data can become invalid, such as in road construction sites. In other situations, localization in the map might not be accurate enough, which can happen, for example, in dense urban areas. In this work, a novel approach to road course estimation is presented that is based on path planning through grid maps under non-holonomic and velocity constraints. With this approach, it is possible to estimate the road boundaries on a wide range of roads, including roads with continuous as well as discontinuous borders, roads exhibiting strong curvatures or S-shapes and road junctions. Furthermore, a plausibility measure is given to validate the road course and it is shown how the road center can be smoothed.

In this paper a novel approach for 3D environment mapping using registered robot poses is presented. The proposed algorithm focuses on improving the quality of robot generated 3D maps by incorporating the uncertainty of 3D points and propagating it into the normal vectors of surfaces. The uncertainty of normal vectors is an indicator of the quality of the detected surface. A controlled random search algorithm is applied to optimize a non-convex function of uncertain normal vectors and number of clusters in order to find the optimal threshold parameter for the segmentation process. This approach leads to an improved cluster coherence and thus better maps.

In this paper, an online spatial and appearance based loop closure algorithm is presented. The approach is based on a graph matching formulation using Position Invariant Robust Features (PIRF), extending previous approaches based on PIRF by incorporating spatial information. The vertices of the graph represent visual words/features and edges represent metric information with uncertainty since the spatial distances observed between visual words are prone to errors. This method is capable of detecting loop closure in urban environments based on visual appearance as well as spatial layout of matched visual features. A vocabulary is built in an online and incremental manner, also storing spatial distances between visual words. The algorithm is capable of assigning loop closure with higher confidence values and a higher recall rate while maintaining precision compared to approaches where only visual appearance methods are used. We evaluate this approach on a publicly available dataset and present experimental results.

This video presents a motion planning method enabling an autonomous mobile robot to approach a person to initiate a conversation proactively. The proposed concept implements social aspects to give motions a more human-like appearance. This is intended since experiments discussed in literature have shown that it is easier for people to predict and read the purpose of human-like movements. User study results show that the readability of the approach behavior is improved by including parameters such as path execution speed, distance between goal pose and person, position and orientation in front of the person, and trajectory shape. The video depicts the results of an implemented human approach planner for dynamic environments. It plans a trajectory towards a person applying the named set of social constraints. Simulations and real world experiments show how the approaching behavior smoothly brings the robot close to a person for interaction.

This paper describes a new methodological approach and robot system to trigger more prosocial human reactions towards a robot by transferring social-psychological principles from human-human interaction to human-robot interaction (HRI). The main idea is to trigger increased helpfulness by proactively creating similarity through dynamic emotional adaption of the robot to the mood of the human. This is achieved in an explicit and implicit way: Explicitly, by a similarity-statement of the robot of being in the same mood as the user, and implicitly by controlling the affective parameters of facial and verbal expressions of a robot head in an interaction scenario such that the current values of the human mood in the dimensions of pleasure, arousal, and dominance (PAD) are matched. In a first step, this is accomplished by an initial self-assessment by the human participant to be extended by automatic emotion recognition modules in a later stage. The effectiveness of the approach is confirmed by significant experimental results.

This work presents a novel approach for collision assessment for automotive environments. As collision prevention and risk analysis are key challenges for today's intelligent transport systems, sophisticated solutions for a collision-free trajectory generation get indispensable. The presented collision checker is integrated into an optimal control based planning framework that generates minimum jerk trajectories for arbitrary maneuvers. In contrast to common collision checkers, the developed method does not need to discretize the time space, but gives an algebraic solution which covers nearly all situations and has a constant response time. Due to its fast evaluation it is predestined for use in a Monte Carlo simulation respecting the probabilistic nature of real world traffic scenes. An example implementation of the proposed method is applied to simulation scenarios that demonstrates the benefits to comparable approaches. The obtained results have proven the feasibility of our approach to risk analysis of traffic situations.

This paper presents a method for reasoning about the safety of traffic situations. More precisely, the problem of safety assessment for partial trajectories for vehicles is addressed. Therefore, the Inevitable Collision States (ICS) as well as its probabilistic generalization the Probabilistic Collision States (PCS) are used. Thereby, the assessment is performed for an infinite time horizon. For solving the ICS computation nonlinear programming is applied. In addition to the safety assessment an evaluation of the disturbance of the other traffic participants by the ego vehicle is presented. The results are integrated into an optimal control based planning approach that generates minimum jerk trajectories. An example implementation of the proposed framework is applied to simulation scenarios that demonstrates the necessity of the presented method for guaranteeing motion safety.

It is the aim of our research to explore how multimodal feedback can help a robot to carry out itinerary requests effectively and satisfactory for a human interaction partner. We conducted two studies to evaluate the feedback setup of the Interactive Urban Robot (IURO), which navigates through public space autonomously and finds its way by asking pedestrians for directions. In a Wizard-of-Oz (WOz) experiment with novice users, different feedback modalities and various combinations of them were tested against each other to ascertain the ideal setup of the robot. Subsequently, a cognitive walkthrough with HRI experts was performed to validate the results from the experiment. The results from both studies show that for itinerary requests verbal feedback is most prominent but other feedback modalities may support the conversation by providing reassurance or positive emotions.

This paper presents a probabilistic threat assessment method for reasoning about the safety of robot trajectories in uncertain and dynamic environments. For safety evaluation, the overall collision probability is used to rank candidate trajectories by considering the collision probability of known objects as well as the collision probability beyond the planning horizon. Monte Carlo sampling is used to estimate the collision probabilities. This concept is applied to a navigation framework that generates and selects trajectories in order to reach the goal location while minimizing the collision probability. Simulation scenarios are used to validate the overall crash probability and show its necessity in the proposed navigation approach.

This work presents an extension of the Dynamic Window Approach. The reactive collision avoidance algorithm is generalized to the case of omni-directional kinematics for any polygonal shaped, mobile robot. This paper includes a superior implementation to former realizations and has already been successfully tested with an omni-directional robot. The implementation is efficient and produces collision-free trajectories even in narrow and crowded environments.

For reasoning about the safety of a robot system, it is sufficient to pretend the robot to reach an Inevitable Collision Sate (ICS). Otherwise, there exists no future trajectory which can avoid a collision. The usage of ICS is limited due to its computational complexity. One reason for this is, that the ICS computation cannot be done separately for each obstacle. Hence, ICS needs to be recomputed from scratch if another object appears in the scene. The main contribution of this paper is a modified ICS calculation which allows to compute the union of ICS sets in a sequential manner, thus reducing the computational requirements in case of new obstacles. Therefore, two novel ICS-Checker algorithms are presented reducing the computational effort. Furthermore, this novel calculation is used to reduce the probability of being in an ICS regarding an unforeseen obstacle.

This paper implements a real-time hand gesture recognition algorithm based on the inexpensive Kinect sensor. The use of a depth sensor allows for complex 3D gestures where the system is robust to disturbing objects or persons in the background. A Haarlet-based hand gesture recognition system is implemented to detect hand gestures in any orientation, and more in particular pointing gestures while extracting the 3D pointing direction. The system is integrated on an interactive robot (based on ROS), allowing for real-time hand gesture interaction with the robot. Pointing gestures are translated into goals for the robot, telling him where to go. A demo scenario is presented where the robot looks for persons to interact with, asks for directions, and then detects a 3D pointing direction. The robot then explores his vicinity in the given direction and looks for a new person to interact with.

As communication quality in public spaces often is impaired by noisy environment, it is difficult for a robot to retrieve missing task-information from humans. In this paper, different dialog strategies are modeled and evaluated with respect to user experience and error handling capabilities in HRI in order to cope with erroneous speech recognition. Since correct recognition of spoken language is a bottleneck for real-world dialog systems, special emphasis is placed on the issue of adapting dialog strategies to the conditions under which the dialog is held to thereby provide for adaptability of the dialog strategy to variable speech recognition performance. Experimental evaluations are conducted in a fully automated indoor setting, and in a Wizard-of-Oz outdoor setting. Results indicate that a critical point exists, up to which the use of requests for handling miscommunication improves the user experience of a dialog strategy.

In this paper, the impact of facial expressions on HRI is explored. To determine their influence on empathy of a human towards a robot and perceived subjective performance, an experimental setup is created, in which participants engage in a dialog with the robot head EDDIE. The web-based gaming application “Akinator” serves as a backbone for the dialog structure. In this game, the robot tries to guess a thought-of person chosen by the human by asking various questions about the person. In our experimental evaluation, the robot reacts in various ways to the human's facial expressions, either ignoring them, mirroring them, or displaying its own facial expression based on a psychological model for social awareness. In which way this robot behavior influences human perception of the interaction is investigated by a questionnaire. Our results support the hypothesis that the robot behavior during interaction heavily influences the extent of empathy by a human towards a robot and perceived subjective task-performance, with the adaptive modes clearly leading compared to the non-adaptive mode.

Abstract — In this paper, an approach is presented that allows a robot to navigate in an urban environment by following natural language route instructions. In this situation, neither maps nor GPS information are available to the robot thus it has to rely solely on the human-given route description and the observations from its sensors. An architecture for solving problems such as navigation on the sidewalk, street direction inference, and environment labeling that arise in this situation is presented. Our initial experiments indicate that the proposed methods enable a robot to safely navigate in urban environments by following abstract route descriptions and reach previously unknown points in a city. I.

This paper reports about a human-robot interaction field trial conducted with the autonomous mobile robot ACE (Autonomous City Explorer) in a public place, where the ACE robot needs the support of human passers-by to find its way to a target location. Since the robot does not possess any prior map knowledge or GPS support, it has to acquire missing information through interaction with humans. The robot thus has to initiate communication by asking for the way, and retrieves information from passers-by showing the way by gestures (pointing) and marking goal positions on a still image on the touch screen of the robot. The aims of the field trial where threefold: (1) Investigating the aptitude of the navigation architecture, (2) Evaluating the intuitiveness of the interaction concept for the passers-by, (3) Assessing people's willingness to support the ACE robot in its task, i.e. assessing the social acceptability. The field trial demonstrates that the architecture enables successful autonomous path finding without any prior map knowledge just by route directions given by passers-by. An additional street survey and observational data moreover attests the intuitiveness of the interaction paradigm and the high acceptability of the ACE robot in the public place.

For path planning algorithms of robots it is important that the robot does not reach a state of inevitable collision. In crowded environments with many humans or robots, the set of possible inevitable collision states (ICS) is often unacceptably high, such that the robot has to stop and wait in too many situations. For this reason, the concept of ICS is extended to probabilistic collision states (PCS), which estimates the collision probability for a given state. This allows to efficiently run planning algorithms through crowded environments when accepting a certain collision probability. A further novelty is that the obstacles possibly react to the robot in order to mitigate the risk of a collision. The results show a significant difference in interaction behavior. Thus, this approach is especially suited for active and non-deterministic moving obstacles in the robot workspace.

Ball dribbling is a central element of basketball. One main challenge for realizing basketball robots is to stabilize periodic motions of the ball. This task is nontrivial due to the discrete-continuous nature of the corresponding dynamics. The ball can be only controlled during ball-manipulator contact and moves freely otherwise. We propose a manipulator equipped with a spring that gets compressed when the ball bounces against it. Hence, we can have continuous-time control over this underactuated Ball-Spring-Manipulator system until the spring releases its accumulated energy back to the ball. This paper illustrates a systematic way of planning such a modified dribbling motion, computing an analytical transverse linearization and achieving orbital stabilization.

Ball dribbling is a central element of basketball. One main challenge for realizing basketball robots is to stabilize periodic motion of the ball. The task is nontrivial due to the discrete-continuous nature of the corresponding dynamics. This paper proposes to add an elastic element to the manipulator so the ball can be controlled in a continuous-time phase instead of an intermittent contact. Optimal catching and pushing trajectories are planned for the underactuated system based on the virtual holonomic constraints approach. First experimental studies are presented to evaluate the approach.

The overall performance of a robotic system is commonly expressed by a single scenario-specific metric which is supposed to be optimized. However, the metric describing the performance of a single subtask within a scenario may be different. Nevertheless, the scenario performance is most likely dependent on the subtask performances but a mutual transformation is not straightforward in general, especially in complex robotic systems. This leads to what we call the common pricing problem, i.e. the problem to determine the functional relationship among a set of different performance criteria and then account for this relationship in the various optimizations throughout all system layers. In this paper we present an approach to first learn a probabilistic model of the metric interdependencies, and thereafter utilize this model for performance estimation and optimal task parameterization during planning and execution respectively. The proposed method is validated in a simulation.

The verification of evasive maneuvers for autonomous vehicles driving with constant velocity is considered. Modeling uncertainties, uncertain measurements, and disturbances can cause substantial deviations from an initially planned evasive maneuver. From this follows that the maneuver, which is safe under perfect conditions, might become unsafe. In this work, the possible set of deviations is computed with methods from reachability analysis, which allows to verify evasive maneuvers under consideration of the mentioned uncertainties. Since the presented approach has a short response time, it can be applied for real time safety decisions. The methods are presented for a numerical example where two autonomous cars plan a coordinated evasive maneuver in order to prevent a collision with a wrong-way driver.

This paper presents a first theoretical framework for a dialog strategy handling miscommunication in natural language Human-Robot Interaction (HRI). On the one hand the dialog strategy is deduced from findings about human-human communication patterns and coping strategies for miscommunication. On the other hand, relevant cognitive theories concerning human perception serve as a conceptual basis for the dialog strategy. The novel approach is firstly to combine these communication patterns with coping strategies and cognitive theories from human-human interaction (HHI) and secondly transfer them to HRI as a general dialog strategy for handling miscommunication. The presented approach is applicable to any task-oriented dialog. In a first step the conversational context is confined to route descriptions, given that asking for directions is an restricted but nevertheless challenging example for task-oriented dialog between humans and a robot.

Ball dribbling is a central element of basketball and a main challenge for creating basketball robots is to achieve stability of the periodic dribbling task. In this paper two control designs for ball dribbling with an industrial robot are compared. For the two strategies, the ball position is determined either through force/torque or visual sensor feedback and the ball trajectory is predicted with a recursive least squares algorithm. The end effector trajectory for each dribbling cycle is generated based on the predicted ball position/velocity at the dribbling height and the estimated coefficient of restitution. For both tracking approaches, dribbling for multiple cycles is achieved. The vision-based approach performs better as compared to the force/torque-based approach, in particular for imprecise estimates of the coefficient of restitution.

This video presents the Autonomous City Explorer (ACE) project. Its goal was to create a robot capable of navigating unknown urban environments without the use of GPS data or prior map knowledge. The robot had to find its way solely by interacting with pedestrians and building a topological representation of its surroundings. This video outlines the necessary ingredients for successful low-level navigation on sidewalks, information retrieval from pedestrians as well as the construction of a semantic representation of an urban environment. A system architecture for outdoor localization, traversability assessment, path planning, behavior selection and topological abstraction in urban environments is presented.

The creation of a robot capable of navigating in unknown urban environments without the use of GPS data or prior map knowledge is envisioned in the autonomous city explorer (ACE) project. The robot has to retrieve direction information solely by interacting with humans. This work presents a human-robot communication system that enables the robot to ask for directions and store the retrieved route information as internal knowledge. The system incorporates theories from linguistics in a mixed-modalities communication interface. It stores acquired information into a topological route graph which is used to give feedback to the human and to navigate in unknown environments.

As mobile robotics is gradually moving towards a level of semantic environment understanding, robust 3D object recognition plays an increasingly important role. One of the most crucial prerequisites for object recognition is a set of fast algorithms for geometry segmentation and extraction, which in turn rely on surface normal vectors as a fundamental feature. Although there exists a plethora of different approaches for estimating normal vectors from 3D point clouds, it is largely unclear which methods are preferable for online processing on a mobile robot. This paper presents a detailed analysis and comparison of existing methods for surface normal estimation with a special emphasis on the trade-off between quality and speed. The study sheds light on the computational complexity as well as the qualitative differences between methods and provides guidelines on choosing the dasiarightpsila algorithm for the robotics practitioner. The robustness of the methods with respect to noise and neighborhood size is analyzed. All algorithms are benchmarked with simulated as well as real 3D laser data obtained from a mobile robot.

In mobile robotics, the segmentation of range data is an important prerequisite to object recognition and environment understanding. This paper presents an algorithm for realtime segmentation of a continuous stream of incoming range data. The method is an extension of the previously developed RBNN algorithm and proceeds in two phases: Firstly, the normal vector of each incoming point is estimated from its neighborhood, which is continuously monitored. Secondly, new points are clustered according to their Euclidean and angular distance to previously clustered points. An outline of the algorithm complexity as well as the parameters that influence the segmentation performance is provided. Three benchmark scenarios in which the algorithm is deployed on a mobile robot with a laser range finder confirm that the method can robustly segment incoming data at high rates.

One of the greatest challenges nowadays in robotics is the advancement of robots from industrial tools to companions and helpers of humans, operating in natural, populated environments. In this respect, the Autonomous City Explorer (ACE) project aims to combine the research fields of autonomous mobile robot navigation and human robot interaction. A robot has been created that is capable of navigating in an unknown, highly populated, urban environment, based only on information extracted through interaction with passers-by and its local perception capabilities. This paper describes the algorithms and architecture that make up the navigation subsystem of ACE. More specifically, the algorithms used for Simultaneous Localization and Mapping (SLAM), path planning in dynamic environments and behavior selection are presented, as well as the system architecture that integrates them to a complete working system. Results from an extended field experiment, where the robot navigated autonomously through the downtown city area of Munich, are analyzed and show that the robot is capable of long-term, safe navigation in real-world settings.

Autonomous mobile robots are deployed in a variety of application domains, resulting in scenario specific implementations. However these systems share common components responsible for perception, path planning and task execution. In order to find a formal way to identify the influence of the environmental complexity to the used methods, an approach for quantitative system interdependence analysis is introduced. The coherence between several performance indicators of different system components, as well as the influence of environmental parameters on the system, are learned and quantitatively evaluated. Performance evaluation of an autonomous robot navigating in two different urban environments is conducted and presented results demonstrate the applicability of the proposed approach.

In this paper we present a novel method for the efficient segmentation of 3D laser range data. The proposed algorithm is based on a radially bounded nearest neighbor strategy and requires only two parameters. It yields deterministic, repeatable results and does not depend on any initialization procedure. The efficiency of the method is verified with synthetic and real 3D data.

Building a basketball robot is a recently launched project at the Institute of Automatic Control Engineering (LSR) for investigating fast manipulation with non-negligible dynamics and changing contact situation. This study presents one of the main preliminary results of the project which is balancing a basketball on a plate using a six degrees of freedom serial industrial robot based on pure haptic information. The Ball-On-Plate system has been one of the classical applications of control theory and many studies have been conducted on this issue. However, most of these studies employed vision systems to update the current state of the ball on the plate. In this paper, the concept of balancing a ball on a plate purely based on haptic information is discussed. The velocity of the basketball which rolls on a plate is estimated based on the force data measured by a force-torque (F/T) sensor mounted on the end-effector and the proposed control scheme brings the basketball back to a still standing on the plate. Experimental results are presented to validate the efficiency of the proposed control scheme.

In order to be truly autonomous, robots that operate in natural, populated environments must have the ability to create a model of these unpredictable dynamic environments and make use of this self-acquired uncertain knowledge to decide about their actions. A formal Bayesian framework is introduced, which enables recursive estimation of a dynamic environment model and action selection based on this estimate. Existing methods are combined to produce a working implementation of the proposed framework. A Rao-Blackwellized particle filter (RBPF) is deployed to address the simultaneous localization and mapping (SLAM) problem and combined with recursive conditional particle filters in order to track people in the vicinity of the robot. In this way, a complete model is provided, which is utilized for selecting the actions of the robot so that its uncertainty is kept under control and the likelihood of achieving its goals is increased. All developed algorithms have been applied to the domain of the autonomous city explorer robot and results from the implementation on the robotic platform are presented.

Robots acting in populated environments must be capable of safe but also time efficient navigation. Trying to completely avoid regions resulting from worst case predictions of the obstacle dynamics may leave no free space for a robot to move, especially in environments with high dynamic. This work presents an algorithm for a ldquosoftrdquo risk mapping of dynamic objects leaving the complete space free of static objects for path planning. Markov Chains are used to model the dynamics of moving persons and predict their potential future locations. These occlusion estimations are mapped into risk regions which serve to plan a path through potentially obstructed space searching for the trade-off between detour and time delay. The offline computation of the Markov Chain model keeps the computational effort low, making the approach suitable for online applications.

Several variations of methodological approaches are used to study the social acceptance in human-robot interaction. Due to the introduction of robots in the home, working practice and usage typically informing the design of new forms of technology are missing. Studying social acceptance in human-robot interaction thus needs new methodological concepts. We propose a so called breaching experiment with additional ethnographic observation to close this gap. To investigate the methodological concept we have been conducting a field trial on a public place. We gathered feedback using questionnaires, in order to estimate whether this method can be beneficially to evaluate social acceptance. We could show that breaching experiments can be a useful method to investigate social acceptance in the field.

In this paper, a control scheme that combines trajectory planning and gaze direction control for robotic exploration is presented. The objective is to calculate the gaze direction and simultaneously plan the trajectory of the robot over a given time horizon, so that localization and map estimation errors are minimized while the unknown environment is explored. Most existing approaches perform a greedy optimization for the trajectory generation only over the next time step and usually neglect the limited field of view of visual sensors and consequently the need for gaze direction control. In the proposed approach an information-based objective function is used, in order to perform multiple step planning of robot motion, which quantifies a trade-off between localization, map accuracy and exploration. Relative entropy is used as an information metric for the gaze direction control. The result is an intelligent exploring mobile robot, which produces an accurate model of the environment and can cope with very uncertain robot models and sensor measurements

As robots are gradually leaving highly structured factory environments and moving into human populated environments, they need to possess more complex cognitive abilities. Not only do they have to operate efficiently and safely in natural populated environments, but also be able to achieve higher levels of cooperation and interaction with humans. The Autonomous City Explorer (ACE) project envisions to create a robot that will autonomously navigate in an unstructured urban environment and find its way through interaction with humans. To achieve this, research results from the fields of autonomous navigation, path planning, environment modeling, and human-robot interaction are combined. In this paper a novel hardware platform is introduced, a system overview is given, the research foci of ACE are highlighted, approaches to the occurring challenges are proposed and analyzed, and finally some first results are presented.

An online posture modification method termed Jacobi compensation is proposed which is suitable to modify precalculated step trajectories for a humanoid robot in certain task coordinate directions. This method can account for modeling errors in trajectory precalculation by shifting e.g. the center of mass (CoM) or certain parts of the humanoid mechanism to increase walking stability and performance. A theoretical analysis of stability properties is given.

Methods for modeling, simulating and optimizing the dynamics, stability and performance of legged robot locomotion are discussed in this paper. It is demonstrated how these tools are used in the design, implementation and operation of a humanoid robot. The selection and integration of fundamental hard- and software needed for autonomous operation and high agility is presented for a recently developed fully-actuated 17 DoF humanoid. The results are additionally reported form simulations and gait optimizations completed during its development using a 3D dynamic biped model coupled with multiple physical and stability constraints.

This paper discusses the design concept and system development of a small and relatively fast walking, autonomous humanoid robot with 17 degrees-of-freedom (DoF). The selection of motor size and gear ratios is based on numerical optimization of detailed multibody dynamics and optimal control corresponding to fast steps of the robot with an envisioned target speed of more than 0.5 m/s. In this paper the design considerations based on numerical optimal control studies and the mechanical realization of the robot are presented including first investigations on the achievable performance of a decentralized, microcontroller-based control architecture.