In this work, we propose a novel Decision Transformer-based framework for tuning the parameters of Model Predictive Control (MPC). First, we show that an MPC scheme with quadratic cost, linear constraints, and a nominal linear model can reproduce the optimal solution of an infinite-horizon nonlinear regulation problem when the cost and constraint parameters are tuned based on the history of states and control inputs. Then, we formulate parameter tuning as a sequence modeling problem and develop a Decision Transformer-based framework, referred to as MPC-Decisioner, which leverages the attention mechanism of Decision Transformers to exploit historical and contextual information and generate MPC parameters online, conditioned on trajectories of costs, states, and past parameters. The resulting framework offers interpretability through the attention scores of the Transformer and achieves improved closed-loop performance compared to baseline MPC parameter tuning methods. Its effectiveness is demonstrated through simulation studies.

In this work, we develop an analytical framework that integrates opinion dynamics with a recommendation system. By incorporating elements such as collaborative filtering, we provide a precise characterization of how recommendation systems shape interpersonal interactions and influence opinion formation. Moreover, the property of the coevolution of both opinion dynamics and recommendation systems is also shown. Specifically, the convergence of this coevolutionary system is theoretically proved, and the mechanisms behind filter bubble formation are elucidated. Our analysis of the maximum number of opinion clusters shows how recommendation system parameters affect opinion grouping and polarization. Additionally, we incorporate the influence of propagators into our model and propose a reinforcement learning-based solution. The analysis and the propagation solution are demonstrated in simulation using the Yelp data set.

Grasping and manipulating various kinds of objects cooperatively is the core skill of a dual-arm robot when deployed as an autonomous agent in a human-centered environment. This requires fully exploiting the robot's versatility and dexterity. In this work, we propose a general framework for dual-arm manipulators that contains two correlative modules. The learning-based dexterity-reachability-aware perception module deals with vision-based bimanual grasping. It employs an end-to-end evaluation network and probabilistic modeling of the robot's reachability to deliver feasible and dexterity-optimum grasp pairs for unseen objects. The optimization-based versatility-oriented control module addresses the online cooperative manipulation control by using a hierarchical quadratic programming formulation. Self-collision avoidance and dual-arm manipulability ellipsoid tracking with high reliability and fidelity are simultaneously achieved based on a learned lightweight distance proxy function and a speed-level tracking technique on Riemannian manifold. Intrinsic system safety is guaranteed, and a novel interface for skill transfer is enabled. A long-horizon rearrangement experiment, a bimanual turnover manipulation, and multiple comparative performance evaluation verify the effectiveness of the proposed framework.

This paper presents a model predictive control (MPC) method for redundant robots controlling multiple hierarchical tasks formulated as multi-layer constrained optimal control problems (OCPs). The proposed method, named hierarchical incremental MPC (HIMPC), is robust to dynamic uncertainties, untethered from kinematic/algorithmic singularities, and capable of handling input and state constraints such as joint torque and position limits. To this end, we first derive robust incremental systems that approximate uncertain system dynamics without computing complex nonlinear functions or identifying model parameters. Then the constrained OCPs are cast as quadratic programming problems which result in linear MPC, where dynamically-consistent task priority is achieved by deploying equality constraints and optimal control is attained under input and state constraints. Moreover, hierarchical feasibility and recursive feasibility are theoretically proven. Since the computational complexity of HIMPC drastically decreases compared with nonlinear MPC-based methods, it is implemented under the sampling frequency of 1 kHz for physical experiments with redundant manipulator setups, where robustness (high tracking accuracy and enhanced dynamic consistency), admissibility of multiple constraints, and singularity-avoidance nature are demonstrated and compared with state-of-the-art task-prioritized controllers.

Observer-based methods are widely used to estimate the disturbances of different dynamic systems. However, a drawback of the conventional disturbance observers is that they all assume persistent excitation (PE) of the systems. As a result, they may lead to poor estimation precision when PE is not ensured, for instance, when the disturbance gain of the system is close to the singularity. In this paper, we propose a novel disturbance observer based on concurrent learning (CL) with time-variant history stacks, which ensures high estimation precision even in PE-free cases. The disturbance observer is designed in both continuous and discrete time. The estimation errors of the proposed method are proved to converge to a bounded set using the Lyapunov method. A history-sample-selection procedure is proposed to reduce the estimation error caused by the accumulation of old history samples. A simulation study on epidemic control shows that the proposed method produces higher estimation precision than the conventional disturbance observer when PE is not satisfied. This justifies the correctness of the proposed CL-based disturbance observer and verifies its applicability to solving practical problems.

For the safe application of reinforcement learning algorithms to high-dimensional nonlinear dynamical systems, a simplified system model is used to formulate a safe reinforcement learning (SRL) framework. Based on the simplified system model, a low-dimensional representation of the safe region is identified and used to provide safety estimates for learning algorithms. However, finding a satisfying simplified system model for complex dynamical systems usually requires a considerable amount of effort. To overcome this limitation, we propose a general data-driven approach that is able to efficiently learn a low-dimensional representation of the safe region. By employing an online adaptation method, the low-dimensional representation is updated using the feedback data to obtain more accurate safety estimates. The performance of the proposed approach for identifying the low-dimensional representation of the safe region is illustrated using the example of a quadcopter. The results demonstrate a more reliable and representative low-dimensional representation of the safe region compared with previous works, which extends the applicability of the SRL framework.

This article proposes an off-policy risk-sensitive reinforcement learning (RL)-based control framework to jointly optimize the task performance and constraint satisfaction in a disturbed environment. The risk-aware value function, constructed using the pseudo control and risk-sensitive input and state penalty terms, is introduced to convert the original constrained robust stabilization problem into an equivalent unconstrained optimal control problem. Then, an off-policy RL algorithm is developed to learn the approximate solution to the risk-aware value function. During the learning process, the associated approximate optimal control policy is able to satisfy both input and state constraints under disturbances. By replaying experience data to the off-policy weight update law of the critic neural network, the weight convergence is guaranteed. Moreover, online and offline algorithms are developed to serve as principled ways to record informative experience data to achieve a sufficient excitation required for the weight convergence. The proofs of system stability and weight convergence are provided. The Simulation results reveal the validity of the proposed control framework.

This article investigates the tracking problem of an uncertain$n$-link robot manipulator with guaranteed safety and performance. To tackle parametric uncertainties, the torque filtering-augmented concurrent learning (CL) method is introduced for online identification of the unknown system without requirements of joints acceleration. By using CL, the parameter convergence is guaranteed by exploiting the current and historical data simultaneously. This technique enjoys practicability compared with common methods that need to incorporate external noises to satisfy the persistence of excitation condition for the parameter convergence. Based on the estimated model, we design a barrier Lyapunov function (BLF)-based adaptive control law by the backstepping technique and Lyapunov analysis. By ensuring the boundness of the BLF, the system output and the tracking error are proved to lie in the safety set and performance set, respectively. Numerical simulation results and experiment tests validate the proposed strategy.

In this article, we investigate the adaptive output tracking control for multiinput–multioutput piecewise affine systems with prescribed performance. Both direct and indirect adaptation approaches are studied. Given a desired trajectory, both control approaches ensure the output tracking error to be confined within a performance bound, which prescribes the steady-state tracking error as well as the transient behavior, such as decaying rate and overshoot. We establish novel common Lyapunov functions without solving the conventional Lyapunov equations. Based on these common Lyapunov functions, the stability of the closed-loop system under arbitrary switching is established. Furthermore, the parameter convergence for both direct and indirect approaches is proved under the persistently exciting condition of the input signals. The dynamic gain adjustment technique is incorporated to counter the singularity problem in the indirect adaptation case. Finally, the numerical simulation validates the effectiveness and correctness of the proposed approaches in both direct and indirect adaptation cases.

Piecewise affine (PWA) systems are attractive models that can represent various hybrid systems with local affine subsystems and polyhedral regions due to their universal approximation properties. The identification problem of PWA systems amounts to estimating the number of subsystems, parameters of each subsystem, and the corresponding polyhedral partitions via state-input vectors. In this paper, we propose a novel approach to address the online identification problem of continuous-time PWA systems in state-space form. Specifically, an online active mode recognition algorithm and a generalized integral concurrent learning identifier are presented to acquire the number of subsystems, the switching sequence, and the parameter of each subsystem. In addition, we develop the optimization problem for the polyhedral partition estimation, which is solved by using the estimated switching sequence and subsystem parameters. The effectiveness of the proposed identification approach is demonstrated via simulation results.

Although the state-of-the-art learning approaches exhibit impressive results for dynamical systems, only a few applications on real physical systems have been presented. One major impediment is that the intermediate policy during the training procedure may result in behaviors that are not only harmful to the system itself but also to the environment. In essence, imposing safety guarantees for learning algorithms is vital for autonomous systems acting in the real world. In this article, we propose a computationally effective and general safe learning framework, specifically for complex dynamical systems. With a proper definition of the safe region, a supervisory control strategy, which switches the actions applied on the system between the learning-based controller and a predefined corrective controller, is given. A simplified system facilitates the estimation of the safe region for the high-dimensional dynamical system. During the learning phase, the belief of the safe region is updated with the actual execution results of the corrective controller, which in turn enables the learning-based controller to have more freedom in choosing its actions. Two examples are given to demonstrate the performance of the proposed framework, one simple inverted pendulum to illustrate the online adaptation method, and one quadcopter control task to show the overall performance.

Reinforcement learning (RL) is a general framework to acquire intelligent behavior by trial-and-error and many successful applications and impressive results have been reported in the field of robotics. In robot control problem settings, it is oftentimes characteristic that the algorithms have to learn online through interaction with the system while it is operating, and that both state and action spaces are continuous. Least-squares policy iteration (LSPI) based approaches are therefore particularly hard to employ in practice, and parameter tuning is a tedious and costly enterprise. In order to mitigate this problem, we derive an automatic online LSPI algorithm that operates over continuous action spaces and does not require an a-priori, hand-tuned value function approximation architecture. To this end, we first show how the kernel least-squares policy iteration algorithm can be modified to handle data online by recursive dictionary and learning update rules. Next, borrowing sparsification methods from kernel adaptive filtering, the continuous action-space approximation in the online least-squares policy iteration algorithm can be efficiently automated as well. We then propose a similarity-based information extrapolation for the recursive temporal difference update in order to perform the dictionary expansion step efficiently in both algorithms. The performance of the proposed algorithms is compared with respect to their batch or hand-tuned counterparts in a simulation study. The novel algorithms require less prior tuning and data is processed completely on the fly, yet the results indicate that similar performance can be obtained as by careful hand-tuning. Therefore, engineers from both robotics and AI can benefit from the proposed algorithms when an LSPI algorithm is faced with online data collection and tuning by experiment is costly.

When manipulating an object with multiple effectors such as in multidigit grasping or multiagent collaboration, forces and torques (i.e., wrench) applied to the object at different contact points generally do not fully contribute to the resultant object wrench, but partly compensate each other. The current literature, however, lacks a physically plausible decomposition of the applied wrench into its manipulation and internal components. We formulate the wrench decomposition as a convex optimization problem, minimizing the Euclidean norms of manipulation forces and torques. Physical plausibility in the optimization solution is ensured by constraining the internal and manipulation wrench by the applied wrench. We analyze specific cases of three-fingered grasping and 2-D beam manipulation, and show the applicability of our method to general object manipulation with multiple effectors. The wrench decomposition method is then extended to quantification of measures that are important in evaluating physical human-human and human-robot interaction tasks. We validate our approach via comparison to the state of the art in simulation and via application to a human-human object transport study.

The planning and execution of real-world robotic tasks largely depends on the ability to generate feasible motions online in response to changing environment conditions or goals. A spline deformation method is able to modify a given trajectory so that it matches the new boundary conditions, e.g., on positions, velocities, accelerations, etc. At the same time, the deformed motion preserves velocity, acceleration, jerk, or higher derivatives of motion profile of the precalculated trajectory. The deformed motion possessing such properties can be expressed by translation of original trajectory and spline interpolation. This spline decomposition considerably reduces the computational complexity and allows real-time execution. Formal feasibility guarantees are provided for the deformed trajectory and for the resulting torques. These guarantees are based on the special properties of Bernstein polynomials used for the deformation and on the structure of the chosen computed-torque control scheme. The approach is experimentally evaluated in a number of planar volleyball experiments using 3 degree-of-freedom robots and human participants.

This paper addresses a remaining gap between today's academic catching robots and their future in industrial applications: reliable task execution. A novel parameterization is derived to reduce the three-dimensional (3-D) catching problem to 1-D on the ballistic flight path. Vice versa, an efficient dynamical system formulation allows reconstruction of solutions from 1-D to 3-D. Hence, the body of the work in hybrid dynamical systems theory, in particular on the 1-D bouncing ball problem, becomes available for robotic catching. Uniform Zeno asymptotic stability from bouncing ball literature is adapted, as an example, and extended to fit the catching problem. A quantitative stability measure and the importance of the initial relative state between the object and end-effector are discussed. As a result, constrained dynamic optimization maximizes convergence speed while satisfying all kinematic and dynamic limits. Thus, for the first time, a quantitative success-oriented comparison of catching motions becomes available. The feasible and optimal solution is then validated on two symmetric robots autonomously playing throw and catch.

Reinforcement learning has become more and more popular in robotics for acquiring feedback controllers. Many approaches aim for learning a controller from scratch, i.e., data-driven without any modeling of the physical plant. However, stability properties of the closed loop are often not considered, or established only a-posteriori or ad hoc. We propose to employ reinforcement learning in the context of model-based control, allowing to learn in a framework of stabilizing controllers built by using only little prior model knowledge. This way, the action space is suitably structured for safe learning of a feedback controller to compensate for uncertainties due to model mismatch or external disturbances. The resulting scheme is developed around a decentralized PD feedback controller. Therefore, given such a controller, by the proposed method one can also add a learning module for performance enhancement. We demonstrate our approach both in simulation and in a hardware experiment using a two degree of freedom robot manipulator.

This paper introduces an online motion planning algorithm and a motion generation methodology for underactuated dynamic planar walking on uneven terrain. The key idea is to utilize a database of Motion Primitives and use them as training examples in a regression methodology, which is utilized when there is no match between the terrain variation and the Motion Primitives in the database. Among the key features which enable the algorithm to be suitable for real-time purposes is the proposed best first graph search approach and the small inference time of the regression methodology, which in this paper is the Gaussian Process.

This article discusses the quadratization of Markov Logic Networks, which enables efficient approximate MAP computation by means of maximum flows. The procedure relies on a pseudo-Boolean representation of the model, and allows handling models of any order. The employed pseudo-Boolean representation can be used to identify problems that are guaranteed to be solvable in low polynomial-time. Results on common benchmark problems show that the proposed approach finds optimal assignments for most variables in excellent computational time and approximate solutions that match the quality of ILP-based solvers.

Cooperative dynamic object manipulation increases the manipulation repertoire of multiagent teams. As a first step toward cooperative dynamic object manipulation, we present an energy-based controller for cooperative swinging of two-agent pendulum-like objects. Projection of the complex underactuated mechanism onto an abstract cart-pendulum allows us to separate desired and undesired oscillations. The desired oscillation is excited up to a desired energy level, while an undesired oscillation can be actively damped. Communication between the agents is restricted to force feedback. The controller can render leader and follower agents. The follower actively assists the swinging task by imitating the leader's energy flow. Real-world experiments with a robot and a human swinging complex pendulum-like objects are presented. The experimental results indicate that no simultaneous damping of the undesired oscillation is needed, also because it disturbs the human partner. A successful contribution of the robotic follower to the swing-up effort in interaction with a human leader supports the proposed control approach.

In this paper we propose to track online computed force trajectories to control the energy of various types of pendulum-like objects. The considered pendulum-like objects can be controlled by multiple agents and swing in different oscillation degrees of freedom. Our goal is to excite one specific oscillation, the intended oscillation, while damping all other disturbance oscillations. By approximating the intended oscillation as a simple pendulum oscillation, we can specify a desired force trajectory. Tracking of this force trajectory results in a controlled swing-up of the intended oscillation accompanied by a simultaneous damping of the disturbance oscillations. Simulation experiments with a two-agent trapezoidal pendulum show convincing control performance. A human-robot virtual reality experiment shows the transferability of the control approach to a human interaction partner. The limitations of the approach are discussed based on simulation results obtained for a single-agent double pendulum.

This paper presents an extension of the standard occupancy grid for 3D environment mapping. The presented approach adds a fusion process after the occupancy update which modifies the resolution of the grid cells in an incremental manner. Consequently, the proposed approach requires fewer grid cells for 3D representation in comparison to a standard occupancy grid. The resolution adaptation process is based on the occupancy probabilities of the grid cells and leads to the relaxation of the cubic grid cell assumption common to most 3D occupancy grids. The aim of this paper is to show the advantage of the proposed incremental fusion process which leads to the approximation of the 3D environment using rectangular cuboids. Evaluation on a large scale dataset and comparison to the state of the art shows that the proposed approach has faster access time for all occupied grid cells and requires a smaller number of cells for 3D environment representation.

Throwing an object by a powered robot system is of great importance in unmanned environments. In this paper, we consider the problem of throwing a point-mass object to minimize uncertainty in the object's landing position, given uncertainty in (1) the robot's initial configuration and (2) friction at the joints. Our analysis assumes that the robot's throw is executed using open-loop torque commands, and it relies on the linearized sensitivities of (a) landing location with respect to release state, (b) release state with respect to initial robot configuration and (c) joint friction. Moreover, the effectiveness of the proposed method is evaluated by Monte-Carlo simulations.

This paper discusses an online dynamic motion generation scheme for nonprehensile object manipulation by using a set of predefined motions and a trajectory deformation algorithm capable of incorporating positional and velocity boundary constraints. By creating optimal trajectories offline and deforming them online, computational complexity during execution is reduced considerably. As tight convex hulls of the deformed trajectories can be found, possible obstacles or workspace boundaries can be circumnavigated precisely without collision. The approach is verified through experiments on an inclined planar air-table for volleyball scenario using two 3-DoF robots.

One of the key challenges in dynamic manipulation is to establish continuous contact between a moving object and a manipulator. Direct robotic catching is a benchmark in this context, especially without grasping devices. In this paper, we explain why it is unrewarding to aim for an ideal initial dynamic contact if only imprecise knowledge of the object states is available. Robust initial contacts are proposed such that a successful quasi-direct catch can be predicted. The proposed robustness originates in a negative relative acceleration between object and catching end-effector. Sets, for which an upper negative relative acceleration bound holds, are evaluated throughout an exemplary catching motion. Reachable set computations allow to express and visualize these sets of successful object states at any point in time before contact. The paper closes with a robot-robot experiment that shows successful quasi-direct catches without feedback on the object states for a robotic throw.

This paper introduces a novel way to improve the settling time of transitions between different walking controllers. This improvement is achieved by commanding a sequence of intermediate transitions to the target controller. As a result, the state of the system enters the domain of attraction of the target controller closer to the fixed point of the Poincaré Map. The method is applicable to any walking robot with one degree of underactuation. The problem is expressed as a Markov Decision Process and then solved with Reinforcement Learning. In order to simplify the stability analysis of underactuated walking the Hybrid Zero Dynamics framework is utilized. Another advantage of using the Hybrid Zero Dynamics is the dimensionality reduction of the state representation in the Markov Decision Process. The experimental results suggest that the proposed methodology performs better than a one-step transition for 84.34% of all the considered transitions for a simulated walking robot matching the parameters of RABBIT [1].

This paper presents a new device and a method for dynamic manipulation. The device consists of a planar robotic arm and an elastic beam as an end-effector. Using it the elastic end-effector will tend to increase performance and energy efficiency while executing dynamic and repetitive tasks. Through the control of the beam vibration and resonant modes, we modify the state of manipulated objects. For lightweight objects the control is provided through the intermittent contacts without changing dynamics of the beam. However, we show that by using proper synchronization technique continuous-phase contacts are also possible. Juggling and dribbling of a ball are considered to be an alternating non-prehensile catching and throwing task. Such alternating decelerating and accelerating impacts on the ball and the curvature of the beam at the time of impact will stabilize the cyclic orbit of the ball. By proper analysis of continuous-time contact and dynamics of the beam we establish a rhythmic movement of the system. With the variation of frequency and amplitude of the beam it is possible to switch between different dynamic actions such as juggling, dribbling, throwing, catching and balancing.

Mapping and tracking in dynamic environments for autonomously-moving robots is still challenging, despite being essential tasks. They are often done separately using occupancy grids and established object tracking algorithms. In this work, an approach is presented that estimates a uniform, low-level, grid-based world model including dynamic and static objects, their uncertainties, as well as their velocities. It does not require existing object tracks to filter out data points not used for creating and updating the map. Nor does it require that measurements can be classified into belonging to a static or to a moving object. Promising results from experiments with an autonomous vehicle equipped with a laser scanner demonstrate the usefulness of the approach.

In this paper the states of inevitable collision for mobile robots are determined using reachable set theory. With this theory the safety for robotic platforms can be guaranteed, still allowing maximum flexibility for navigation. Making use of reachability analysis, limitations due to input sampling as in previous approaches are avoided. Using reachability analysis the obstacles are grown in the state space. The mathematical background is shown in this paper and an exemplary algorithm is given for static environments. This implementation can handle arbitrary environments with multiple obstacles and different high-dimensional linear and non-linear system dynamics including the car-like kinematic model. By means of experimental results in simulated environments, the validity of the proposed concept is shown.

Cooperative dynamic object manipulation can extend the manipulation capabilities of robot-robot and human-robot teams. In order to be able to inject energy into various suspended objects of unknown parameters, in this paper we propose an adaptive controller which combines reinforcement learning with energy based swing-up control. The proposed controller is successfully verified in a single robot and human-robot experimental setup for different types of suspended objects.

Catching is one of the most complex tasks in the area of dynamic manipulation. Exact information on the position and orientation of a rigid object is crucial in order to accomplish manipulation tasks. Both motion planner and control strategy use these data to achieve the desired contact of a predefined surface with a nonprehensile end-effector, e.g. flat plate. This paper presents a multi-level approach for robust task planning and execution for planar catching of rigid bodies. On the top level the choice of the best catching strategy is made. Different catching actions are introduced and classified based on relative translational and rotational velocities between the end-effector and the object. A motion planner is implemented on the middle level that produces smooth motion trajectories depending on the chosen strategy. Yet, some uncertainties occur during task execution due to sensory data, trajectory tracking and unmodeled dynamics. Therefore, a robust tracking control is implemented on the bottom level to guarantee task execution in presence of uncertainty in robot parameters. A sustainable framework is being used taking the dynamics of the robot, the object and the environment into account to create a consistent and versatile catching system.

This work presents a trajectory clustering approach that groups trajectories without the need of manually-tuned distance thresholds. Contrary to trajectory clustering approaches that use continuous, often geometrically-motivated similarity measures, path similarity is binary. Similar to homotopy classes, path equivalence is based on the obstacles in the environment. The goal states are, however, not fixed, but the paths have certain length restrictions. The equivalence is efficiently checked by closing the paths with sampled intermediate trajectories and using point-in-polygon tests. The proposed algorithm has linear complexity in the number of paths for non-overlapping clusters and, under certain assumptions, also in the case of overlapping clusters. Experimental results from an integration into a path-planning-based road course estimation system are shown and compared to a traditional distance-similarity cluster analysis to demonstrate the performance.

Humans are more successful in planning collision free, continuous trajectories through populated environments than any motion planning algorithm so far. This is due to the fact that they consider the conditionally cooperative, interactive behavior of the surrounding persons, for example the possibility of mutual avoidance maneuvers. In this paper, interaction during navigation is regarded from a game theoretic perspective and the concept of Nash equilibria is applied to analyze human motion. In contrast to other methods, the game theoretic approach does not necessarily rely on learning the interaction itself and is extendable. Our approach is based on human motion data that is captured during experiments. Two hypotheses are verified: for one thing, interaction exists during human navigation, for another thing, the mutual avoidance behavior of humans can be modeled with the theory of Nash equilibria in non-cooperative games. This knowledge can be used to enhance existing motion planning algorithms for autonomous robots.

Cooperative dynamic object manipulation extends the manipulation capabilities of human-robot dyads. This paper investigates cooperative swinging of rigid objects with the goal of reaching a desired level of energy, i.e. a desired object height. A human-human pilot study indicates that the arm-object-arm system can be approximated by a simple pendulum with two-sided unidirectional pulsed torque actuation. Based on the results of the human-human experiments, a robotic leader and follower controller are synthesized. Multi-body simulations based on human-like parameters successfully replicate the characteristics observed in the human-human experiments.

Emotion recognition from EEG signals allows the direct assessment of the “inner” state of a user, which is considered an important factor in human-machine-interaction. Many methods for feature extraction have been studied and the selection of both appropriate features and electrode locations is usually based on neuro-scientific findings. Their suitability for emotion recognition, however, has been tested using a small amount of distinct feature sets and on different, usually small data sets. A major limitation is that no systematic comparison of features exists. Therefore, we review feature extraction methods for emotion recognition from EEG based on 33 studies. An experiment is conducted comparing these features using machine learning techniques for feature selection on a self recorded data set. Results are presented with respect to performance of different feature selection methods, usage of selected feature types, and selection of electrode locations. Features selected by multivariate methods slightly outperform univariate methods. Advanced feature extraction techniques are found to have advantages over commonly used spectral power bands. Results also suggest preference to locations over parietal and centro-parietal lobes.

Collision checking is the major computational bottleneck for many robot path and motion planning applications, such as for autonomous vehicles, particularly with grid-based environment representations. Apart from collisions, many applications benefit from incorporating costs into planning; cost functions or cost maps are a common tool. Similar to checking a single configuration for collision, evaluating its cost using a grid-based cost map also requires examining every cell under the robot footprint. This work gives theoretical and practical insights on how to efficiently check a large number of configurations for collision and cost. As part of this work, configuration space costs are formulated, which can be seen as generalization of configuration space obstacles allowing a complete configuration check incorporating the robot geometry to be done using a single lookup. Furthermore, this paper presents two efficient algorithms for their calculation: FAMOD, an approximate method based on convolution, which is independent of the size and the shape of the robot mask, and vHGW-360, an exact method based on the van Herk-Gil-Werman morphological dilation algorithm, which can be used if the robot shape is rectangular. Both algorithms were implemented and evaluated on graphics hardware to demonstrate the applicability and benefit to real-time path and motion planning systems.

Emotion recognition from physiological signals like electroencephalography (EEG) can be performed using different underlying emotion models. While dimensional emotion models have recently gained attention, measures to evaluate recognition methods that are based on these models differ from study to study. This paper offers an analysis of proposed evaluation measures by comparing recognition results achieved on a self recorded dataset. Emotions are estimated using ridge regression and estimation results are compared using different evaluation measures. Additionally, three different baselines are studied, two types of random regression as well as naive estimation. Among the investigated evaluation measures, bandwidth accuracy was found to have many desirable characteristics.

Emotion recognition from EEG signals allows the direct assessment of the “inner” state of the user which is considered an important factor in Human-Machine-Interaction. Given the vast amount of possible features from scalp recordings and the high variance between subjects, a major challenge is to select electrodes and features that separate classes well. In most cases, this decision is made based on neuro-scientific knowledge. We propose a statistically-motivated electrode/feature selection procedure, based on Cohen's effect size f2. We compare inter- and intra-individual selection on a self-recorded database. Classification is evaluated using quadratic discriminant analysis (QDA). We found both feature selection versions based on f2yield comparable results. While highest accuracies up to 57,5% (5 classes) are reached by applying intra-individual selection, inter-individual analysis successfully finds features that perform with lower variance in recognition rates across subjects than combinations of electrodes/features suggested in literature.

This paper investigates goal-directed cooperative object swinging as a novel physical human-robot interaction scenario. We develop an energy-based control concept, which enables a robot to cooperate with a human in a goal-directed swing-up task. The robot can be assigned to be a leader or an actively contributing follower. We conduct a virtual reality experiment to compare effort sharing and performance of mixed human-human and human-robot dyads. The leader and the follower controllers yield similar results compared to their human standard.

This paper presents an approach to combine automatic semantic place labeling of robot-generated maps with reasoning on human route descriptions. Enabling robots to understand human route descriptions can simplify HRI situations in household or industrial settings. However, solving this problem requires handling the ambiguity present in route descriptions and the possible unreliability of the semantic perception capabilities of the robot. We address this problem by absorbing these uncertainties in a probability distribution measuring the likelihood of the different interpretations (paths) of a given route description and selecting its MAP solution. The approach is evaluated on a dataset of route descriptions transcribed into a suitable representation using standard information retrieval metrics. These performance measurements indicate that the method can correctly interpret route descriptions even in challenging environments.

This paper investigates human-robot cooperative swinging of a complex pendulum-like object. The complexity of the object results in two possible swinging modes. The goal is to excite one mode such that a desired energy level of the pendulum is reached while simultaneously damping the other mode. The energy based control concept relies on the projection of the complex mechanism onto an abstract simple pendulum with two-sided actuation. An actively contributing robot leader and robot follower are implemented. The controller performance is analyzed through simulations. A virtual reality experiment shows the transferability of the control approach to a human interaction partner.

In this paper energy optimal solutions for the approach of red traffic lights are derived. As cars waste most of the fuel in city traffic and especially in queuing at traffic lights, the presented framework provides solutions to save fuel and to protect the environment. The solutions are obtained using the definition of spent physical work which has to be minimized. It covers both cases, that the time of switching of the traffic lights is known and that the time of switching can only be modeled as a stochastic process. For a known time of switching a continuous solution is derived using Pontryagin Minimum Principle; in the stochastic case a modified Bellmann equation is formulated. The latter is solved with dynamic programming techniques. The presented solutions can be used for autonomous driving as well as for driving assistant systems. Simulation results show the potential savings using the presented approach.

This paper presents a throwing motion planner based on a goal manifold for two-point boundary value problem. The article outlines algorithmic and geometric issues for planar throwing of rigid objects with a nonprehensile end-effector. Special attention is paid to the challenge of controlling a desired 6-dimensional state of the object with a planar 3-DoF robot. Modeling of the contacts is discussed using a state vector of the coupled robot and object dynamics. Robustness against uncertainty due to varying model parameters such as object inertia and friction between the end-effector and the object is investigated. An approach for obtaining manifolds of terminal constraints from the goal configuration is described. Classification of these constraints is given. Finally, feasible trajectory generation conditions for successful execution of the generated optimal controls are discussed.

In this work, a framework for motion prediction of vehicles and safety assessment of traffic scenes is presented. The developed framework can be used for driver assistant systems as well as for autonomous driving applications. In order to assess the safety of the future trajectories of the vehicle, these systems require a prediction of the future motion of all traffic participants. As the traffic participants have a mutual influence on each other, the interaction of them is explicitly considered in this framework, which is inspired by an optimization problem. Taking the mutual influence of traffic participants into account, this framework differs from the existing approaches which consider the interaction only insufficiently, suffering reliability in real traffic scenes. For motion prediction, the collision probability of a vehicle performing a certain maneuver, is computed. Based on the safety evaluation and the assumption that drivers avoid collisions, the prediction is realized. Simulation scenarios and real-world results show the functionality.

The road course is an essential feature for many driver assistance systems and for autonomously-maneuvering vehicles. It is commonly stored in a map and hence assumed to be known a-priori. There are however situations in which the map data can become invalid, such as in road construction sites. In other situations, localization in the map might not be accurate enough, which can happen, for example, in dense urban areas. In this work, a novel approach to road course estimation is presented that is based on path planning through grid maps under non-holonomic and velocity constraints. With this approach, it is possible to estimate the road boundaries on a wide range of roads, including roads with continuous as well as discontinuous borders, roads exhibiting strong curvatures or S-shapes and road junctions. Furthermore, a plausibility measure is given to validate the road course and it is shown how the road center can be smoothed.

This paper introduces a computer-assisted teleoperation system, where the control over the teleoperator is shared between a human operator and computer assistance in order to improve the overall task performance. Two units, an action recognition and an assistance unit are introduced to provide context-specific assistance. The action recognition unit can evaluate haptic data, handle high sampling rates, and deal with human behavior changes caused by the actived haptic assistance. Repairing of a broken hard drive is selected as scenario and three different task-specific assistance functions are designed. The overall computer-assisted teleoperation system is evaluated in two steps: first, the performance of the action recognition unit is evaluated and then, the performance of the integrated computer-assisted teleoperation system is compared with an unassisted system by means of a user study with 15 participants. Overall action recognition rates of about 65% are achieved. Multivariate paired comparisons show that the computer-assisted teleoperation system significantly reduces the human effort and damage possibility compared with a teleoperation system without assistance.

In this paper, an online spatial and appearance based loop closure algorithm is presented. The approach is based on a graph matching formulation using Position Invariant Robust Features (PIRF), extending previous approaches based on PIRF by incorporating spatial information. The vertices of the graph represent visual words/features and edges represent metric information with uncertainty since the spatial distances observed between visual words are prone to errors. This method is capable of detecting loop closure in urban environments based on visual appearance as well as spatial layout of matched visual features. A vocabulary is built in an online and incremental manner, also storing spatial distances between visual words. The algorithm is capable of assigning loop closure with higher confidence values and a higher recall rate while maintaining precision compared to approaches where only visual appearance methods are used. We evaluate this approach on a publicly available dataset and present experimental results.

We propose an online compensation scheme for rollover prevention of mobile manipulators based on the invariance control framework, and that makes use of recursively computed analytic gradients of the zero-moment point (ZMP) function. Our controller relaxes many of the assumptions made in existing approaches, and enhances robustness as well as effectiveness through the use of exact gradient information. Several case studies demonstrate the improved performance of our controller over existing rollover prevention schemes.

In this video we present a teleoperation system which is capable of solving complex tasks in human-sized wide area environments. The system consists of two mobile teleoperators controlled by two operators, and offers haptic, visual, and auditory feedback. The task examined here, consists of repairing a robot by removing a computer and replacing a defective hard-drive. To cope with the complexity of such a task, we go beyond classical teleoperation by integrating several advanced software algorithms into the system.

This video presents a motion planning method enabling an autonomous mobile robot to approach a person to initiate a conversation proactively. The proposed concept implements social aspects to give motions a more human-like appearance. This is intended since experiments discussed in literature have shown that it is easier for people to predict and read the purpose of human-like movements. User study results show that the readability of the approach behavior is improved by including parameters such as path execution speed, distance between goal pose and person, position and orientation in front of the person, and trajectory shape. The video depicts the results of an implemented human approach planner for dynamic environments. It plans a trajectory towards a person applying the named set of social constraints. Simulations and real world experiments show how the approaching behavior smoothly brings the robot close to a person for interaction.

This work presents a novel approach for collision assessment for automotive environments. As collision prevention and risk analysis are key challenges for today's intelligent transport systems, sophisticated solutions for a collision-free trajectory generation get indispensable. The presented collision checker is integrated into an optimal control based planning framework that generates minimum jerk trajectories for arbitrary maneuvers. In contrast to common collision checkers, the developed method does not need to discretize the time space, but gives an algebraic solution which covers nearly all situations and has a constant response time. Due to its fast evaluation it is predestined for use in a Monte Carlo simulation respecting the probabilistic nature of real world traffic scenes. An example implementation of the proposed method is applied to simulation scenarios that demonstrates the benefits to comparable approaches. The obtained results have proven the feasibility of our approach to risk analysis of traffic situations.

Early stages in the development of a Brain-and-Body-Computer Interface controlled robot avatar are presented. The robot is aimed at performing well-defined daily tasks upon the choice and on behalf of a user. We built on recent advances in neuroscience, robotics and machine learning to demonstrate that it is possible to control a robot, accurately and reliably, by decoding scalp-recorded non-invasive Electroencephalographic (EEG) potentials into user intentions.

In comparison to industrial settings with structured environments, the operation of autonomous robots in unstructured and uncertain environments is more challenging. This video presents a generic control and system architecture ARCADE, applicable for real-time robot control in complex task situations. Several methods to cope with uncertainties are demonstrated with the example task of changing tires on a car. Approaches of object detection (applied to car, tires, and humans), robust real-time control of robot arms under perception uncertainty, and human-friendly haptic interaction are detailed. The video shows two robots jointly performing the task of mounting a mock-up tire to a real car using the proposed methods, realizing robust performance in an uncertain environment.

While humans can manipulate deformable objects smoothly and naturally, this is still a challenge for autonomous robots due to the complex object dynamics. The presence of rigid environment constraints and altering contact phases between the deformable object, the manipulator, and the environment makes this problem even more challenging. This paper presents a framework for deformable object manipulation that makes use of a single human demonstration of the task. The recorded trajectories are automatically segmented into a sequence of haptic control primitives involving contact with the rigid environment and vision-guided grasp primitives. The recorded motion/force trajectories serve as reference for a compliant control scheme in contact situations. In order to cope with positioning uncertainties a variable admittance control is proposed. The proposed approach is validated in an experimental mounting task for a deformable linear object with multiple re-grasping. The task is demonstrated with a multimodal teleoperation system and transfered to a robotic platform with a pair of seven degrees of freedom manipulators.

This paper presents a method for reasoning about the safety of traffic situations. More precisely, the problem of safety assessment for partial trajectories for vehicles is addressed. Therefore, the Inevitable Collision States (ICS) as well as its probabilistic generalization the Probabilistic Collision States (PCS) are used. Thereby, the assessment is performed for an infinite time horizon. For solving the ICS computation nonlinear programming is applied. In addition to the safety assessment an evaluation of the disturbance of the other traffic participants by the ego vehicle is presented. The results are integrated into an optimal control based planning approach that generates minimum jerk trajectories. An example implementation of the proposed framework is applied to simulation scenarios that demonstrates the necessity of the presented method for guaranteeing motion safety.

Haptic teleoperation enables the human to perform manipulation tasks in distant, scaled, hazardous, or inaccessible environments. The human closes the control loop sending haptic command signals to and receiving haptic feedback signals from the remote teleoperator. The main research question is how to design the control such that human decision making and action is supported in the best possible way while ensuring robust operation of the system. The human in the loop induces two major challenges for control design: 1) the dynamics of the human operator and the teleoperation system are tightly coupled, i.e., stability of the overall system is affected by the human operator dynamics; and 2) the performance of the teleoperation system is subjectively evaluated by the human, which typically means that standard control performance metrics are not suitable. This paper discusses recent control design successes in the area of haptic teleoperation. In particular, the importance and need of dynamic human haptic closed-loop behavior models and human perception models for the further improvement of haptic teleoperation systems is highlighted and discussed for real-world problem domains.

In a haptic shared control system, a virtual assistant and a human share the control over performed actions to facilitate execution of manipulation tasks. The assistance level determines the amount of support provided by the assistant. It should be adapted autonomously such that task performance and human effort are optimized. The effect of the assistance level on task performance and human effort may, however, be different depending on whether human and assistant agree on the actions or not. In this paper, we investigate the effect of the assistance level on task performance and effort for a scenario, in which human and assistant agree and for a scenario, where they disagree. We present a force-based criterion for distinguishing between the two scenarios and introduce an approach to optimize the assistance levels for each of the scenarios. Finally we sketch, how the results can be used to develop novel assistance adaptation schemes.

Human-machine collaborative teleoperation systems were introduced to overcome limitations of state-of-the-art teleoperation systems by using a virtual assistant that supports the human operator in the execution of a task. Since assistances are highly task-dependent a correct classification of the currently performed task is paramount. In this paper, we present a novel approach for improving task classification for a human-machine collaborative teleoperation system. Starting from a classical HMM-based classifier implemented in our previous research, we introduce a method for correcting erroneous task classifications by evaluating an agreement criterion. This criterion is based on interactive forces and is used to distinguish between situations in which human and assistant agree/disagree in their execution of the task. Using disagreement as indicator for the activation of an unsuitable/suboptimal assistance, erroneous task classifications are identified and the original classification result is revised. The proposed approach shows significant improvements in task classification coming along with a comparable low implementation effort.

This paper presents a probabilistic threat assessment method for reasoning about the safety of robot trajectories in uncertain and dynamic environments. For safety evaluation, the overall collision probability is used to rank candidate trajectories by considering the collision probability of known objects as well as the collision probability beyond the planning horizon. Monte Carlo sampling is used to estimate the collision probabilities. This concept is applied to a navigation framework that generates and selects trajectories in order to reach the goal location while minimizing the collision probability. Simulation scenarios are used to validate the overall crash probability and show its necessity in the proposed navigation approach.

This work presents an extension of the Dynamic Window Approach. The reactive collision avoidance algorithm is generalized to the case of omni-directional kinematics for any polygonal shaped, mobile robot. This paper includes a superior implementation to former realizations and has already been successfully tested with an omni-directional robot. The implementation is efficient and produces collision-free trajectories even in narrow and crowded environments.

For reasoning about the safety of a robot system, it is sufficient to pretend the robot to reach an Inevitable Collision Sate (ICS). Otherwise, there exists no future trajectory which can avoid a collision. The usage of ICS is limited due to its computational complexity. One reason for this is, that the ICS computation cannot be done separately for each obstacle. Hence, ICS needs to be recomputed from scratch if another object appears in the scene. The main contribution of this paper is a modified ICS calculation which allows to compute the union of ICS sets in a sequential manner, thus reducing the computational requirements in case of new obstacles. Therefore, two novel ICS-Checker algorithms are presented reducing the computational effort. Furthermore, this novel calculation is used to reduce the probability of being in an ICS regarding an unforeseen obstacle.

This paper implements a real-time hand gesture recognition algorithm based on the inexpensive Kinect sensor. The use of a depth sensor allows for complex 3D gestures where the system is robust to disturbing objects or persons in the background. A Haarlet-based hand gesture recognition system is implemented to detect hand gestures in any orientation, and more in particular pointing gestures while extracting the 3D pointing direction. The system is integrated on an interactive robot (based on ROS), allowing for real-time hand gesture interaction with the robot. Pointing gestures are translated into goals for the robot, telling him where to go. A demo scenario is presented where the robot looks for persons to interact with, asks for directions, and then detects a 3D pointing direction. The robot then explores his vicinity in the given direction and looks for a new person to interact with.

As communication quality in public spaces often is impaired by noisy environment, it is difficult for a robot to retrieve missing task-information from humans. In this paper, different dialog strategies are modeled and evaluated with respect to user experience and error handling capabilities in HRI in order to cope with erroneous speech recognition. Since correct recognition of spoken language is a bottleneck for real-world dialog systems, special emphasis is placed on the issue of adapting dialog strategies to the conditions under which the dialog is held to thereby provide for adaptability of the dialog strategy to variable speech recognition performance. Experimental evaluations are conducted in a fully automated indoor setting, and in a Wizard-of-Oz outdoor setting. Results indicate that a critical point exists, up to which the use of requests for handling miscommunication improves the user experience of a dialog strategy.

Abstract — In this paper, an approach is presented that allows a robot to navigate in an urban environment by following natural language route instructions. In this situation, neither maps nor GPS information are available to the robot thus it has to rely solely on the human-given route description and the observations from its sensors. An architecture for solving problems such as navigation on the sidewalk, street direction inference, and environment labeling that arise in this situation is presented. Our initial experiments indicate that the proposed methods enable a robot to safely navigate in urban environments by following abstract route descriptions and reach previously unknown points in a city. I.

This paper reports about a human-robot interaction field trial conducted with the autonomous mobile robot ACE (Autonomous City Explorer) in a public place, where the ACE robot needs the support of human passers-by to find its way to a target location. Since the robot does not possess any prior map knowledge or GPS support, it has to acquire missing information through interaction with humans. The robot thus has to initiate communication by asking for the way, and retrieves information from passers-by showing the way by gestures (pointing) and marking goal positions on a still image on the touch screen of the robot. The aims of the field trial where threefold: (1) Investigating the aptitude of the navigation architecture, (2) Evaluating the intuitiveness of the interaction concept for the passers-by, (3) Assessing people's willingness to support the ACE robot in its task, i.e. assessing the social acceptability. The field trial demonstrates that the architecture enables successful autonomous path finding without any prior map knowledge just by route directions given by passers-by. An additional street survey and observational data moreover attests the intuitiveness of the interaction paradigm and the high acceptability of the ACE robot in the public place.

In a haptic teleoperation system, the incorporation of knowledge about the remote environment in the controller design can improve stability and performance. Model-mediated teleoperation adopts this idea by rendering an estimated model of the remote environment on local site instead of transmitting force/velocity flows. Thus, the user perceives locally generated forces corresponding to the estimated and transmitted model parameters and the control loop between master and slave is opened. Less conservative stability boundaries and the applicability to teleoperation systems with arbitrary time delay are the main advantages of this approach. In order to guarantee a high fidelity, the estimation has to fit well with the measurements. In this paper, we extend the approach of model-mediated teleoperation to a full 6 degrees-of-freedom (DOF) teleoperation system with negligible time delay. We furthermore propose a hybrid approach for the estimation of the remote environment by combining the classical Kelvin-Voigt model and the nonlinear Hunt-Crossley model. Persistent excitation and device-dependent limitations of the estimation algorithm are discussed. Experimental results show stability and accuracy of the estimation technique as well as a superior fidelity of the proposed approach compared to a position-based admittance controller with fixed parameters even with negligible time delay.

For path planning algorithms of robots it is important that the robot does not reach a state of inevitable collision. In crowded environments with many humans or robots, the set of possible inevitable collision states (ICS) is often unacceptably high, such that the robot has to stop and wait in too many situations. For this reason, the concept of ICS is extended to probabilistic collision states (PCS), which estimates the collision probability for a given state. This allows to efficiently run planning algorithms through crowded environments when accepting a certain collision probability. A further novelty is that the obstacles possibly react to the robot in order to mitigate the risk of a collision. The results show a significant difference in interaction behavior. Thus, this approach is especially suited for active and non-deterministic moving obstacles in the robot workspace.

The collaborative research center SFB453 (www.sfb453.de) aims to realize high-fidelity telepresence and teleaction systems. Telepresence and teleaction systems extend the human workspace to remote locations in order to overcome barriers like distance, scaling, danger or the human skin. Using a human-system interface the human operator controls a remotely located teleoperator. Multi-modal feedback in form of visual, auditory, and haptic data is used to increase the feeling of telepresence. Different application areas including minimally invasive surgery, on-orbit servicing, microassembly as well as tele-manufacturing and tele-maintenance are targeted.

When robots leave industrial settings, they have to be designed allowing intuitive communication with the humans they interact with. The current paper focuses on collaboration in kinesthetic tasks. Herein, we investigate decision situations. This way, the need of communication between partners can be addressed. The current paper introduces for the first time an experimental paradigm which allows studying the effect of decision making in haptic collaboration. Because reciprocal haptic feedback is challenging to provide, we analyze its efficiency in human-human collaboration to understand when it is worth to invest in this additional modality. A one degree of tracking experiment with two human partners revealed that the additional physical effort accompanying reciprocal haptic feedback is directly transformed into higher performance (compared to a control condition without reciprocal haptic feedback). Thus, the presented results motivate further research on the nature of the haptic negotiation between human partners to achieve the same performance benefits in kinesthetic collaboration with robotic partners.

Despite the great diversity of teleoperator designs and applications, their underlying control systems have many similarities. These similarities can be exploited to enable inter-operability between heterogeneous systems. We have developed a network data specification, the Interoperable Telerobotics Protocol, that can be used for Internet based control of a wide range of teleoperators. In this work we test interoperable telerobotics on the global Internet, focusing on the telesurgery application domain. Fourteen globally dispersed telerobotic master and slave systems were connected in thirty trials in one twenty four hour period. Users performed common manipulation tasks to demonstrate effective master-slave operation. With twenty eight (93%) successful, unique connections the results show a high potential for standardizing telerobotic operation. Furthermore, new paradigms for telesurgical operation and training are presented, including a networked surgery trainer and upper-limb exoskeleton control of micro-manipulators.

An online intention recognition algorithm for computer-assisted teleoperation is introduced. The algorithm is able to distinguish between phases of a typical object manipulation task. It adopts a new advanced feature extraction algorithm which extracts features from haptic data and uses a Hidden Markov Model for stochastic classification. The method is implemented and validated on a real hardware setup. The obtained results reveal a robust and fast intention recognition.

A classification of model-mediated teleoperation systems according to the model type and its application is introduced. While models of the human operator estimating position trajectories have been already applied in teleoperation, in the present paper, we propose the incorporation of force-based human haptic interaction models. This new approach allows to transfer the strength of advanced model-mediated teleoperation, i.e. increased stability and fidelity, to scenarios where forces applied to the remote environment are of importance. As an application example we present a tele-rehabilitation scenario which was implemented on a 1 DoF teleoperation system. Position and force fusion algorithms integrating human haptic interaction models are defined for time delay and packet loss compensation. The results demonstrate clearly the benefit of incorporating force-based human haptic interaction models into teleoperation systems.

Knowledge about the remote environment can be used in the control law to improve robustness and fidelity of haptic teleoperation systems. Model-mediated teleoperation adopts this idea by rendering an estimated model of the remote environment on local site instead of transmitting force/velocity flows. In this paper, we extend the original model-mediated teleoperation approach to multi-operator multi-robot teleoperation systems. A theoretic robustness and fidelity analysis is conducted. The theoretical results show a superior performance of the proposed method compared to a classic bilateral approach. Experimental results confirm the practical efficiency of the presented approach.

The overall performance of a robotic system is commonly expressed by a single scenario-specific metric which is supposed to be optimized. However, the metric describing the performance of a single subtask within a scenario may be different. Nevertheless, the scenario performance is most likely dependent on the subtask performances but a mutual transformation is not straightforward in general, especially in complex robotic systems. This leads to what we call the common pricing problem, i.e. the problem to determine the functional relationship among a set of different performance criteria and then account for this relationship in the various optimizations throughout all system layers. In this paper we present an approach to first learn a probabilistic model of the metric interdependencies, and thereafter utilize this model for performance estimation and optimal task parameterization during planning and execution respectively. The proposed method is validated in a simulation.

The verification of evasive maneuvers for autonomous vehicles driving with constant velocity is considered. Modeling uncertainties, uncertain measurements, and disturbances can cause substantial deviations from an initially planned evasive maneuver. From this follows that the maneuver, which is safe under perfect conditions, might become unsafe. In this work, the possible set of deviations is computed with methods from reachability analysis, which allows to verify evasive maneuvers under consideration of the mentioned uncertainties. Since the presented approach has a short response time, it can be applied for real time safety decisions. The methods are presented for a numerical example where two autonomous cars plan a coordinated evasive maneuver in order to prevent a collision with a wrong-way driver.

This paper presents a first theoretical framework for a dialog strategy handling miscommunication in natural language Human-Robot Interaction (HRI). On the one hand the dialog strategy is deduced from findings about human-human communication patterns and coping strategies for miscommunication. On the other hand, relevant cognitive theories concerning human perception serve as a conceptual basis for the dialog strategy. The novel approach is firstly to combine these communication patterns with coping strategies and cognitive theories from human-human interaction (HHI) and secondly transfer them to HRI as a general dialog strategy for handling miscommunication. The presented approach is applicable to any task-oriented dialog. In a first step the conversational context is confined to route descriptions, given that asking for directions is an restricted but nevertheless challenging example for task-oriented dialog between humans and a robot.

This paper studies dancing as a haptic interaction task with the aim of replacing one of the human dancing partners by an interactive robot. Starting from haptic recordings of human dancing couples, the interactive behavior of a male dancing partner is synthesized. For this purpose recorded dancing steps are approximated using multiple vector fields and faded into each other by simply switching between them. Adaptation to the female style of dancing is realized by implementing an adaptation law which changes the step size of the robot. Experimental results performed on a mobile haptic interface show the validity of the presented approach.

Integration of emotions enhances naturalness of human-robot interaction (HRI). This requires that the robot is equipped with hardware to express emotions. Believability and recognition of expressions is increased if the same emotional state is expressed in all modalities which the robot is capable of. Within this aspect, our work analyzes if a walking robot can express emotions in the way it walks and if these expressions are recognizable. The emotive gait patterns are derived from human characteristics for emotive gait. The parameters step length, height and time for a single step vary depending on the emotion. Mapping these changes to the kinematics of the robot and exaggeration of the walking styles leads to distinguishable expressions for the dimensions pleasure, arousal and dominance. Experimental results on a hexapod show that differences in arousal are best expressed and thus recognized. Comparing an animation of the hexapod with the real robot indicates that the robot is perceived as slightly more pleasant and active. This study shows that by changing its walking style the hexapod expresses emotions, in particular differences in arousal, which can be used to increase expressiveness in HRI.

In this paper, autonomous switching between two basic attention selection mechanisms, i.e., top-down and bottom-up, is proposed. This approach fills a gap in object search using conventional top-down biased bottom-up attention selection, which fails, if a group of objects is searched whose appearances cannot be uniquely described by low-level features used in bottom-up computational models. Three internal robot states, such as observing, operating, and exploring, are included to determine the visual selection behavior. A vision-guided mobile robot equipped with an active stereo camera is used to demonstrate our strategy and evaluate the performance experimentally. This approach facilitates adaptations of visual behavior to different internal robot states and benefits further development toward cognitive visual perception in the robotics domain.

To provide a means for recognition of affect from a distance, this paper analyzes the capability of gait to reveal a person's affective state. We address interindividual versus person-dependent recognition, recognition based on discrete affective states versus recognition based on affective dimensions, and efficient feature extraction with respect to affect. Principal component analysis (PCA), kernel PCA, linear discriminant analysis, and general discriminant analysis are compared to either reduce temporal information in gait or extract relevant features for classification. Although expression of affect in gait is covered by the primary task of locomotion, person-dependent recognition of motion capture data reaches 95% accuracy based on the observation of a single stride. In particular, different levels of arousal and dominance are suitable for being recognized in gait. It is concluded that gait can be used as an additional modality for the recognition of affect. Application scenarios include monitoring in high-security areas, human-robot interaction, and cognitive home environments.

We are interested in whether humans create a model of their partner when they jointly manipulate an object in a virtual task without haptic feedback. In such a scenario the partner is perceived as a disturbance because she/he is responsible for inconsistencies between the visual and proprioceptive feedback of the individual. To gain basic knowledge on the predictability of such disturbances we compare a pre-recorded human partner with predictable (time delay) and unpredictable (random) disturbances and two additional control conditions in a pursuit tracking task. Results indicate that the influence of the pre-recorded partner is partly predictable, therefore we assume that a model of the partner's behavior is built by the human.

Ball dribbling is a central element of basketball and a main challenge for creating basketball robots is to achieve stability of the periodic dribbling task. In this paper two control designs for ball dribbling with an industrial robot are compared. For the two strategies, the ball position is determined either through force/torque or visual sensor feedback and the ball trajectory is predicted with a recursive least squares algorithm. The end effector trajectory for each dribbling cycle is generated based on the predicted ball position/velocity at the dribbling height and the estimated coefficient of restitution. For both tracking approaches, dribbling for multiple cycles is achieved. The vision-based approach performs better as compared to the force/torque-based approach, in particular for imprecise estimates of the coefficient of restitution.

This video presents the Autonomous City Explorer (ACE) project. Its goal was to create a robot capable of navigating unknown urban environments without the use of GPS data or prior map knowledge. The robot had to find its way solely by interacting with pedestrians and building a topological representation of its surroundings. This video outlines the necessary ingredients for successful low-level navigation on sidewalks, information retrieval from pedestrians as well as the construction of a semantic representation of an urban environment. A system architecture for outdoor localization, traversability assessment, path planning, behavior selection and topological abstraction in urban environments is presented.

The creation of a robot capable of navigating in unknown urban environments without the use of GPS data or prior map knowledge is envisioned in the autonomous city explorer (ACE) project. The robot has to retrieve direction information solely by interacting with humans. This work presents a human-robot communication system that enables the robot to ask for directions and store the retrieved route information as internal knowledge. The system incorporates theories from linguistics in a mixed-modalities communication interface. It stores acquired information into a topological route graph which is used to give feedback to the human and to navigate in unknown environments.

As mobile robotics is gradually moving towards a level of semantic environment understanding, robust 3D object recognition plays an increasingly important role. One of the most crucial prerequisites for object recognition is a set of fast algorithms for geometry segmentation and extraction, which in turn rely on surface normal vectors as a fundamental feature. Although there exists a plethora of different approaches for estimating normal vectors from 3D point clouds, it is largely unclear which methods are preferable for online processing on a mobile robot. This paper presents a detailed analysis and comparison of existing methods for surface normal estimation with a special emphasis on the trade-off between quality and speed. The study sheds light on the computational complexity as well as the qualitative differences between methods and provides guidelines on choosing the dasiarightpsila algorithm for the robotics practitioner. The robustness of the methods with respect to noise and neighborhood size is analyzed. All algorithms are benchmarked with simulated as well as real 3D laser data obtained from a mobile robot.

In mobile robotics, the segmentation of range data is an important prerequisite to object recognition and environment understanding. This paper presents an algorithm for realtime segmentation of a continuous stream of incoming range data. The method is an extension of the previously developed RBNN algorithm and proceeds in two phases: Firstly, the normal vector of each incoming point is estimated from its neighborhood, which is continuously monitored. Secondly, new points are clustered according to their Euclidean and angular distance to previously clustered points. An outline of the algorithm complexity as well as the parameters that influence the segmentation performance is provided. Three benchmark scenarios in which the algorithm is deployed on a mobile robot with a laser range finder confirm that the method can robustly segment incoming data at high rates.

A new lossy compression method is proposed for haptic (force, velocity) data as exchanged in bilateral telepresence systems. The method is based on the passive extrapolative compression strategy proposed in the work of Kuschel et al. (2006). The innovation is that the extrapolations do not have a stiff horizon, but are triggered by considerable changes (events) in the target environment. This enables longer average extrapolation horizons and thus, higher compression. Experiments are conducted using two DLR light weight robots. The results indicate that the method outperforms older implementations.

One of the greatest challenges nowadays in robotics is the advancement of robots from industrial tools to companions and helpers of humans, operating in natural, populated environments. In this respect, the Autonomous City Explorer (ACE) project aims to combine the research fields of autonomous mobile robot navigation and human robot interaction. A robot has been created that is capable of navigating in an unknown, highly populated, urban environment, based only on information extracted through interaction with passers-by and its local perception capabilities. This paper describes the algorithms and architecture that make up the navigation subsystem of ACE. More specifically, the algorithms used for Simultaneous Localization and Mapping (SLAM), path planning in dynamic environments and behavior selection are presented, as well as the system architecture that integrates them to a complete working system. Results from an extended field experiment, where the robot navigated autonomously through the downtown city area of Munich, are analyzed and show that the robot is capable of long-term, safe navigation in real-world settings.

In the autonomous city explorer (ACE) project a mobile robot is developed, which is capable of finding its way to a given destination in an unknown urban environment. An exemplary mission is to find the way from our institute to the Marienplatz, a public place in the center of Munich, without any prior knowledge or GPS information. Inspired by the behavior of humans in unknown environments, ACE must find its way by asking pedestrians. The route is about 1.5 kilometers far and includes heavily traveled roads and crowded public places. In order to navigate safely in an unknown urban environment, some challenges arise for the vision system. Robust human detection, tracking and the estimation of human body poses is essential for natural interaction with pedestrians. Furthermore, the robot needs to be able to detect sidewalk and crossroads. A visual odometry system is used to support the conventional navigation. Outdoor experiments were conducted twice successfully. After about 5 hours and interacting with 25 and 38 persons respectively, ACE arrived the Marienplatz. This paper describes both, an architecture of the vision system used for ACE and the algorithms used to deal with the described challenges.

The influence of ankle compliance on bipedal robot locomotion is investigated in this paper. The focus is on reduction of energy consumption. The concept of hybrid zero dynamics is adapted to design walking gaits with three phases: underactuated heel roll, full actuation and underactuated toe roll. Ankle springs work in parallel with the ankle actuators. Stiffness and offset of the linear torsional springs at the ankle and gait parameters are optimized simultaneously. It is shown that simultaneous optimization of spring properties and gait is superior to optimizing the spring after the gait. Optimal spring stiffness and offset lead to a major reduction in energy consumption. Furthermore, a more human-like gait is observed for simultaneous optimization of gait and spring parameters compared to gait optimization with zero stiffness.

In this paper, an insect-inspired motion detector (Reichardt-model) is applied to visual servo control to ensure the stability of the system with high gain and time delay in its feedback. A Reichardt-based control scheme is compared with a conventional visual servoing approach. As a consequence of the specific velocity dependence of the Reichardt-model, the stability margin of the visual servo control is increased and high overall gains, thus, better performance are achievable. The response of the Reichardt-model in the experiment and the control performance of velocity control approach with the Reichardt-model in the closed loop are investigated. The velocity control model is tested on a 1-DOF linear motor module with different feedback gain and different time delay in the loop. The results of simulation and realtime experiments demonstrate the stabilizing character of the Reichardt-based approach.

In this paper a novel implementation of the saliency map model on a multi-GPU platform using CUDA technology is presented. The saliency map model is a well-known computational model for bottom-up attention selection and serves as a basis of many attention control strategies of cognitive vision systems. A real-time implementation is the prerequisite of an application of bottom-up attention on mobile robots and vehicles. Parallel computation on graphics processing unit (GPU) provides an excellent solution for this kind of compute-intensive image processing. Running on 1 to 4 NVIDIA GeForce 8800 (GTX) graphics cards a frame rate of 313 fps at resolution of 640 times 480 is achieved, which is approximately 8.5 times faster than the standard implementations on CPUs. The implementation is also evaluated using a high-speed camera at 200 Hz. Using two GPUs only 2 ms extra computational time for the saliency map generation in addition to the camera capture time is required for images of 640 times 480 pixels.

A biologically inspired foveated attention system in an object detection scenario is proposed. Thereby, a high-performance active multi-focal camera system imitates visual behaviors such as scan, saccade and fixation. Bottom-up attention uses wide-angle stereo data to select a sequence of fixation points in the peripheral field of view. Successive saccade and fixation of high foveal resolution using a telephoto camera enables high accurate object recognition. Once an object is recognized as target object, the bottom-up attention model is adapted to the current environment, using the top-down information extracted from this target object. The bottom-up attention model and the object recognition algorithm based on SIFT are implemented using CUDA technology on Graphics Processing Units (GPUs), which highly accelerates image processing. In the experimental evaluation, all the target objects were detected in different backgrounds. Evident improvements in accuracy, flexibility and efficiency are achieved.

This paper is the continuation of our previous work in intercontinental, collaborative teleoperation with a humanoid robot. Our new achievement consists in an extension of the former single-arm bilateral teleoperation setting to include bimanual manipulation and walking. A coupling scheme for simultaneous manipulation and locomotion is developed. Furthermore, a task-based control framework, including a force-based control for the arms as well as a walking pattern generation, is presented to realize stable whole-body motions of the highly redundant humanoid robot. Experiments have been performed to assess the proposed control scheme. They bring to light additional scientific challenges that remain in order to reach a smooth and natural telepresent collaboration.

Although it is reported in the literature that haptic feedback leads to improved performance in kinesthetic collaborative tasks, it has not been investigated so far whether this advantage is accompanied by a higher physical workload. This paper is an initial effort to examine efficiency in haptic interaction: We relate physical effort to a performance outcome in a virtual pursuit tracking task. An experimental study is conducted to compare efficiency in a collaborative mutual haptic feedback condition to three control conditions, where participants either acted alone or collaboratively without haptic feedback from the partner. Results show that reciprocal haptic feedback does not improve efficiency, although participants' performance was generally improved when doing the task with a partner, relative to executing it alone. This is due to the greater effort associated with physical connection between partners. However, the effort is more fairly distributed between partners when haptic feedback from the partner is provided. Haptic feedback may be more efficient when the amount of necessary communication between partners increases compared to the task studied here.

Autonomous mobile robots are deployed in a variety of application domains, resulting in scenario specific implementations. However these systems share common components responsible for perception, path planning and task execution. In order to find a formal way to identify the influence of the environmental complexity to the used methods, an approach for quantitative system interdependence analysis is introduced. The coherence between several performance indicators of different system components, as well as the influence of environmental parameters on the system, are learned and quantitatively evaluated. Performance evaluation of an autonomous robot navigating in two different urban environments is conducted and presented results demonstrate the applicability of the proposed approach.

In this paper an autonomous switching between two basic attention selection mechanisms, top-down and bottom-up, is proposed, substituting manual switching. This approach fills the gab in object search using conventional top-down biased bottom-up attention selection: the latter one fails, if a group of objects is searched whose appearances can not be uniquely described by low-level features used in bottom-up computation models. Two internal robot states, observing and operating, are included to determine the visual selection behavior. A vision guided mobile robot, equipped with an active stereo camera, is used to demonstrate our strategy and evaluate the performance experimentally.

The goal of the autonomous city explorer (ACE) is to navigate autonomously, efficiently and safely in an unpredictable and unstructured urban environment. To achieve this aim, an accurate localization is one of the preconditions. Due to the characteristics of our navigation environment, an elaborated visual odometry system is proposed to estimate the current position and orientation of the ACE platform. The existing algorithms of optical flow computation are experimentally evaluated and compared. The method based on pyramidal Lucas-Kanade algorithm with high-speed performance is selected. Based on the optical flow in 2D images, the camera ego-motion is estimated using image Jacobian matrix and least squares method. The kinematic model is set up to map the camera ego-motion to the robot motion. To eliminate systematic errors, a novel system calibration approach is proposed. Finally the odometry system is evaluated in experiments.

Telesurgery systems integrate multimodal communication and robotic technologies to enable surgical procedures to be performed from remote locations. They allow human surgeons to intuitively control laparoscopic instruments and to navigate within the human body. In this paper, we present selected topics on multimodal interaction in the context of telesurgery applications. These are results from the collaborative research project SFB 453 on ldquoHigh-Fidelity Telepresence and Teleactionrdquo which is funded by the German Research Foundation in the larger Munich area. The focus in this paper is on multimodal information processing and communication including simulation of surgical targets in the human body. Furthermore, we present an overview of our advanced multimodal telesurgery demonstrators that provide a comprehensive platform for our collaborative telepresence research.

Achieving natural and intuitive interaction is one of the main challenges in physical human-robot interaction. We approach this challenge by modeling haptic human-human interaction with the final goal of transferring found relationships to human-robot interaction. The focus of this paper is on two human operators performing collaboratively a joint object manipulation, i.e. a pursuit tracking task. McRuer's crossover model is a well established method to describe the behavior of one human operator performing such a task. In this paper, we extent McRuer's approach to two human operators performing the task collaboratively. Results based on experimetally gained data show that the interacting partners adapt their behavior to each other and to the task in such a way that the crossover model can still be applied to the interacting dyad. It is also shown that the individual's behavior changes when interacting with a partner in contrast to performing the task alone.

Recent research focuses on developing robots that are meant to be partners of humans instead of pure machines. This makes enhanced communication necessary. Especially in scenarios embedding physical interaction between the two partners dominance is an urgent matter. To overcome one-sided dominance as in passive following or trajectory replay in favor of intuitive collaboration, human-human collaboration and the involved dominance distribution needs to be addressed. Even though some attempts are reported in literature, to our best knowledge no experimental analysis of dominance distribution in a kinesthetic task reports actual values of dominance. Therefore, the current paper discusses dominance measures appropriate in haptic interaction and investigates the dominance distribution in a tracking-task experiment. In the analysis we focus on the influence of mutual haptic feedback between the partners on dominance distribution by contrasting this condition to vision-only partner feedback trials. Furthermore, this paper investigates the consistency of dominance behavior across different partners based on methodologies transferred from social psychology. Results show that participants work with a dominance distribution, whereby the feedback condition does not effect this distribution. A high amount of variability in individual dominance behavior can be considered person dependent. Here, feedback has an effect as the dominance behavior is even more stable across partners when mutual haptic feedback is provided.

This work proposes a Piecewise Linear (PL) system to model transitions of affect. Parameters of the model are identified based on a psychological experiment. The PL system describes affective reactions of humans to an external affective stimulus depending on the previous affective state. Results of the statistical analysis support that the previous affective state influences significantly the current affective state. Evaluation of the model shows that it is suitable as mathematical representation for the development of affect over time under the influence of an external stimulus. A following system-theoretic analysis of the model reveals that the PL system shows complex dynamic characteristics. It suggests that internal affective fluctuations exist.

In the project Autonomous City Explorer, an interactive robot is designed to find its way to a given destination in unknown urban environments by interacting with pedestrians. Considering applications in a human dominated environment, the robot can be sent to a destination by tracking a landmark selected by users and described by 2D image features. To achieve a natural landmark selection from the user perspective and an accurate feature tracking for a safe robot navigation, the robot preselects visual features and presents the users only the image regions providing higher tracking accuracy. Furthermore, a multi-focal camera system is used to extend the sensing range. SIFT, Harris corner and optical flow used for tracking and self-localization are compared and applied to different visual sensors. A coordination strategy is realized, in which the camera with wide field of view is used for robot orientation control and the high-resolution camera is applied for robot forward motion control. The performance is experimentally evaluated.

The safety of the planned paths of autonomous cars with respect to the movement of other traffic participants is considered. Therefore, the stochastic occupancy of the road by other vehicles is predicted. The prediction considers uncertainties originating from the measurements and the possible behaviors of other traffic participants. In addition, the interaction of traffic participants, as well as the limitation of driving maneuvers due to the road geometry, is considered. The result of the presented approach is the probability of a crash for a specific trajectory of the autonomous car. The presented approach is efficient as most of the intensive computations are performed offline, which results in a lean online algorithm for real-time application.

In this paper, lossy-compression methods for haptic (velocity and force) data as exchanged in telepresence or virtual reality systems are introduced. Based on the proposed interpolative and extrapolative compression strategies, arbitrary passive compression algorithms can be implemented. The derived algorithms do not affect the stability of the presence system. Two algorithms were implemented and experimentally evaluated. The results show that constant data-rate savings of 89% still lead to a perceptually transparent presence system.

Knowledge about the environment is essential for humanoid and mobile robots to move and act safely. The most intuitive way to perceive information about the environment is through the vision system. However, the accuracy provided by stereo vision is insufficient for many tasks. A more accurate representation is created by a laser range-finder, which delivers no color information. This paper describes a novel approach to merge data obtained by stereo vision and laser range-finders by using a genetic ICP algorithm, which is able to register noisy point clouds with different resolutions and a small overlap. Furthermore, it describes an easy-to-use and robust method to calibrate the extrinsic parameters of two or more laser range-finders.

Inspired by the expectation-based perception of humans, a surprise-driven active vision system is proposed. This vision system not only considers spatial saliency of objects in the environment, but also investigates temporal novelty in the neighborhood. Surprise is defined as the difference of the saliency probability distributions of two consecutive input images, which is measured using Kullback-Leibler divergence. The high-speed gaze shift capability of the camera platform and the parallel computation with the aid of GPUs enable a real-time tracking of the surprising event.

Goal-directed guidance of gaze control based on coordinated task and stimulus parameters is essential for steering a mobile cognitive system efficiently and autonomously through the real world. This paper focuses on coordination mechanisms of top-down and bottom-up attentional allocation, with particular consideration of the current local environment. The top-down attention selection in the task-space and the bottom-up attention selection in the image-space is evaluated and combined using information theory. An information-based scene context classification considering scene dynamics is proposed to bias attention selection, which is the main contribution of this paper. Experiments are conducted to evaluate the performance.

In this paper we present a novel method for the efficient segmentation of 3D laser range data. The proposed algorithm is based on a radially bounded nearest neighbor strategy and requires only two parameters. It yields deterministic, repeatable results and does not depend on any initialization procedure. The efficiency of the method is verified with synthetic and real 3D data.

The hardware and control concept of a mobile haptic interface is presented. It is intended to provide spatially unrestricted, dual-handed haptic interaction. The device is composed of two haptic displays mounted on an omnidirectional mobile base which is controlled in such a way that the haptic displays are not driven to their workspace limits. A simple algorithm, based on end-effector positions only, and a more sophisticated approach, incorporating also the body position of the operator, are presented and compared. Experimental results show that the latter algorithm performs better in most use cases.

In this paper, an array of biologically inspired elementary motion detectors (EMDs) is implemented on an FPGA (Field Programmable Gate Array) platform. The well-known Reichardt-type EMD, modeling the insect’s visual signal processing system, is very sensitive to motion direction and has low computational cost. A modified structure of EMD is used to detect local optical flow. Six templates of receptive fields, according to the fly’s vision system, are designed for simple ego-motion estimation. The results of several typical experiments demonstrate local detection of optical flow and simple motion estimation under specific backgrounds. The performance of the real-time implementation is sufficient to deal with a video frame rate of 350 fps at 256 x 256 pixels resolution. The execution of the motion detection algorithm and the resulting time delay is only 0.25 μs. This hardware is suited for obstacle detection, motion estimation and UAV/MAV attitude control.

The video shows a rich multi-modal multi-user telepresence system, which was developed within the SFB453 funded by the German Research Foundation (www.sfb453.de). As a complex application scenario, the remote repairing of a broken pipe is presented in this paper. The system basically consists of two operator-teleoperator- pairs. While one of the operators interacts with a stationary human- system-interface, the other operator uses a mobile one. Both systems provide visual, auditory, and haptic feedback and enable to control the motion of head, arms, and grippers as well as the locomotion of the corresponding teleoperator.

This paper presents a new redundancy resolution approach based on the gradient projection method, which allows to avoid man-machine and machine-machine collisions as well as singularities of a redundant haptic interface. On this account cost functions for singularity and collision avoidance are formulated. Hereby the collision avoidance cost function is derived from a kinematic analysis of the human arm. Appropriate weighting factors are used to guarantee a simultaneous optimization of all defined side criteria. The performance of the presented method is analyzed by simulation.

In order to be truly autonomous, robots that operate in natural, populated environments must have the ability to create a model of these unpredictable dynamic environments and make use of this self-acquired uncertain knowledge to decide about their actions. A formal Bayesian framework is introduced, which enables recursive estimation of a dynamic environment model and action selection based on this estimate. Existing methods are combined to produce a working implementation of the proposed framework. A Rao-Blackwellized particle filter (RBPF) is deployed to address the simultaneous localization and mapping (SLAM) problem and combined with recursive conditional particle filters in order to track people in the vicinity of the robot. In this way, a complete model is provided, which is utilized for selecting the actions of the robot so that its uncertainty is kept under control and the likelihood of achieving its goals is increased. All developed algorithms have been applied to the domain of the autonomous city explorer robot and results from the implementation on the robotic platform are presented.

One of the main challenges in telerobotics is the selection of control architectures and control parameters, which are able to robustly stabilize the overall teleoperation system despite of changing human operator and environment impedances. In this paper robust stability of different types of bilateral control algorithms for admittance-type devices is analyzed. Hereby stability of the system is investigated by using the parameter space approach, which allows the analysis of uncertain systems with varying plant parameters. Stability of the teleoperation system is analyzed by using linear models for human-system interface and teleoperator. The parameter space approach is adopted for controller design as well as for robustness analysis. Robust stability of the presented control architectures is evaluated for a real mechatronic teleoperation system.

In multimodal tele-cooperation as considered in this paper two humans in distant locations jointly perform a task requiring multimodal including haptic feedback. One human operator teleoperates a remotely placed humanoid robot which is collocated with the human cooperator. Time delay in the communication channel as destabilizing factor is one of the multiple challenges associated with such a tele-cooperation setup. In this paper we employ a control architecture with force-position exchange accounting for the admittance type of the haptic input device and the telerobot, which both are position-based admittance controlled. Llewellynpsilas stability criteria are employed for the parameter tuning of the virtual impedances in the presence of time delay. The control strategy is successfully validated in an intercontinental tele-cooperation experiment with the humanoid telerobot HRP-2 located in Japan/Tsukuba and a multimodal human-system-interface located in Germany/Munich, see also the corresponding video submission. The proposed setup gives rise to a large number of exciting new research questions to be addressed in the future.

The video shows an intercontinental cooperative telemanipulation task, whereby the operator site is located in Munich, Germany and the teleoperator site in Tsukuba, Japan. The human operator controls a remotely located teleoperator, which performs a task in the remote environment. Hereby the human operator is assisted by another person located at the remote site. The task consists in jointly grasping an object, moving it to a new position and finally releasing it, see Fig. 1.

Robots acting in populated environments must be capable of safe but also time efficient navigation. Trying to completely avoid regions resulting from worst case predictions of the obstacle dynamics may leave no free space for a robot to move, especially in environments with high dynamic. This work presents an algorithm for a ldquosoftrdquo risk mapping of dynamic objects leaving the complete space free of static objects for path planning. Markov Chains are used to model the dynamics of moving persons and predict their potential future locations. These occlusion estimations are mapped into risk regions which serve to plan a path through potentially obstructed space searching for the trade-off between detour and time delay. The offline computation of the Markov Chain model keeps the computational effort low, making the approach suitable for online applications.

In this work, we present novel VR training strategies that incorporate a hybrid trainer model to train force. For modeling the trainer skill, weighted K-means algorithm in parameter space with LS optimization is implemented. The efficiency of the training strategies is verified via user tests in frame of a bone drilling training application. An objective evaluation method based on n dimensional Euclidean distances is introduced to assess user tests results. It is shown that the proposed strategies improve the student skill and accelerate force learning.

Estimating the human body pose is of great interest for many tasks, such as human robot interaction, people tracking and surveillance. During the recent years, several approaches have been presented, which still have weaknesses regarding occlusions or complex scenes. In this paper, we present a novel algorithm for human body pose estimation using any three-dimensional representation of the environment, like stereo vision. The presented algorithm is able to leave out body parts and is therefore able to deal with occluded body parts. In a first step, possible humans need to be detected, e.g. by using a skin color filter. A disparity map containing depth information is computed using a stereo matching algorithm. It leads to a three-dimensional representation of the scene. Starting with the detected skin parts, our algorithm segments this point cloud into smaller clusters. The possible matches are then verified, and the body pose is estimated using a kinematic human model with 28 degrees of freedom. As our algorithm is capable of dealing with arbitrary three-dimensional representations, it can easily be adapted to use a three-dimensional laser range finder instead of a stereo camera system.

If a teleoperation scenario foresees complex and fine manipulation tasks a multi-fingered telemanipulation system is required. In this paper a multi-fingered telemanipulation system is presented, whereby the human hand controls a three-finger robotic gripper and force feedback is provided by using an exoskeleton. Since the human hand and robotic grippers have different kinematic structures, appropriate mappings for forces and positions are applied. A point-to-point position mapping algorithm as well as a simple force mapping algorithm are presented and evaluated in a real experimental setup.

Several variations of methodological approaches are used to study the social acceptance in human-robot interaction. Due to the introduction of robots in the home, working practice and usage typically informing the design of new forms of technology are missing. Studying social acceptance in human-robot interaction thus needs new methodological concepts. We propose a so called breaching experiment with additional ethnographic observation to close this gap. To investigate the methodological concept we have been conducting a field trial on a public place. We gathered feedback using questionnaires, in order to estimate whether this method can be beneficially to evaluate social acceptance. We could show that breaching experiments can be a useful method to investigate social acceptance in the field.

Perception of compliant environments through a human system interface with visual and proprioceptive feedback is investigated. Participants had to explore the virtual environment by gripping with two fingers. Haptically, compliance was generated by an admittance control scheme. Perception of compliance under conflicting multimodal information was analyzed using an adaptive staircase method. Inter alia, experiments showed that conflicts could be detected when compliances by both modalities differ more than 55%.

Common works on emotion expressing robots are theoretically based on a dimensional (continuous) model of emotions. Nevertheless, performance tests, which are used to evaluate the emotion expressing robots, are based on categorical (discrete) models of emotions. In this paper the use of dimensional-based tests is suggested, e.g. semantic differential approaches like the pleasure-arousal-dominance-model. By deriving the test from the theory on which the design of an object is based, the validity of the test raises significantly. Major benefits are explicit guidelines for design improvement and the possible integration of arbitrary actuated expressive features for which no common framework as, for example, the facial action coding system (FACS) exists. For illustration purposes, a comparative evaluation study of the robot EDDIE is conducted: one test is based on a categorical model and one test is based on a dimensional model of emotion. A third study based on a dimensional model demonstrates the evaluation of the influence of animal like features on the perceived emotion state.

In this paper, a control scheme that combines trajectory planning and gaze direction control for robotic exploration is presented. The objective is to calculate the gaze direction and simultaneously plan the trajectory of the robot over a given time horizon, so that localization and map estimation errors are minimized while the unknown environment is explored. Most existing approaches perform a greedy optimization for the trajectory generation only over the next time step and usually neglect the limited field of view of visual sensors and consequently the need for gaze direction control. In the proposed approach an information-based objective function is used, in order to perform multiple step planning of robot motion, which quantifies a trade-off between localization, map accuracy and exploration. Relative entropy is used as an information metric for the gaze direction control. The result is an intelligent exploring mobile robot, which produces an accurate model of the environment and can cope with very uncertain robot models and sensor measurements

Conventional approaches for collision avoidance often use the gradient projection method on minimum distance function. This paper discusses several shortcomings to draw special attention when it is applied to avoid collision with multiple obstacles. As a remedy to the identified problems, a novel method of collision avoidance based on Jacobian transpose method is proposed, which does not require to calculate the gradient of the minimum distance function, hence resulting in a computationally simple algorithm. In addition, an efficient collision detection algorithm for all possible spatial as well as planar configurations is presented. Simulation results with a four Degrees of Freedom planar manipulator are given to illustrate the several shortcomings of conventional methods and to validate the proposed method.

As robots are gradually leaving highly structured factory environments and moving into human populated environments, they need to possess more complex cognitive abilities. Not only do they have to operate efficiently and safely in natural populated environments, but also be able to achieve higher levels of cooperation and interaction with humans. The Autonomous City Explorer (ACE) project envisions to create a robot that will autonomously navigate in an unstructured urban environment and find its way through interaction with humans. To achieve this, research results from the fields of autonomous navigation, path planning, environment modeling, and human-robot interaction are combined. In this paper a novel hardware platform is introduced, a system overview is given, the research foci of ACE are highlighted, approaches to the occurring challenges are proposed and analyzed, and finally some first results are presented.

The concept of a mobile haptic interface for bimanual manipulations in 6 d.o.f. is presented. The design of this mobile haptic interface is based on a modular system consisting of two components: two haptic interfaces and a mobile platform. This work mainly addresses the design and control concepts of the haptic interfaces, which are planned to be mounted on the mobile platform. The interfaces dispose of a large workspace and a high force/torque capability. The design and control concepts of these new interfaces enable a decoupling of translational from rotational movements. Such a decoupling helps to simplify significantly the control algorithms which take care of the interaction between mobile platform and haptic interfaces. Evaluation results concerning the Cartesian position tracking performance and the impedance display fidelity provide an insight into the performance of the developed system.

The principle of series elastic actuation offers considerable advantages for haptic displays compared to stiff actuators. The interaction force between motor and load is directly proportional to their relative position, which corresponds to the elongation of the elastic element. This way, the torque control task is transformed to a position control task, which comes natural to traditional DC motors. In this paper, several existing control strategies are analyzed and compared with respect to passivity concerns. Cascaded control with a fast inner velocity loop results to be the best option. Based on the analysis, boundaries for the parameters are presented, such that the force controller may contain integral action without jeopardizing passivity.

In this paper a multi-user virtual training system concept for force skill training is presented. A trainer and a student connect to a common virtual environment via their individual haptic displays. The student realizes the virtual task, while the trainer observes him/her visually, acoustically and/or haptically. The trainer has a chance to correct the student verbally or haptically. Alternatively, the observation duty is given to the student, while the trainer demonstrates a task. Depending on the combinations of these observation/correction possibilities, three novel training strategies have been developed. Force information is exchanged on a separate communication line, which minimizes the time delay. The efficiency of the developed training strategies is verified via user tests on the bone drilling medical training system that was developed in our laboratories previously. Last, but not least, a special attention is paid to objective assessment of the user tests' results. Thus, a method based on distances in n dimensional Euclidean space is developed and implemented.

A novel multi-focal four camera vision system is presented. The dynamic performance reaches velocities and accelerations exceeding far beyond human capabilities. Extreme wide-angle aperture angles to narrow foveated are provided simultaneously and are independently controllable. The vision system is designed in the context of the new development of the successor model of the humanoid robot Johnnie targeting at real-world wide distance range applications. This paper presents the design, kinematic and perception models, and mechatronical integration. The dynamic performance is evaluated in experiments. A novel multi-camera view direction stabilization strategy is presented and experimentally validated

Telepresence systems are often deployed in scenarios where communication bandwidth is limited. Consequently, data exchanged between operator and teleoperator has to be reduced. In case of haptic telepresence, data reduction has an influence on the stability of the overall system. This paper provides a step towards a systematic framework for communication data bandwidth reduction in haptic telepresence systems discussing stability for a class of lossy data reduction (LDR) algorithms. Simulation and experimental results validate the efficacy

A novel camera view direction stabilization strategy based on a multi-camera vision system is proposed. Relative motion information of an observed target with respect to the camera system is acquired by at least two cameras: a tracking and a stabilizing camera. The stabilizing camera provides a velocity contribution to the tracking camera motion controlled by visual servoing. Thereby, a significant reduction of the tracking error is achieved. The proposed strategy is evaluated in simulations and experiments based on a novel multi-focal vision system designed for the humanoid LOLA

This paper presents a novel method of redundancy resolution with multiple performance criteria. In this method the homogenous solution of differential kinematics is derived as a weighted sum of joint velocities lying in the null space of the manipulator Jacobian instead of establishing an overall objective function as a weighted sum of multiple criteria. It remarkably increases flexibility in adding new secondary performance criteria since it does not require to recalculate the gradient of new overall objective function. A systematic design of variable weights is introduced to actively reflect the instant states of each criterion in real-time. Experimental results with a 7-DOF redundant robot illustrate the validity of the proposed scheme

The video shows a telepresence system which allows to perfom an assembly task in an extensive remote environment. It comprises a telemanipulator with two anthropomorphic 7-DOF arms controlled by a hyper-redundant 10-DOF haptic display. Dexterous manipulation capabilities are achieved by employing 3-finger grippers at the teleoperator site and data gloves at the operator site. Mounting the telemanipulator on an omnidirectional mobile platform additionally enables the teleoperator to freely move around in the remote environment. Thereby the locomotion is controlled via a specifically designed 3-DOF pedal. The video shows an assembly experiment with toy bricks which involves locomotion, navigation and dextrous manipulation tasks.

This paper focuses on the development of EDDIE, a flexible low-cost emotion-display with 23 degrees of freedom. Actuators are assigned to particular action units of the facial action coding system (FACS). Emotion states represented by the circumplex model of affect are mapped to individual action units. Thereby, continuous, dynamic, and realistic emotion state transitions are achieved. EDDIE is largely developed and manufactured in a rapid-prototyping process. Miniature off-the-shelf mechatronics components are used providing high functionality at low-cost. Evaluations conducted in a user-study show that emotions can be recognized very well. Further experiments show that additional features adapted from animals have significant but small influence on the display of the human emotion 'disgust'

EDDIE, a novel mechatronical emotion-display designed for dynamic non-verbal human-robot interaction is presented. A special feature are dynamic and realistic emotional state transitions. Therefore, the emotional state-space based on the circumplex model of affect is directly mapped to joint space. The display is largely developed and manufactured in a rapid-prototyping process. Only miniature off-the-shelf mechatronic components are used providing high functionality at low cost.

The performance of EEG-based Brain-Computer-Interfaces (BCIs) critically depends on the extraction of features from the EEG carrying information relevant for the classification of different mental states. For BCIs employing imaginary movements of different limbs, the method of Common Spatial Patterns (CSP) has been shown to achieve excellent classification results. The CSP-algorithm however suffers from a lack of robustness, requiring training data without artifacts for good performance. To overcome this lack of robustness, we propose an adaptive spatial filter that replaces the training data in the CSP approach by a-priori information. More specifically, we design an adaptive spatial filter that maximizes the ratio of the variance of the electric field originating in a predefined region of interest (ROI) and the overall variance of the measured EEG. Since it is known that the component of the EEG used for discriminating imaginary movements originates in the motor cortex, we design two adaptive spatial filters with the ROIs centered in the hand areas of the left and right motor cortex. We then use these to classify EEG data recorded during imaginary movements of the right and left hand of three subjects, and show that the adaptive spatial filters outperform the CSP-algorithm, enabling classification rates of up to 94.7 % without artifact rejection.

This paper focuses on the development of EDDIE, a flexible low-cost emotion-display with 23 degrees of freedom. Actuators are assigned to particular action units of the facial action coding system (FACS). Emotion states represented by the circumplex model of affect are mapped to individual action units. Thereby, continuous, dynamic, and realistic emotion state transitions are achieved. EDDIE is largely developed and manufactured in a rapid-prototyping process. Miniature off-the-shelf mechatronics are used providing high functionality while extremely low-cost. Evaluations are conducted based on a user-study

Given a linear and instantaneous mixture model, we prove that for blind source separation (BSS) algorithms based on mutual information, only sources with non-Gaussian distribution are consistently reconstructed independent of initial conditions. This allows the identification of non-Gaussian sources and consequently the identification of signal and noise subspaces through BSS. The results are illustrated with a simple example, and the implications for a variety of signal processing applications, such as denoising and model identification, are discussed.

One of the key challenges in telepresence and teleaction systems is the fact that a global control loop is closed over a communication network. The transmission delay of haptic information is extremely critical. Therefore, new data samples from the haptic sensors are typically immediately forwarded to the receiver which leads to a large number of packets being generated when using the Internet as the communication infrastructure. We present a novel approach to reduce the number of packets and, therefore, the amount data communicated in a telepresence and teleaction system. Our method uses a passive deadband transmission approach which only delivers data packets over the network when the sampled sensor data changes more than a given threshold value. The threshold value is determined by psychophysical experiments. This approach leads to a considerable reduction (up to 90%) of packet rate and data rate without sacrificing the fidelity and immersiveness of the system.

In the paper an approach for motion control of a wide class of robotic systems is presented. In this approach, the advantages of the sliding mode control (SMC) like robustness and simplicity of the control law are used. At the same time, the main disadvantage of the SMC - chattering - is avoided or at least reduced. It is shown that this approach is applicable to a wide class of robots. For this class of robots, the stability of the closed loop system is proven. The performance of the presented approach is demonstrated in a real experiment with a two arm 2/spl times/7 DoF humanoid manipulator.

Visual servoing has become an important and powerful tool in robot control. However, the operating range is restricted by the properties of the vision sensor. Thus, in a wide range visual servoing workspace the utilization of one sensor with fixed properties is not advisable. In this work the impact of focal length and operating distance on visual servoing performance in presence of noise and quantization is investigated. The impact of sensor selection for different distances with respect to control performance is revealed. A multifocal visual servoing strategy is presented.

This work addresses some practical issues regarding development of a telerobotic system for 6 degrees of freedom (DoF) tasks. The system consists of a hyper redundant haptic input device ViSHaRD10, a redundant 7DoF manipulator and a stereo vision system. The redundancy of the devices is exploited to assure large convex workspace and singularity-free operation. The anthropomorphic construction of the telemanipulator enables intuitive manipulation and increases the "user-friendliness" of the overall system. As a practical benchmark an assembly experiment in 6DoF for a case of a negligible time delay was successfully performed. Issues regarding inverse kinematics, spatial interaction control, transparency, and intuitiveness of teleoperation are discussed.

Legged locomotion with its variable contact situations between feet and ground is of interest in today's research on humanoid locomotion. This paper uses a hybrid modeling framework to account for different possible ground contact situations of a simple biped gymnast experimental platform with five joints. For the considered biped unactuated rotation around foot edges as well as plane foot contact is considered in modeling and trajectory planning. Presented is a planning algorithm for periodic walking trajectories. The stability of the resulting hybrid periodic orbits is investigated in numerical experiments.

Based on existing knowledge on human tactile movement perception, we constructed a prototype of a novel tactile multipin display that controls lateral pin displacement and, thus produces shear force. Two experiments focus on the question of whether the prototype display generates tactile stimulation that is appropriate for the sensitivity of human tactile perception. In particular, Experiment I studied human resolution for distinguishing between different directions of pin displacement and Experiment II explored the perceptual integration of information resulting from the displacement of multiple pins. Both experiments demonstrated that humans can discriminate between directions of the displacements, and also that the technically realized resolution of the display exceeds the perceptual resolution (>14°). Experiment II demonstrated that the human brain does not process stimulation from the different pins of the display independent of one another at least concerning direction. The acquired psychophysical knowledge based on this new technology will in return be used to improve the design of the display.

In this work a virtual reality medical training system (MTS) for bone drilling skill training is presented. For this purpose a novel controller algorithm, two graphical user interfaces for teaching and simulating purposes are developed. A 3 DOF haptic display that is developed in our laboratory is used for force feedback. Several user tests are performed to validate how the developed MTS helps for skill improvement. The effects of adding acoustic feedback and the advantages of using a 3 DOF haptic display instead of 1 DOF are investigated. Preliminary user studies suggest that the proposed system can be a powerful tool for teaching how to drill into bone.

We discuss the modeling and control of mechatronic tilting systems. Mechatronic tilting systems are for example walking machines where the feet tilt over their edges. Tilting means that the mechanical system rolls passively (nonactuated) over one of its edges in contact with the ground. A mathematical model and a periodic trajectory planning method using a hybrid (discrete-continuous) dynamical systems approach are developed. The simple model system consists of an actuated arm, which is fixed on a base plate that can tilt over its edges. The planned trajectories are then used in simulation and proper control strategies are discussed.

This work addresses the manipulation and control problems of a teleoperated redundant manipulator. The choice of a benchmark of extremely high requirements is technical aid in catastrophic situations like road accidents, fires and natural disasters. Such scenarios demand intuitive though powerful operation minimizing user fatigue and tension. For this reason, replicating the human motion abilities by the telemanipulator is the main challenge in the presented design. A recently developed 6 degrees of freedom (DoF) haptic input device is used as master arm and an anthropomorphic, human sized 7 DoF manipulator as the slave arm. One of the problems studied here is the connection of two kinematically dissimilar devices working in master-slave configuration with 6 degrees of freedom kinesthetic feedback, issues regarding kinematic control, compliant motion, transparency, and intuitiveness of teleoperation are discussed.

An online posture modification method termed Jacobi compensation is proposed which is suitable to modify precalculated step trajectories for a humanoid robot in certain task coordinate directions. This method can account for modeling errors in trajectory precalculation by shifting e.g. the center of mass (CoM) or certain parts of the humanoid mechanism to increase walking stability and performance. A theoretical analysis of stability properties is given.

Methods for modeling, simulating and optimizing the dynamics, stability and performance of legged robot locomotion are discussed in this paper. It is demonstrated how these tools are used in the design, implementation and operation of a humanoid robot. The selection and integration of fundamental hard- and software needed for autonomous operation and high agility is presented for a recently developed fully-actuated 17 DoF humanoid. The results are additionally reported form simulations and gait optimizations completed during its development using a 3D dynamic biped model coupled with multiple physical and stability constraints.

In this paper we present the optimized design of the 6 degree-of-freedom (DOF) haptic display VISHARD6, which is directed towards modularity, universal applicability, and high force capability. To investigate the performance regarding force, velocity, and acceleration a generic method for output performance analysis of haptic interfaces is proposed. The device is evaluated according to these measures and experimental results are presented.

This paper discusses the design concept and system development of a small and relatively fast walking, autonomous humanoid robot with 17 degrees-of-freedom (DoF). The selection of motor size and gear ratios is based on numerical optimization of detailed multibody dynamics and optimal control corresponding to fast steps of the robot with an envisioned target speed of more than 0.5 m/s. In this paper the design considerations based on numerical optimal control studies and the mechanical realization of the robot are presented including first investigations on the achievable performance of a decentralized, microcontroller-based control architecture.

This paper presents an approach to fast object manipulation by dextrous re-grasping of multi-fingered hands. The approach is based on a real-time grasping force optimization (GFO) algorithm and a fingertip impedance control scheme. Both the controller and the GFO make use of 6D contact force sensor data at run-time. The latter keeps the contact forces as small as possible while considering friction limits at the contact points during (re)grasping. The impedance controller is used to impose the optimized contact forces onto the grasped object while simultaneously enabling active control of the fingertip positions. Experiments demonstrate the robustness of the approach and the increase in task speed during multi-fingered manipulation.

This paper presents an approach towards modeling and accurate dynamical simulation of multi-fingered dextrous manipulation. A hybrid discrete-continuous systems approach allows one to integrate time-driven mechanical characteristics and discrete-event aspects due to varying contact situations into one comprehensive modeling framework. A formal description is derived which can straightforwardly be embedded into a simulation environment like e.g. MATLAB. Dynamical simulations of a four-fingered hand manipulating an object demonstrate the high accuracy of our approach.

In this paper we present a control approach to dextrous manipulation based on a hybrid (discrete-continuous) dynamical system model. The discrete event aspect is the grasp state specifying whether fingers are in contact or not. The three components of the proposed discrete-continuous control approach to (re)grasping are a hybrid planning scheme for regrasping and discrete grasp state error compensation, an efficient grasping force optimization algorithm, and a variable structure grasp impedance controller. Experimental results are presented to validate the approach.

This paper presents the theoretical background and the implementation of a real-time interactive stereo vision system including vergence control. The vision system enables human operators to actively observe a remote real environment and allows high-fidelity visual sensing, useful in multimodal telepresence and teleaction tasks. The use of Internet protocol (IP) communication technology facilitates the application of the system in most wide area communication infrastructures. Evaluation experiments demonstrate the benefits of interactive stereo vision with respect to improved human task performance.

Human-system-interfaces for haptic display with emphasis on tactile information are research fields of growing interest. Their application to system design in multi-modal telepresence and virtual environment (VE) systems implies mechatronics engineering challenges as well as the need for consideration of human factors. A tactile actuator array for dynamic display of distributed tactile shape information is presented. Actuator design focus was directed at high pin forces and a bandwidth sufficient for most one-fingered tactile shape exploration tasks performed by dynamic interaction. The resulting overall device dimensions currently prohibit an attachment to the effector of typical kinesthetic feedback devices. Experimental results are presented for a virtual presence scenario where tactile stimulation emerges from dynamic interaction between virtual objects on a conveyer belt moving relative to a resting fingertip.

Presents a combined system for hand and finger kinesthetic feedback for the exploration and manipulation of different kinds of rigid objects in virtual environments. A three-degrees-of-freedom haptic interface, called DeKiFeD3 (Desktop Kinesthetic Feedback Device 3), with the capability of generating kinesthetic hand feedback, is combined with a commercial haptic glove to generate additional kinesthetic feedback to the fingers of a human operator. This system results in a new quality of kinesthetic perception and needs appropriate rendering algorithms, as described in this paper. The paper also presents an approach to improve the operator's immersion into a virtual scenario by adding auditory and visual feedback.

We present an algorithm which allows qualitative and quantitative analysis of the general force closure property of a grasp when fingertip contact forces are limited by unilateral friction constraints. It is applied to grasp synthesis during regrasping sequences. Contact points for moving fingers which result in structurally stable grasps are calculated. The proposed method is suitable for two and three dimensional grasps with an arbitrary number of fingers. The approach is validated by numerical examples and in dynamical simulations.

Display of information to the operator in advanced physical and virtual telepresence systems has to comprise different modalities of human perception. Especially concerning the haptic modalities, model-based generation of appropriate stimuli to the operator is an important task. For this reason future research in telepresence and virtual reality requires a mathematical model of human perception due to external stimuli produced by feedback devices in a multi-modal telepresence system. The paper presents a mathematical framework for describing the principles of human perception in the terminology of systems theory. Formal descriptions of experiments performed in a virtual environment show the viability and advantage from the presented approach.

We present an approach to compensate for discrete contact state errors in regrasping tasks with multi-fingered robotic hands. An adaptation of reference trajectories connected with a hybrid discrete continuous control architecture allows stable grasps even if moving fingers cannot contact at the desired position on a grasped object. We propose a robust control setup that allows multiple regrasping tasks of objects to be performed using this compensation strategy and contact force optimization. Experimental results with the hydraulic robotic hand of the TUM verify the proposed approach and illustrate the increase in robustness for multi-fingered manipulation.

We present a distributed PC-based multimodal (haptic, visual and acoustic) telepresence and virtual presence system. Two desktop kinesthetic devices (DeKiFeD3 and DeKiTop3) with 3 degrees-of-freedom have been developed for multi-modal telepresence. Feedback to the human modalities of the visual, auditory, kinesthetic, tactile and temperature senses is generated using appropriate actuator hardware. We present several applications in virtual presence and teleoperation in physical remote environments.

In this paper a switching control strategy for robust stabilization of the 2nd-order nonholonomic 2-DOF SCARA robot R2D1 is presented. The first joint is actuated by a direct drive motor, whereas the second joint is equipped with a brake. The unactuated second joint is controlled by non-collocated linearization and a PD-controller. A stability region is derived and robustness is achieved by exploiting the contractive character of the perturbed stability region. The proposed switching control strategy assures global and robust position control of the second joint. Experimental results confirm the efficiency of the proposed approach.

We propose an approach to model a robotic hand as a hybrid discrete-continuous dynamical system. The discrete event aspect is the grasp state specifying whether fingers are in contact or not. The hybrid approach yields a comprehensive model of a closed-hoop controller for dextrous (re)grasping. Insights in the closed-loop behavior and compensation strategies for discrete state errors due to object geometry modeling errors are presented. Simulations validate the hybrid systems approach to regrasping.

In dextrous robotic hand grasping the external object force needs to be balanced with contact forces maintaining grasp stability. Redundancy in the grasp yields the optimization problem to minimize the grasp effort subject to nonlinear friction stability constraints. We reformulate the optimization problem as a semidefinite program with affine constraints. In this paper, two versions of strictly convex cost functions including a barrier term, one of them self-concordant, are considered. For the general class of such cost functions Dikin-type algorithms are considered. It is shown that the proposed algorithms guarantee convergence to the unique solution of the underlying semidefinite program. Numerical examples demonstrate the simplicity of implementation and the good numerical properties of the approach.

Buss et al. previously (1995, 1996) presented a real-time applicable grasping force optimization scheme for stable dextrous grasping. The authors present a novel approach to planning multifingered regrasping based on a modification of the optimization algorithm. This novel algorithm allows for smooth grasping force transitions during regrasping tasks. The optimization scheme is implemented combined with an impedance control law. Simulation results show the efficiency and simplicity of our approach to regrasping.

A common approach to control multi-fingered grippers during stable grasping is the stiffness control scheme. "Internal" stiffness parameters and suitable references for internal grasping forces-often determined heuristically-ensure a stable grasp. In this paper, optimal internal forces are obtained in real-time by linearly constrained gradient flows on the smooth manifold of positive definite matrices. This optimization approach is a generic one for any number of fingers in contact with the object. Experiments with the 3-fingered Darmstadt hand show the simplicity and efficiency of the approach.

Intelligent human behaviour models can serve as a basis for semi-autonomous mobile robot systems. In this paper the necessity of hierarchical supervisory control for service task solution using a human-robot-interface (MRI) is motivated. The demands on information exchange between operator, MRI and service robot are outlined and categorised. A distributed planner to control the robot system is presented, enabling both flexible robot behaviour and online operator support. Effectiveness of the approach is demonstrated by experimental results achieved with the proposed MRI and the service robot ROMAN.

We propose a novel approach to reference behavior specification for multiple robotic mechanisms using hybrid system models. Hybrid automata can specify discrete states (operational modes) and continuous variable reference values in one unified framework. An example mechanism with 3 degrees-of-freedom and a 6-legged walking machine with a combination of several hybrid automata for reference generation and synchronization are discussed. Simulation results show that a hybrid system model is an effective method for robotic behavior specification. Using a model verification tool we show that behavior correctness verification and parametric analysis are possible.

A key goal in dextrous robotic hand grasping is to balance external forces and at the same time achieve grasp stability and minimum grasping energy by choosing an appropriate set of internal grasping forces. Since it appears that there is no direct algebraic optimization approach, a recursive optimization, which is adaptive for application in a dynamic environment, is required. One key observation in this paper is that friction force limit constraints and force balancing constraints are equivalent to the positive definiteness of a certain matrix subject to linear constraints. Based on this observation, we formulate the task of grasping force optimization as an optimization problem on the smooth manifold of linearly constrained positive definite matrices for which there are known globally exponentially convergent solutions via gradient flows. There are a number of versions depending on the Riemannian metric chosen, each with its advantages, Schemes involving second derivative information for quadratic convergence are also studied. Several forms of constrained gradient flows are developed for point contact and soft-finger contact friction models. The physical meaning of the cost index used for the gradient flows is discussed in the context of grasping force optimization. A discretized version for real-time applicability is presented. Numerical examples demonstrate the simplicity, the good numerical properties, and optimality of the approach.

In this paper we describe the 4-fingered dextrous robotic hand with 24 degrees-of-freedom (DOF) we have developed. At each finger-tip a 6 DOF force/torque sensor is attached to measure contact forces directly. A heterogeneous parallel computing architecture based on transputers is used to implement the proposed grasp controller and the underlying hybrid position/force controllers for each finger. Experimental results of object trajectory following, object stiffness and friction coefficient estimation are shown.

For dextrous robotic hand grasping a key goal is to balance external and internal grasping forces to assure a stable grasp and low grasping energy. In this paper, the authors view the task of grasping force optimization as a linearly constrained semidefinite programming problem for which there are known globally exponentially convergent solutions via gradient flows. A key observation is that the nonlinear friction constraint at each contact point is equivalent to the positive definiteness of a suitable matrix containing the contact wrench intensities. A recursive version is presented, and numerical examples demonstrate the simplicity, the good numerical properties, and optimality of the approach.

This paper proposes the idea of a manipulation skill mapping for dexterous manipulation task modeling. Mapping between desired object task trajectories and finger contact point locations, this skill mapping can efficiently describe complex manipulation tasks, where contact states of the fingers change during the task. Further issues on how this skill mapping can be acquired and transferred for execution by dexterous robotic hands, are discussed. Examples show the applicability of the proposed scheme.>

Intelligent cooperative manipulation is proposed as a new paradigm for human-machine interaction and cooperation. Aiming at intelligent assistance of human operators the information and power flow between operator-system-task environment is carefully analyzed using a virtual reality to simulate the task environment. Central issue of this virtual reality is the dynamic force simulator (DFS) for feedback force calculation proposed in this paper. The DFS simulates object dynamics, contact model and friction characteristics of the human hand interacting with objects in the virtual reality. An example shows the efficiency of the proposed realization scheme of the DFS.>

This paper proposes a manipulation skill mapping for dextrous manipulation task modeling in the intelligent cooperative manipulation system-ICMS. Mapping between desired object task trajectories and finger contact point locations, this skill mapping can efficiently describe complex manipulation tasks, where contact states of the fingers change during the task. Further issues discussed are how this skill mapping can be acquired into the manipulation skill database of the ICMS and transferred for execution by dextrous robotic hands. Examples show the applicability of the proposed scheme.>

This paper discusses information and power flow in the Intelligent Assisting System (IAS). The IAS includes an intelligent manipulation assistant to a human operator, where information as well as forces between the operator, the IAS and the task environment are exchanged. A manipulation skill database enables the IAS to perform complex manipulations on the motion control level and intelligently assist the operator. As a first approach to the IAS the authors have been developing a skill acquisition and transfer system using a sensor glove for force feedback to the human operator. The dynamic behavior of the grip transformation matrix is regarded as the essence of the performed manipulation skill shown with a simple manipulation example. A detailed analysis of force transformations between hand and object space yields a method for acquisition of the grip transform dynamic behavior accumulated in the skill database. Further this paper defines a control algorithm realizing object task trajectories and its feasibility is shown by simulation.

A dynamically reconfigurable robotic system (DRRS) is one that can reconfigure itself to an optimal structure, depending on the purpose and environment. To realize this concept, the authors propose CEBOT (cell structured robot), which is a distributed robotic system consisting of separable autonomous units. These functional cells are able to communicate with each other and to approach, connect, and separate automatically. If single cells of CEBOT are damaged, they can be repaired or replaced automatically. Since CEBOT is capable of adapting itself to changing environments, it is a very flexible system applicable in space, factory, and hostile environments. The authors propose an optimal structure decision method that can determine cell type, arrangement, degree of freedom, and link length. It is applicable to fixed-base and mobile-base manipulators. A structure evaluation function, which is the sum of parameters relating to the number of work points, required positioning accuracy, torque, distance between given work points, and cell cost, is presented. Simulation results are given.>