Few-shot object detection is especially interesting for applications with mobile robots and becomes even more challenging when task-related classes are very similar. This work focuses on such a scenario: detecting different types of household and industrial tools. Such tools can be rare and specific and are usually not covered by existing large datasets, except for common ones such as screwdrivers. Additionally, the target classes might change frequently depending on the robot’s missions. Therefore, we propose DE-fine-ViT, a fine-grained few-shot object detection model that does not require fine-tuning. We build our architecture on top of the elaborate DE-ViT model, extending it with specialized components to improve the fine-grained detection capabilities. The user can construct class and part prototypes tailored to the task in an interactive preparation phase. During inference, our proposed reevaluation module leverages the multi-granularity of prototypes for fine-grained class differentiation. We evaluate our model in multiple realistic experiments, including a specifically created fine-grained dataset, demonstrating its efficacy and suitability for scenarios with little data and low inter-class variance.

Mobile robots can perform increasingly impressive feats in controlled environments. Many real applications, though, especially for walking robots, introduce a high degree of unforeseen difficulties, yet require very robust robot operation. In these cases, it is still often not possible to guarantee the needed reliability.We present an approach to utilize unsupervised anomaly detection to implement a fear-based adaptation of robot behavior. This allows robots to automatically and quickly react to any type of unexpected problems. Neither the environment nor the type of disturbance has to be known beforehand, as the system requires only a small amount of baseline data for training, which can be collected in a laboratory environment. Additionally, it can work on arbitrary robot hardware and be integrated in all types of robot control structures.We evaluated our approach in simulation and on state of the art walking robots, ANYmal, Spot and our own six-legged walking robot prototype, in a realistic field test environment in the Tabernas desert in Spain. Our results showcase that we can quickly detect arbitrary problems based on significantly different types of sensor data and decrease robot fall rates in the most extreme scenarios from 56% to 4%. This promises significant increases in robustness for all types of walking robots in highly challenging and previously unknown environments.

Enhancing production efficiency, ensuring consistent quality, and significantly reducing manufacturing costs are core benefits of deploying robotic solutions to manufacturing tasks. Leading to automation of assembly tasks being a focus of robotics research since many years, and many solutions have already been successfully deployed into the industry. However, applying those approaches in a human-robot collaboration (HRC) scenario is challenging. Humans introduce an additional uncertainty factor that prohibits pure offline planning and requires plan adaptation during execution. Intelligent planning approaches are therefore needed that consider human intervention from the start and enable further progress toward flexible HRC assembly systems. This paper presents a novel approach to flexible robotic task planning by utilizing assembly affordances, which are perceived opportunities for actions that a component offers in terms of assembly. Our approach integrates affordance-based reasoning within a semantic planning framework to allow online refinement of assembly plans to be used in fully automated scenarios as well as HRC assembly scenarios. The paper presents the developed concept and framework and evaluates it based on the success rate on assemblies of different complexities.

In real-world few-shot image classification tasks the lack of abundant data makes training and testing very challenging. The classification model must learn the most meaningful features using only a few sample images without context knowledge. Here, interpretability methods for deep models are helpful for increased comprehensibility and verification. However, these advantages are limited without the ability to correct the model directly. Therefore, we propose an interpretable approach for few-shot object recognition that includes optional interactive teaching to close the feedback loop. We leverage pretrained vision transformers as backbones and a part-based inference particularly favors interpretability. We use a visual concept bank to translate semantic visual features between the human and the model. Even without any human interaction, our model performs competitively compared to state-of-the-art methods in few-shot image classification tasks. Beyond that, we demonstrate the benefits of our interactive interfaces. We show how they can significantly improve the robustness in fine-grained recognition tasks and help to quickly adapt the model without complex fine-tuning.

While individual robots are becoming increasingly capable, the complexity of expected missions increases exponentially in comparison. To cope with this complexity, heterogeneous teams of robots have become a significant research interest in recent years. Making effective use of the robots and their unique skills in a team is challenging. Dynamic runtime conditions often make static task allocations infeasible, requiring a dynamic, capability-aware allocation of tasks to team members. To this end, we propose and implement a system that allows a user to specify missions using Behavior Trees (BTs), which can then, at runtime, be dynamically allocated to the current robot team. The system allows to statically model an individual robot’s capabilities within our ros_bt_py BT framework. It offers a runtime auction system to dynamically allocate tasks to the most capable robot in the current team. The system leverages utility values and pre-conditions to ensure that the allocation improves the overall mission execution quality while preventing faulty assignments. To evaluate the system, we simulated a find-and-decontaminate mission with a team of three heterogeneous robots and analyzed the utilization and overall mission times as metrics. Our results show that our system can improve the overall effectiveness of a team while allowing for intuitive mission specification and flexibility in the team composition.

Deciding where to go next is a challenging task for humans. However, for robots in unknown environments, this becomes even more demanding. In planetary explorations, the robots are continuously challenged with the task of exploring novel areas, yet so far, humans decide for the robots where to go. Even then, prioritizing the next target based on previous knowledge is complex. In our proposed work, the robot utilizes data about its surroundings from drone or satellite images. Alternatively, a volumetric representation can be reduced to form a suitable input. From the input, tiles are selected and embedded by different autoencoder variants. The robot can select the most promising next exploration goal through the distance in the embedding to the previous samples. In this work, a variational autoencoder, a Wasserstein autoencoder, and a spherical autoencoder are evaluated against each other. The latter two variants yield a high information gain when evaluated on satellite data from the Netherlands. Additionally, the framework was employed on data from an analog mission in the Tabernas desert. Through the framework, the robots get an understanding of which goals yield the most information gain and, therefore, can quickly improve their knowledge about their surroundings.

As intelligent systems become increasingly important in our daily lives, new ways of interaction are needed. Classical user interfaces pose issues for the physically impaired and are partially not practical or convenient. Gesture recognition is an alternative, but often not reactive enough when conventional cameras are used. This work proposes a Spiking Convolutional Neural Network, processing event- and depth data for gesture recognition. The network is simulated using the open-source neuromorphic computing framework LAVA for offline training and evaluation on an embedded system. For the evaluation three open source data sets are used. Since these do not represent the applied bi-modality, a new data set with synchronized event- and depth data was recorded. The results show the viability of temporal encoding on depth information and modality fusion, even on differently encoded data, to be beneficial to network performance and generalization capabilities.

The use of mobile robots has become increasingly common in multiple areas of daily life. To increase their autonomy for performing various tasks, efficient navigation skills are essential. The most crucial component of such navigation is the ability to calculate a global path between two points. The global path planning problem for mobile robots is typically limited to two-dimensional environments, in which the environment is projected onto a planar surface. While this approach works well in structured environments like industrial settings, it may not be suitable for all applications of mobile robots. With modern walking robots, capable of navigating complex terrain, more advanced path planning approaches are necessary. This work proposes a path-planning approach that utilizes the entire three-dimensional space, allowing for navigation in even the most challenging terrain. The central idea is to extend a traditional A* path planner to work directly on a fast volumetric map structure to generate optimal paths through the environment. Multiple optimizations and adjustments are introduced to improve the algorithm’s performance. By applying morphology operators to sparse maps, sensor inaccuracies during the map construction are mitigated. Additionally, adjustments are made to handle the added complexity introduced by the extra search space dimension and to comply with the limitations of autonomous walking robots. This is paired with an efficient caching strategy to enhance the overall path-planning speed. The capability of the path planning approach is evaluated using both artificial and real-world maps. The results demonstrate that this approach shows great potential for enabling mobile ground robots to autonomously navigate even the most demanding terrains utilizing the entire three-dimensional space.

Autonomous Mobile Robots are used with increasing frequency in inspection and maintenance tasks completing fixed goal sequences. The downtime robots experience between goals offers an opportunity to gather additional environment information instead of resting. Uncertainty in the amount of downtime available rules out the definition of a pre-determined schedule set by an external operator. Instead, the robot itself should decide dynamically, what information it should gather before its next task begins. This results in a multi-objective optimization problem trying to maximize information gain while utilizing as much of the available time as possible. We propose a genetic algorithm to solve the presented optimization problem and introduce two different models for artificial curiosity used inside the fitness function for gathering as much information as possible. For planning the genetic algorithm utilizes a multi-map approach using information and obstacle maps. We evaluated our models in a pre-defined and pre-mapped Gazebo environment with a given information map and evaluated their performance against an information-agnostic coverage algorithm. In this work, we show that utilizing artificial curiosity in path planning can result in major information gains by effectively using downtime.

Several successful approaches exist for solving the complex problem of multi-robot planning and coordination. Due to the lack of adequate benchmarking tools, comparing these approaches and judging their suitability for use in realistic scenarios is currently difficult. Therefore, we propose an open-source benchmark suite that aims to close this gap. Unlike existing benchmarks, our approach uses full-stack multi-robot navigation systems in realistic 3D simulated environments from the intralogistic and household domains. Using the open-source frameworks ROS 2, Gazebo and RMF allows the user to add other robot platforms easily. The framework provides easy-to-use abstractions, typical metrics and interfaces to several established planning libraries for multi-robot systems. With all these features, our framework successfully aids practitioners and researchers in comparing multi-robot planning and coordination systems to the state of the art. Our experiments show how the proposed benchmark simplifies gaining insights on relevant close to real-life robotics use cases.

Motion planning for mobile robot platforms is one of the long-established research fields in robotics. In this paper, we propose a trajectory planner for mobile holonomic robots to steer non-holonomic conventional passive wheelchairs in dynamic environments. The challenges to overcome when steering a wheelchair are to find smooth feasible trajectories, maintain a fast reactive response to dynamic obstacles and to satisfy a set of additional constraints such as limiting physical forces acting on the wheelchair occupants. Our approach is a variant of the timed-elastic-bands (TEB) planner, which includes a footprint of the wheelchair during optimization, and generates a steering angle which is then consumed by an arm controller to actuate the relative orientation between the wheelchair and the mobile platform. This is realized by posing new non-holonomic and kinodynamic constraints on the TEB planner and an implementation of a suitable real-time dual-arm controller for executing steering commands. We demonstrate our results based on a TEB baseline comparison in simulation using functional models of our robot HoLLiE and a wheelchair.

Exploring unknown environments, such as caves or planetary surfaces, requires a quick understanding of the surroundings. Beforehand, only aerial footage from satellites or images from previous missions might be available. The proposed ensemble based anomaly detection framework utilizes previously gained knowledge and incorporates it with insights gained during the mission. The modular system consists of different networks which are combined to determine anomalies in the current surroundings. By utilizing data from other missions, simulations or aerial photos, a precise anomaly detection can be achieved at the start of a mission. The system can further be improved by training new networks during the mission, which can be incorporated into the ensemble at runtime. This allows for synchronous execution of mission and training of models on a base station. The proposed system is tested and evaluated on an ANYmal C walking robot in different scenarios, however the approach is applicable for different kinds of mobile robots. The results show a clear improvement of ensembles compared to individual networks, while keeping a small memory footprint and low inference time on the mobile system.

Complex robots in challenging scenarios require constant monitoring of their state and adaptation of their behavior to ensure robustness, reliability and longevity. While known possible errors can be specifically surveilled, other prob-lems can be fully unforeseen, requiring detection systems able to identify novel faults. We detect possible faults as anomalies on various internal sensor data, utilizing unsupervised learning techniques. A bidirectional Wasserstein GAN approach for anomaly detection on multivariate, highly dependent time-series data is implemented and trained on a small amount of non-anomalous robot sensor data. This model is then used for inference on the on-board hardware of a robot without parallel processing units. We evaluate multiple variants of the architecture using manually introduced anomalies in the form of different weights attached to the robot's legs. Overall we are able to show that RoBiGAN is able to consistently detect and localize small anomalies in an online scenario, with little to no robot specific modeling needed.

We present a biologically inspired approach for path planning with dynamic obstacle avoidance. Path plan-ning is performed in a condensed configuration space of a robot generated by self-organizing neural networks (SONN). The robot itself and static as well as dynamic obstacles are mapped from the Cartesian task to the configuration space by precomputed kinematics. The condensed space represents a cognitive map of the environment, which is inspired by place cells and the concept of cognitive maps in mammalian brains. Generation of training data as well as the evaluation are performed on a real industrial robot accompanied by simulations. To evaluate reactive collision-free online planning within a changing environment, a demonstrator was realized. Then, a comparative study regarding sample-based planners was carried out. The robot is able to operate in dynamically changing environments and re-plan its motion trajectories within impressing 0.02 seconds, which proofs the real-time capability of our concept.

Speech is a convenient hands-free communication channel where humans are already experienced users. It can implicitly create trustfulness between two operators and lead to a comfortable and natural collaborative environment. As stated in existing literature, speech interaction could increase efficiency and improve certain aspects of Human-Robot Collaboration (HRC). Anyway, speech recognition in industrial scenarios presents different challenges: the typical noisy environment can affect dramatically the interaction performance, leading to an unacceptable inaccuracy in understanding the uttered intention.In this work, we propose and evaluate a modular system for robust and natural speech interaction in challenging acoustical environments. The system has been integrated and tested in a realistic HRC scenario in which the acoustic interaction and efficiency have been evaluated. The developed framework focuses on decreasing the requirements in terms of signal-to-noise ratio, providing a methodology to evaluate the naturalness of the interaction and improvements in efficiency.The solution is designed with a modular approach, providing an easy configuration for ROS-based systems. In this way, it allows a simple integration in existing applications and future research projects, where a dual speech-based interaction can increase the overall performance of the HRC.

Field and service robots that do not understand the hazards in their environment limit their potential by acting overly careful or navigating into potentially dangerous areas. We present an extended modular mapping framework which allows to model different types of hazards from a multitude of inputs. The proposed approach is generalized for storage of arbitrary data, therefore not limiting the usage to one use case. Furthermore the framework allows the fusion of risks to calculate the overall risk for an individual robot from its surroundings. The system was tested with LAURON V, a hexapod designed for walking over rough and hazardous terrain.

Robotic automation is a key driver for the advancement of technology. The skills of human workers, however, are difficult to program and seem currently unmatched by technical systems. In this work we present a data-driven approach to extract and learn robot-independent contact skills from human demonstrations in simulation environments, using a Long Short Term Memory (LSTM) network. Our model learns to generate error-correcting sequences of forces and torques in task space from object-relative motion, which industrial robots carry out through a Cartesian force control scheme on the real setup. This scheme uses forward dynamics computation of a virtually conditioned twin of the manipulator to solve the inverse kinematics problem. We evaluate our methods with an assembly experiment, in which our algorithm handles part tilting and jamming in order to succeed. The results show that the skill is robust towards localization uncertainty in task space and across different joint configurations of the robot. With our approach, non-experts can easily program force-sensitive assembly tasks in a robot-independent way.

Bio-inspired robots take advantage of millions of years of evolution to provide interesting and flexible solutions for issues related to motion and perception. Often, they have challenging kinematics classical robotics control mechanisms are not always able to take advantage of them. A concrete example of this is LAURON V, a six-legged robot for space exploration inspired by the stick insects. The main goals of this work is to combine classical behaviour-based control with motor primitives implemented with SNN for motion representation. We extend a previously presented bio-inspired approach to represent hand and arm motion using motor primitives, and combine it with a behaviour-based architecture to model different locomotion behaviours for a multi-legged robot. There are four main components. First, to model the individual leg motions we use two motor primitives implemented with spiking neural networks for the swing and stance phases. Second, to control the motor primitives of each leg there are local behaviours corresponding to each phase, and corresponding to each activation pattern. Third, the activation patterns are used to facilitate multi-leg coordination and generate different walking behaviours. Fourth, a high-level control interface integrates control signals from other sources and activates the patterns. We conducted five different experiments to evaluate our approach in a simulated environment using the Neurorobotics Platform (NRP). The results show that our modelling approach with motor primitives is flexible enough to represent different types of motions, and also highlight the value of the NRP for robotics development.

Most robots consistently repeat their motion with- out changes in a precise and consistent manner. But nowadays there are also robots able to dynamically change their motion and plan according to the people and environment that surround them. Furthermore, they are able to interact with humans and cooperate with them. With no information about the robot targets and intentions, the user feels uncomfortable even with a safe robot. In close human-robot collaboration, it is very important to make the user able to understand the robot intentions in a quick and intuitive way. In this work we have developed a system to use augmented reality to project directly into the workspace useful information. The robot intuitively shows its planned motion and task state. The AR module interacts with a vision system in order to display the changes in the workspace in a dynamic way. The representation of information about possible collisions and changes of plan allows the human to have a more comfortable and efficient interaction with the robot. The system is evaluated in different setups.

Nowadays robots are able to work safely close to humans. They are light-weight, intrinsically safe and capable of avoiding obstacles as well as understand and predict human motions. In this collaborative scenario, the communication between humans and robots is a fundamental aspect to achieve good efficiency and ergonomics in the task execution. A lot of research has been made related to robot understanding and prediction of the human behavior, allowing the robot to replan its motion trajectories. This work is focused on the communication of the robot's intentions to the human to make its goals and planned trajectories easily understandable. Visual and acoustic information has been added to give the human an intuitive feedback to immediately understand the robot's plan. This allows a better interaction and makes the humans feel more comfortable, without any feeling of anxiety related to the unpredictability of the robot motion. Experiments have been conducted in a collaborative assembly scenario. The results of these tests were collected in questionnaires, in which the humans reported the differences and improvements they experienced using the feedback communication system.

Compliant end effectors in robotics are an important prerequisite for the field of object manipulation and environment interactions. However, current manipulators are usually stiff position-controlled systems, generating the need to add compliance. In this work, we present Forward Dynamics Compliance Control (FDCC), a new threefold control concept that realizes Cartesian compliance through combining Admittance, Impedance and Force Control into one control strategy. We close the control loop only through a force-torque sensor, allowing a system independent and decoupled configuration of the end effector compliance. As a key component in FDCC, we leverage forward dynamics simulations of a virtual model to directly map Cartesian inputs to joint control commands, leading to excellent stability in singularities. Experiments on three different robotic manipulators verify the key advantages of this approach.

We present an approach for recognizing indoor scenes in object constellations that require object search by a mobile robot, as they cannot be captured from a single viewpoint. In our approach that we call Active Scene Recognition (ASR), robots predict object poses from learnt spatial relations that they combine with their estimates about present scenes. Our models for estimating scenes and predicting poses are Implicit Shape Model (ISM) trees from prior work [1]. ISMs model scenes as sets of objects with spatial relations in-between and are learnt from observations. In prior work [2], we presented a realization of ASR, limited to choosing orientations for a fixed robot head with an approach to search objects that uses positions and ignores types. In this paper, we introduce an integrated system that extends ASR to selecting positions and orientations of camera views for a mobile robot with a pivoting head. We contribute an approach for Next-Best-View estimation in object search on predicted object poses. It is defined on 6 DoF viewing frustums and optimizes the searched view, together with the objects to be searched in it, based on 6 DoF pose predictions. To prevent combinatorial explosion when searching camera pose space, we introduce a hierarchical approach to sample robot positions with increasing resolution.

We present an approach that uses combinatorial optimization to decide which spatial relations between objects are relevant to accurately describe an indoor scene, made up of objects. We extract scene models from object configurations that are acquired during demonstration of actions, characteristic for a certain scene. We model scenes as graphs with Implicit Shape Models (ISMs), a Generalized Hough Transform variant. ISMs are limited to represent scenes as star-shaped topologies of object relations, leading to false positives in recognizing scenes. To describe other relation topologies, we introduced a representation of trees of ISMs in prior work together with a method to learn such ISM trees from demonstrations. Limited to creating topologies, corresponding to spanning trees, that method omits certain relations so that false positives still occur. In this paper, we introduce a method to convert any relation topology, corresponding to a connected graph, into an ISM tree using a heuristic depth-first-search. It allows using complete graphs as scene models. Despite causing no false positives, complete graphs are intractable for scene recognition. To achieve efficiency, we contribute a method that searches for an optimal relation topology by traversing the space of connected scene graphs, for a given set of objects, using an optimization similar to hill climbing. Optimality is defined as minimizing computational costs during scene recognition, while producing a minimum of false positives. Experiments with up to 15 objects show that both are achievable by the presented method. Costs, growing exponentially with the number of objects, are transferred from online recognition to offline optimization.

New sensor systems, efficient planning algorithms and increased computational power have led to a growing interest in high-resolution 3-D data for challenging applications like mobile manipulation, 3-D mapping and object recognition. 3-D laser scanners built using a rotating or swivelling 2-D laser scanner are a widespread technology for obtaining 3-D point clouds, but convenient calibration methods are needed to increase the sensor precision and loosen the requirements on mechanical tolerances. We present an approach for the automatic self-calibration of such scanners, using only a single scan of a targetless environment recorded over a complete 360° rotation of the external motor that controls the motion of the 2-D sensor. By exploiting the intrinsic redundancies of the recorded point clouds and decimating the clouds using a voxel grid filter, we are able to compute and optimize a point cloud quality measure very quickly and without relying on explicit calibration targets. Our results on simulated and real data show the effectiveness of our approach by creating high-quality 3-D point clouds.

Overcoming the perceptual limitations of humanoid robots requires representations exploitable by highly integrable simulation, sensing, planning and acting components. Therefore, a novel active visual localization component for humanoid robots based on particle filtering in CAD environments is introduced. Specifically, two new components are presented: i) A vector-graphics prediction method employing hierarchical CAD environmental representations is presented. ii) A gaze attention method within the prediction-update cycle of the particle filter increases the available amount of visual features for localization while allowing adjustable task coupling. Finally, large and unobstructive ground-truth validation with the humanoid robot ARMAR-IIIb [1] in a made-for-humans environment shows the robustness, accuracy and performance of the proposed methods.

We present an approach that combines passive scene understanding with object search in order to recognize scenes in indoor environments that cannot be perceived from a single point of view. Passive scene recognition is performed using Implicit Shape Models based on spatial relations between objects. ISMs, a variant of the Generalized Hough Transform, are extended to describe scenes as sets of objects with relations lying between them. Relations are expressed as six-degree-of-freedom (DoF) relative object poses. They are extracted from sensor recordings of human demonstrations of actions usually taking place in the corresponding scene. In a scene ISMs solely represent relations of n objects towards a common reference. Violations of other relations are not detectable. To overcome this limitation, we extend our scene model, using hierarchical agglomerative clustering, to a binary tree consisting of ISMs. Active scene recognition aims to simultaneously detect present scenes and look for objects these scenes consist of. For a pivoting stereo camera rig, we achieve this by performing recognition with ISMs in an object search loop using next-best-view (NBV) estimates. A criterion, on which we greedily choose views the rig shall adopt next, is the confidence to detect objects in them. In each step during the search, confidences on potential positions of objects, not found yet, are calculated based on the best available scene hypothesis. This is done by reversing the principle of ISMs and using spatial relations to predict potential object positions starting from the objects already detected.

We present a framework for quadrupedal locomotion over highly challenging terrain where the choice of appropriate footholds is crucial for the success of the behaviour. We use a path planning approach which shares many similarities with the results of the DARPA Learning Locomotion challenge and extend it to allow more flexibility and increased robustness. During execution we incorporate an on-line force-based foothold adaptation mechanism that updates the planned motion according to the perceived state of the environment. This way we exploit the active compliance of our system to smoothly interact with the environment, even when this is inaccurately perceived or dynamically changing, and update the planned path on-the-fly. In tandem we use a virtual model controller that provides the feed-forward torques that allow increased accuracy together with highly compliant behaviour on an otherwise naturally very stiff robotic system. We leverage the full set of benefits that a high performance torque controlled quadruped robot can provide and demonstrate the flexibility and robustness of our approach on a set of experimental trials of increasing difficulty.

This paper gives an overview on our framework for efficient collision detection in robotic applications. It unifies different data structures and algorithms that are optimized for Graphics Processing Unit (GPU) architectures. A speed-up in various planning scenarios is achieved by utilizing storage structures that meet specific demands of typical use-cases like mobile platform planning or full body planning. The system is also able to monitor the execution of motion trajectories for intruding dynamic obstacles and triggers a replanning or stops the execution. The presented collision detection is deployed in local dynamic planning with live pointcloud data as well as in global a-priori planning. Three different mobile manipulation scenarios are used to evaluate the performance of our approach.

Multi-legged walking robots often make use of sophisticated control architectures to play their strengths in rough and unknown environments. The adaptability of these robots is an essential skill to achieve the maneuverability and autonomy needed in their application fields. In this work we present a reactive control approach for the hexapod LAURONV, which enables it to overcome large obstacles and steep slopes without any knowledge about the environment. A key to this success can also be seen in the increased kinematic adaptability due to the fourth rotational joint in the bio-inspired leg kinematics. An extended experimental evaluation shows that the reactive posture behaviors are able to create an effective and efficient locomotion in challenging environments.

As intelligent vehicles become more and more capable, they must learn to navigate and localize themselves in a wide variety of environments, including GPS-denied and only crudely mapped areas. We argue that since autonomous vehicles must be able to perceive, and semantically interpret, their immediate environment, they should be able to use abstract semantic information as their sole means of localization. This simplifies the level of detail and precision required from environment maps so that, for example, a rough floor plan of a parking garage will suffice to autonomously navigate it. We propose a concept for semantic localization which only requires a conceptual semantic map of the environment, and can be made to work with any kind of sensor data from which the required semantic information can be extracted. We present a localization algorithm which may be used as a base for semantic navigation, e.g. in context of automated driving, and some initial results of its application in a parking garage scenario.

Discrete POMDPs of medium complexity can be approximately solved in reasonable time. However, most applications have a continuous and thus uncountably infinite state space. We propose the novel concept of learning a discrete representation of the continuous state space to solve the integrals in continuous POMDPs efficiently and generalize sparse calculations over the continuous space. The representation is iteratively refined as part of a novel Value Iteration step and does not depend on prior knowledge. Consistency for the learned generalization is asserted by a self-correction algorithm. The presented concept is implemented for continuous state and observation spaces based on Monte Carlo approximation to allow for arbitrary POMDP models. In an experimental comparison it yields higher values in significantly shorter time than state of the art algorithms and solves higher-dimensional problems.

Having a representation of the capabilities of a robot is helpful when online queries, such as solving the inverse kinematics (IK) problem for grasping tasks, must be processed efficiently in the real world. When workspace representations, e.g. the reachability of an arm, are considered, additional quality information such as manipulability or self-distance can be employed to enrich the spatial data. In this work we present an approach of inverting such precomputed reachability representations in order to generate suitable robot base positions for grasping. Compared to existing works, our approach is able to generate a distribution in SE(2), the cross-space consisting of 2D position and 1D orientation, that describes potential robot base poses together with a quality index. We show how this distribution can be queried quickly in order to find oriented base poses from which a target grasping pose is reachable without collisions. The approach is evaluated in simulation using the humanoid robot ARMAR-III [1] and an extension is presented that allows to find suitable base poses for trajectory execution.

A major strength of humanoid robotics platforms consists in their potential to perform a wide range of manipulation tasks in human-centered environments thanks to their anthropomorphic design. Further, they offer active head-eye systems which allow to extend the observable workspace by employing active gaze control. In this work, we address the question where to look during manipulation tasks while exploiting these two key capabilities of humanoid robots. We present a solution to the gaze selection problem, which takes into account constraints derived from manipulation tasks. Thereby, three different subproblems are addressed: the representation of the acquired visual input, the calculation of saliency based on this representation, and the selection of the most suitable gaze direction. As representation of the visual input, a probabilistic environmental model is discussed, which allows to take into account the dynamic nature of manipulation tasks. At the core of the gaze selection mechanism, a novel saliency measure is proposed that includes accuracy requirements from the manipulation task in the saliency calculation. Finally, an iterative procedure based on spherical graphs is developed in order to decide for the best gaze direction. The feasibility of the approach is experimentally evaluated in the context of bimanual manipulation tasks on the humanoid robot ARMAR-III.

In order to realize performance gain of a robot or an artificial arm, the end-effector which exhibits the same function as human beings and can respond to various objects and environment needs to be realized. Then, we developed the new hand which paid its attention to the structure of human being's hand which realize operation in human-like manipulation (called TUAT/Karlsruhe Humanoid Hand). Since this humanoid hand has the structure of adjusting grasp shape and grasp force automatically, it does not need a touch sensor and feedback control. It is designed for the humanoid robot which has to work autonomously or interactively in cooperation with humans and for an artificial arm for handicapped persons. The ideal end-effectors for such an artificial arm or a humanoid would be able to use the tools and objects that a person uses when working in the same environment. If this humanoid hand can operate the same tools, a machine and furniture, it may be possible to work under the same environment as human beings. As a result of adopting a new function of a palm and the thumb, the robot hand could do the operation which was impossible until now. The humanoid hand realized operations which hold a kitchen knife, grasping a fan, a stick, uses the scissors and uses chopsticks.

Humanoid robots should be able to visually recognize objects and estimate their 6D pose in real environmental conditions with their limited sensor capabilities. In order to achieve these visual skills, it is necessary to establish an optimal visual transducer connecting the scene layout with the internal representations of objects and places. This visual transducer should capture the noiseless visual manifold of the scene with high-dynamic-range in an efficient manner. Our endeavor is to develop such a visual transducer using the widespread LDR cameras in humanoid robots. In our previous work, the noiseless acquisition of continuous images [1] and the improved radio-metric calibration [2] already enabled the humanoid robots to attain the desired visual manifold in terms of quality. However, since the radiance range of the scene can be very wide, the required amount of exposures to capture the visual manifold (robustly without radiance inconsistencies) turns impractically large in terms of scope, granularity and acquisition time. In this article, a method for estimating the minimal amount of exposures and their particular integration times is presented. This method integrates our previous work in order to synthesize HDR images with the minimal amount of exposures while ensuring the high quality of the resulting image. Conclusively, the minimal exposure set provides performance improvements without quality trade-off. Experimental evaluation is presented with the humanoid robots ARMAR-III a, b [3].

This paper describes a collaborative human-robot system that provides context information to enable more effective robotic manipulation. We take advantage of the semantic knowledge of a human co-worker who provides additional context information and interacts with the robot through a user interface. A Bayesian Network encodes the dependencies between this information provided by the user. The output of this model generates a ranked list of grasp poses best suitable for a given task which is then passed to the motion planner. Our system was implemented in ROS and tested on a PR2 robot. We compared the system to state-of-the-art implementations using quantitative (e.g. success rate, execution times) as well as qualitative (e.g. user convenience, cognitive load) metrics. We conducted a user study in which eight subjects were asked to perform a generic manipulation task, for instance to pour a bottle or move a cereal box, with a set of state-of-the-art shared autonomy interfaces. Our results indicate that an interface which is aware of the context provides benefits not currently provided by other state-of-the-art implementations.

Most robotic vision algorithms are computationally intensive and operate on millions of pixels of real-time video sequences. But they offer a high degree of parallelism that can be exploited through parallel computing techniques like Invasive Computing. But the conventional way of multi-processing alone (with static resource allocation) is not sufficient enough to handle a scenario like robotic maneuver, where processing elements have to be shared between various applications and the computing requirements of such applications may not be known entirely at compile-time. Such static mapping schemes leads to inefficient utilization of resources. At the same time it is difficult to dynamically control and distribute resources among different applications running on a single chip, achieving high resource utilization under high-performance constraints. Invasive Computing obtains more importance under such circumstances, where it offers resource awareness to the application programs so that they can adapt themselves to the changing conditions, at run-time. In this paper we demonstrate the resource aware and self-organizing behavior of invasive applications using three widely used applications from the area of robotic vision - Optical Flow, Object Recognition and Disparity Map Computation. The applications can dynamically acquire and release hardware resources, considering the level of parallelism available in the algorithm and time-varying load.

Planning motion for humanoid robots requires obeying simultaneous constraints on balance, collision-avoidance, and end-effector pose, among others. Several algorithms are able to generate configurations that satisfy these constraints given a good initial guess, i.e. a configuration which is already close to satisfying the constraints. However, when selecting goals for a planner a close initial guess is rarely available. Methods that attempt to satisfy all constraints through direct projection from a distant initial guess often fail due to opposing gradients for the various constraints, joint-limits, or singularities. We approach the problem of generating a constrained goal by searching for a configuration in the intersection of all constraint manifolds in configuration space (C-space). Starting with an initial guess, our algorithm, Constellation, builds a graph in C-space whose nodes are configurations that satisfy one or more constraints and whose cycles determine where the algorithm explores next. We compare the performance of our approach to direct projection and a previously-proposed cyclic projection method on reaching tasks for a humanoid robot with 33 DOF. We find that Constellation performs the best in terms of the number of solved queries across a wide range of problem difficulty. However, this success comes at higher computational cost.

To be a good helper, grasping and manipulation are the most important abilities of a service robot. It should be able to adapt its manipulation actions to new tasks and environments. During the execution, it is important to rate the success of actions, so that the robot can plan and execute further actions to correct and recover from the failed actions. The successful execution of manipulation actions depends on various factors during the whole execution, such as the position of the robotic hand and forces exerted by the robot. The goal of the manipulation action monitoring is to estimate the success state from the huge amount of data collected during the execution. The main challenge to solve this problem is to identify the success or failure state from the the high dimensional data collection. We propose a method to classify ongoing activities using a set of support vector machines (SVM). After a supervised training process with manually labeled successful or failure results, our system can correctly estimate the resulting state of a manipulation activity. We present experiments on our bimanual manipulation demonstrator and evaluate the results.

Flexible robot assistants allow reducing the costs of acquisition for the automation of small lot sizes in industrial assembly. To additionally decrease the cost for initial start-up and to reach profitability, new concepts for programming these systems are needed. This work describes an approach for a human-machine-interface that enables operators without expert knowledge to commission complex assembly applications in a short time. The user is efficiently led through the whole process by intuitive operator guidance. To reach the requirements of industrial pick-and-place tasks, a concept for moving the robot by specialized jog controls is presented that allows a fast and user-friendly teaching with high precision. Furthermore, an intuitive approach for selection and parameterization of complex object detection sequences is described. Usability tests with machine setters showed that the human-machine-interface meets industrial needs without requiring expert knowledge and the time for commissioning can be shortened to a few hours.

The previously presented Kinematic Bézier Maps (KBM) are a machine learning algorithm that has been tailored to efficiently learn the kinematics of redundant robots. This algorithm relies upon a representation based on projective geometry that uses a special set of polynomial functions borrowed from the field of Computer Aided Geometric Design (CAGD). So far, it has only been possible to learn a model of the forward kinematics function. In this paper, we show how the KBM algorithm can be modified to learn the robot's equation of motion and, hence, its inverse dynamic model. Results from experiments with a simulated serial robot manipulator are presented that clearly show the advantages of our approach compared to general function approximation methods.

The calibration of serial manipulators with high numbers of degrees of freedom by means of machine learning is a complex and time-consuming task. With the help of a simple strategy, this complexity can be drastically reduced and the speed of the learning procedure can be increased. When the robot is virtually divided into shorter kinematic chains, these subchains can be learned separately and hence much more efficiently than the complete kinematics. Such decompositions, however, require either the possibility to capture the poses of all end effectors of all subchains at the same time, or they are limited to robots that fulfill special constraints. In this paper, an alternative decomposition is presented that does not suffer from these limitations. An offline training algorithm is provided in which the composite subchains are learned sequentially with dedicated movements. A second training scheme is provided to train composite chains simultaneously and online. Both schemes can be used together with many machine learning algorithms. In the simulations, an algorithm using parameterized self-organizing maps modified for online learning and Gaussian mixture models (GMMs) were chosen to show the correctness of the approach. The experimental results show that, using a twofold decomposition, the number of samples required to reach a given precision is reduced to twice the square root of the original number.

The kinematics of a robot with many degrees of freedom is a very complex function. Learning this function for a large workspace with a good precision requires a huge number of training samples, i.e., robot movements. In this paper, we introduce the Kinematic Bézier Map (KB-Map), a parameterizable model without the generality of other systems but whose structure readily incorporates some of the geometric constraints of a kinematic function. In this way, the number of training samples required is drastically reduced. Moreover, the simplicity of the model reduces learning to solving a linear least squares problem. Systematic experiments have been carried out showing the excellent interpolation and extrapolation capabilities of KB-Maps and their relatively low sensitivity to noise.

Image-based 6-DoF pose estimation of arbitrarily shaped 3D objects based on their shape is a rarely studied problem. Most existing image-based methods for pose estimation either exploit textural information in form of local features or, if shape-based, rely on the extraction of straight line segments or other primitives. Straight-forward extensions of 2D approaches are potentially more general, but in practice assume a limited range of possible view angles. The general problem is that a 3D object can potentially produce completely different 2D projections depending on its relative pose to the observing camera. One way to reduce the solution space is to exploit temporal information, i.e. perform tracking. Again, existing model-based tracking approaches rely on relatively simple object geometries. In this paper, we propose a particle filter based tracking approach that can deal with arbitrary shapes and arbitrary or even no texture, i.e. it offers a general solution to the rigid object tracking problem. As our approach can deal with occlusions, it is in particular of interest in the context of goal-directed imitation learning involving the observation of object manipulations. Results of simulation experiments as well as real-world experiments with different object types prove the practical applicability of our approach.

In this paper, we present a grasp representation in task space exploiting position information of the fingertips. We propose a new way for grasp representation in the task space, which provides a suitable basis for grasp imitation learning. Inspired by neuroscientific findings, finger movement synergies in the task space together with fingertip positions are used to derive a parametric low-dimensional grasp representation. Taking into account correlating finger movements, we describe grasps using a system of virtual springs to connect the fingers, where different grasp types are defined by parameterizing the spring constants. Based on such continuous parameterization, all instantiation of grasp types and all hand preshapes during a grasping action (reach, preshape, enclose, open) can be represented. We present experimental results, in which the spring constants are merely estimated from fingertip motion tracking using a stereo camera setup of a humanoid robot. The results show that the generated grasps based on the proposed representation are similar to the observed grasps.

Humanoid robots should be able to grasp and handle objects in the environment, even if the objects are seen for the first time. A plausible solution to this problem is to categorize these objects into existing classes with associated actions and functional knowledge. So far, efforts on visual object categorization using humanoid robots have either been focused on appearance-based methods or have been restricted to object recognition without generalization capabilities. In this work, a shape model-based approach using stereo vision and machine learning for object categorization is introduced. The state-of-the-art features for shape matching and shape retrieval were evaluated and selectively transfered into the visual categorization. Visual sensing from different vantage points allows the reconstruction of 3D mesh models of the objects found in the scene by exploiting knowledge about the environment for model-based segmentation and registration. These reconstructed 3D mesh models were used for shape feature extraction for categorization and provide sufficient information for grasping and manipulation. Finally, the visual categorization was successfully performed with a variety of features and classifiers allowing proper categorization of unknown objects even when object appearance and shape substantially differ from the training set. Experimental evaluation with the humanoid robot ARMAR-IIIa is presented.

For complex manipulation with multiple objects a service robot needs information about the structure of its environment including how and where it can manipulate in it. For this purpose, we introduce the Graspability. It is a measure describing the quality of a pose in Cartesian space for grasping or placing an object. The graspability considers kinematic reachability for a grasping robot and available grasps for the object. It is based on the assumption, that in manipulation tasks, objects tend to be located on a planar surfaces and have to be graspable from that plane, based on that assumption, we develop a discrete map of the environment which enables the use of the graspability in an autonomous planning system for complex manipulation tasks with multiple objects. Generated manipulation actions are evaluated on a real robot.

In this paper parameter-free concepts for exact motion planning are investigated. With the proposed RDT+approach the collision detection parameters of a Rapidly exploring Dense Tree (RDT) are automatically adjusted until an exact solution can be found. For efficient planning discrete collision detection routines are used within the RDT planner and by verifying the results with exact collision detection methods, the RDT+ concept allows to compute motions that are guaranteed collision-free. We show the probabilistic completeness of the proposed planner and present an extension for handling narrow passages. The algorithms are evaluated in different experiments, including narrow passages and high-dimensional planning problems, that are solved in simulation and on the humanoid robot ARMAR-III.

An autonomous ice cream serving robot is presented in this video. The video was filmed during the Automatica 2010 trade fair in Munich. Within four days, approximately 250 scoops of different kinds of ice cream have been served to visitors during the fair. Using the KUKA light-weighted robotic arms and the DLR/HIT robotic hands, two scientific aspects are shown in this video: stable grasping of fragile objects and manipulation of cream-like mass using a time-of-flight camera. Impedance control are intensively used both by grasping and manipulation. The video shows an autonomous service robot scenario beyond simple fetch-and-carry chores.

In Programming by Demonstration (PbD), one of the key problems for autonomous learning is to automatically extract the relevant features of a manipulation task, which has a significant impact on the generalization capabilities. In this paper, task features are encoded as constraints of a learned planning model. In order to extract the relevant constraints, the human teacher demonstrates a set of tests, e.g. a scene with different objects, and the robot tries to execute the planning model on each test using constrained motion planning. Based on statistics about which constraints failed during the planning process multiple hypotheses about a maximal subset of constraints, which allows to find a solution in all tests, are refined in parallel using an evolutionary algorithm. The algorithm was tested on 7 experiments and two robot systems.

We introduce an approach for enabling sampling-based planners to compute motions with humanlike appearance. The proposed method is based on a space of blendable example motions collected by motion capture. This space is explored by a sampling-based planner that is able to produce motions around obstacles while keeping solutions similar to the original examples. The results therefore largely maintain the humanlike characteristics observed in the example motions. The method is applied to generic upper-body actions and is complemented by a locomotion planner that searches for suitable body placements for executing upper-body actions successfully. As a result, our overall multi-modal planning method is able to automatically coordinate whole-body motions for action execution among obstacles, and the produced motions remain similar to example motions given as input to the system.

In this work, we present a new software environment for the comparative evaluation of algorithms for grasping and dexterous manipulation. The key aspect in its development is to provide a tool that allows the reproduction of well-defined experiments in real-life scenarios in every laboratory and, hence, benchmarks that pave the way for objective comparison and competition in the field of grasping. In order to achieve this, experiments are performed on a sound open-source software platform with an extendable structure in order to be able to include a wider range of benchmarks defined by robotics researchers. The environment is integrated into the OpenGRASP toolkit that is built upon the OpenRAVE project and includes grasp-specific extensions and a tool for the creation/integration of new robot models. Currently, benchmarks for grasp and motion planningare included as case studies, as well as a library of domestic everyday objects models, and a real-life scenario that features a humanoid robot acting in a kitchen.

Anthropomatics addresses the symbiosis between humans and machines, focusing on a deeper understanding of the cooperation, interaction and coexistence between humans and machines stimulating and strengthen advanced and deep research in response to the challenges of increasingly smart environments and multimodal access to various complex technical systems. At KIT the Focus Anthropomatics and Robotics - APR has been set up by a number of research groups focusing on the research field of Anthropomatics and Robotics with more than 250 researchers. Modelling humans and their capabilities requires a deep understanding of the principle of biomechanics and kinematics, as well as the underlaying neural control principles and the perceptive and actuatoric system. Modelling and understanding of the sensomotoric mechanisms, learning and developement of skills and cognititve capabilities to enable humans to interact with the world is of high importance to design technical systems operating closely and interactively with humans via various modalities like speech, haptics, vision, grasping and locomotion. Typical research fields are related to active vision, interpretation of scenes and human activities, recognition and tracking technologies multimodal & perceptual user interfaces, understanding and translation of speech. Complementary research needed is related to the retrieval & access and summarization of multimedia data sources, translation of spoken text, context aware learning computers, implicit services and many more. The robotics application field ranges from interactive industrial robotics, service robotic companions, humanoids and medical robotics. In all domains the integrating aspects are focusing on algorithms processing real word data as well as open self-organizing architectures which allow autonomy, skill and task learning as well as interaction with humans. Our research emphasizes the critical path from basic understanding of cognitive processes and robotics foundations to applications in various domains. The research includes new approaches to sensor and actuator methodologies, foundations of machine perception, motion and action planning algorithms, simulation and computer graphics, robot machine learning, speech recognition and understanding, multimodal man-machine interaction, and many others. APR addresses the needs of humans in smart living environments and robotics focusing on both, basic foundations and applications. The focus has an intellectual center point on Anthropomatics and a commitment to Robotics as a new human centered discipline. The research is to explore new ideas and to build systematically systems which reflect human daily needs and a basic understanding of intelligent robots and adaptive safe system behaviour in general. Such cognitive robots are able to do tasks skillfully and efficiently together with humans, and they should be able to operate in dangerous or inaccessible areas. Furthermore, future robots should discover new things and learn new capabilities and knowledge about their environment and they should be able to reason about the effects of their actions and interventions. Thus, the basic capabilities to perform intelligent actions in the real world are perception, cognition, locomotion and skillful manipulation. The humanoid robot series ARMAR is used to illustrate recent research and achieved results in the field.

In this paper we propose the basic concept of a tele-presence system for two (or more) anthropomorphic robots located in remote locations. As one robot interacts with a user, it acquires knowledge about the user's behavior and transfers this knowledge to the network. The robot in the remote location accesses this knowledge and according to this information emulates the behavior of the remote user when interacting with its partner. The behavioral patterns are grouped into macro-behavior units (maBUs) and mirco-behavior units (miBUs). maBUs carry information that is specific to a person-to-person communication. miBUs are commonly used behavior patterns within a certain cultural environment (e.g. handshake, bow etc). maBUs are usually chains of miBUs. miBUs are chains of expression actions (e.g. gesture, facial expression etc.). The idea behind this is, that human communication contains several levels or layers of information exchange. This to be emulated by implementing the concept of miBUs and maBUs. We present a preliminary application of this concept to a musical context. The rhythmic motion of a drum-stick during the performance of a drum rhythm by a musician is recorded by a inertial measurements unit and transmitted between two far distance locations (Waseda University in Japan and Karlsruhe Institute of Technology in Germany) using three different transmission methods: direct raw data transmission, miBU based transmission and maBU-based transmission. We present experimental results that show quantitative data to evaluate the suitability of our approach, with the overall goal to implement a telepresence system of larger scale.

Bayesian Occupancy Filtering is an alternative to classical object tracking. Instead of estimating the state of objects in the environment, the latter is separated into equidistant cells. Tracking the occupancy state of these grid-cells is sufficient for many applications in robotics and cell-measurements can be easily produced from almost any kind of sensor. In [6] a sophisticated occupancy filter named BOFUM (Bayesian Occupancy Tracking using prior Map Knowledge) is introduced, which is able to infer velocities solely from occupancy measurements. It also features an advanced process model with motion uncertainty, which can be specialized for different application needs. In this paper we present an approach for recursively applying importance sampling (IS) to approximate the BOFUM calculations. The approach is similar to well known particle filters, but for a discrete cell perspective. In our experiments we achieved a speedup of at least 40-times by using the IS, thus making the algorithm applicable in real-world applications. We evaluate the consequences of approximation in an urban traffic scenario and also show the drawbacks of sampling.

In Programming by Demonstration, a flexible representation of manipulation motions is necessary to learn and generalize from human demonstrations. In contrast to subsymbolic representations of trajectories, e.g. based on a Gaussian Mixture Model, a partially symbolic representation of manipulation strategies based on a temporal satisfaction problem with domain constraints is developed. By using constrained motion planning and a geometric constraint representation, generalization to different robot systems and new environments is achieved. In order to plan learned manipulation strategies the RRT-based algorithm by Stilman et al. is extended to consider, that multiple sets of constraints are possible during the extension of the search tree.

The planning of grasping motions is demanding due to the complexity of modern robot systems. In Programming by Demonstration, the observation of a human teacher allows to draw additional information about grasping strategies. Rosell showed, that the motion planning problem can be simplified by globally restricting the set of valid configurations to a learned subspace. In this work, the transformation of a humanoid grasping strategy to an anthropomorphic robot system is described by a probabilistic model, called variation model, in order to account for modeling and transformation errors. The variation model resembles a soft preference for grasping motions similar to the demonstration and therefore induces a non-uniform sampling distribution on the configuration space. The sampling distribution is used in a standard probabilistic motion planner to plan grasping motions efficiently for new objects in new environments.

In this work, we present an integrated planner for collision-free single and dual arm grasping motions. The proposed Grasp-RRT planner combines the three main tasks needed for grasping an object: finding a feasible grasp, solving the inverse kinematics and searching a collision-free trajectory that brings the hand to the grasping pose. Therefore, RRT-based algorithms are used to build a tree of reachable and collision-free configurations. During RRT-generation, potential grasping positions are generated and approach movements toward them are computed. The quality of reachable grasping poses is scored with an online grasp quality measurement module which is based on the computation of applied forces in order to diminish the net torque.We also present an extension to a dual arm planner which generates bimanual grasps together with corresponding dual arm grasping motions. The algorithms are evaluated with different setups in simulation and on the humanoid robot ARMAR-III.

The autonomous acquisition of object representations which allow recognition, localization and grasping of objects in the environment is a challenging task, which has shown to be difficult. In this paper, we present a systems for autonomous acquisition of visual object representations, which endows a humanoid robot with the ability to enrich its internal object representation and allows the realization of complex visual tasks. More precisely, we present techniques for segmentation and modeling of objects held in the five-fingered robot hand. Multiple object views are generated by rotating the held objects in the robot's field of view. The acquired object representations are evaluated in the context of visual search and object recognition tasks in cluttered environments. Experimental results show successful implementation of the complete cycle from object exploration to object recognition on a humanoid robot.

A main challenge in the application of service robotics is safe and reliable navigation of robots in human everyday environments. Supermarkets, which are chosen here as an example, pose a challenging scenario because they usually have a cluttered and nested character. The robot has to avoid collisions with static and even with moving obstacles while interacting with nearby humans or a dedicated user respectively. This paper presents a hierarchical approach for the proactive avoidance of moving objects as it is used on the robot shopping trolley InBOT. The behavior-based control (bbc) of InBOT is extended by a reflex and a reactive behavior to ensure adequate reaction times when confronted with a possible collision. On top of the bbc a spatio-temporal planner is situated which is able to predict environmental changes and therefore can generate a safe movement sequence accordingly.

Typical tasks of future service robots involve grasping and manipulating a large variety of objects differing in size and shape. Generating stable grasps on 3D objects is considered to be a hard problem, since many parameters such as hand kinematics, object geometry, material properties and forces have to be taken into account. This results in a high-dimensional space of possible grasps that cannot be searched exhaustively. We believe that the key to find stable grasps in an efficient manner is to use a special representation of the object geometry that can be easily analyzed. In this paper, we present a novel grasp planning method that evaluates local symmetry properties of objects to generate only candidate grasps that are likely to be of good quality. We achieve this by computing the medial axis which represents a 3D object as a union of balls. We analyze the symmetry information contained in the medial axis and use a set of heuristics to generate geometrically and kinematically reasonable candidate grasps. These candidate grasps are tested for force-closure. We present the algorithm and show experimental results on various object models using an anthropomorphic hand of a humanoid robot in simulation.

3D data collected by a laser scanner has great potential for robotic applications. Exact geometrical models of the environment surrounding the robot can be created from these point clouds. But, before creating any model, the 3D point cloud has to be segmented and depending on the size and quality of the point cloud, this can be a very challenging task. This article describes a robust 3D scan segmentation technique, which is capable of segmenting a 3D point cloud in a short amount of time. The results of the segmentation are used to assist a teleoperator to manoeuvre a robot through an unknown environment. Our segmentation approach copes with indoor and outdoor environments, using only a minimum of assumptions, which makes it very robust. A 3D visualisation illustrates the segmentation results in a clear and user-friendly way.

In this paper we propose a process which is able to generate abstract service robot mission representations, utilized during execution for autonomous, probabilistic decision making, by observing human demonstrations. The observation process is based on the same perceptive components as used by the robot during execution, recording dialog between humans, human motion as well as objects poses. This leads to a natural, practical learning process, avoiding extra demonstration centers or kinesthetic teaching. By generating mission models for probabilistic decision making as Partially observable Markov decision processes, the robot is able to deal with uncertain and dynamic environments, as encountered in real world settings during execution. Service robot missions in a cafeteria setting, including the modalities of mobility, natural human-robot interaction and object grasping, have been learned and executed by this system.

An important challenge in service robotics is to design user interfaces and interaction capabilities that can be used intuitively to enable the potential users to benefit from the robot's functionalities and to allow effective collaborative task execution. The Interactive Behavior Operated shopping Trolley (InBOT) addresses this issue at the example of supporting complex shopping tasks in large supermarkets. The actual control of the robot is shared or traded between the robot and it's dedicated user ranging from closely coupled haptic-based interaction via observation-based interaction up to loosely coupled command-based interaction. The four modes of operation of InBOT, i.e. the manual steering, following, guiding and autonomous mode, the transitions between them and the relevant modalities of interaction are introduced. Finally the paper is completed by describing the results of user studies.

This paper presents a technique to learn flexible action selection in autonomous, multi-modal human-robot interaction (HRI) from observing multi-modal human-human interaction (HHI). A model is generated using the proposed technique with symbolic states and actions, representing the scope of the observed mission. Variations in human behavior can be learned as stochastic action effects while execution time perception noise is taken into account, using likelihood models. During execution, the model is used for dynamic action selection in HRI situations. The model as well as the evaluation system integrate the interaction elements of spoken dialog, human body configuration and exchanged objects. The technique is evaluated on a multi-modal service robot which is both able to observe the demonstration of two humans as well as execute the generated mission autonomously.

A new approach to the 3D human motion tracking problem is proposed, which combines several particle filters with a physical simulation of a flexible body model. The flexible body model allows the partitioning of the state space of the human model into much smaller subsets, while finding a solution considering all the partial results of the particle filters. The flexible model also creates the necessary interaction between the different particle filters and allows effective semi-hierarchical tracking of the human body. The physical simulation does not require inverse kinematics calculations and is hence fast and easy to implement. Furthermore the system also builds an appearance model on-the-fly which allows it to work without a foreground segmentation. The system is able to start tracking automatically with a convenient initialization procedure. The implementation runs with 10 Hz on a regular PC using a stereo camera and is hence suitable for Human-Robot Interaction applications.

Visual search is a common daily human activity and a prerequisite to the interaction with objects encountered in cluttered environments. Humanoid robots that are supposed to take part in human daily life should possess similar capabilities in terms of representing, attending to and recalling objects of interest in order to ensure robust perception in human-centered environments. In this paper, we present necessary processes, memories and representations which allow to identify and store locations of objects, encountered from different angles of view, in a visual search task. In particular, we introduce the so-called Feature Ego-Sphere (FES) as the scene memory for a humanoid robot. Experiments comprising different visual search tasks have been carried out on an active humanoid head equipped with perspective and foveal stereo camera systems. The scene is analyzed actively using both camera systems in order to find instances of searched objects in a consistent and persistent manner.

The problem of accurate 6-DoF pose estimation of 3D objects based on their shape has so far been solved only for specific object geometries. Edge-based recognition and tracking methods rely on the extraction of straight line segments or other primitives. Straight-forward extensions of 2D approaches are potentially more general, but assume a limited range of possible view angles. The general problem is that a 3D object can potentially produce completely different 2D projections depending on the view angle. One way to tackle this problem is to use canonical views. However, accurate shape-based 6-DoF pose estimation requires more information than matching of canonical views can provide. In this paper, we present a novel approach to 6-DoF pose estimation of single-colored objects based on their shape. Our approach combines stereo triangulation with matching against a high-resolution view set of the object, each view having associated orientation information. The errors that arise from separating the position and orientation computation in first place are corrected by a subsequent correction procedure based on online 3D model projection. The proposed approach can estimate the pose of a single object within 20 ms using conventional hardware.

In the recent past, the recognition and localization of objects based on local point features has become a widely accepted and utilized method. Among the most popular features are currently the SIFT features, the more recent SURF features, and region-based features such as the MSER. For time-critical application of object recognition and localization systems operating on such features, the SIFT features are too slow (500-600 ms for images of size 640×480 on a 3 GHz CPU). The faster SURF achieve a computation time of 150-240 ms, which is still too slow for active tracking of objects or visual servoing applications. In this paper, we present a combination of the Harris corner detector and the SIFT descriptor, which computes features with a high repeatability and very good matching properties within approx. 20 ms. While just computing the SIFT descriptors for computed Harris interest points would lead to an approach that is not scale-invariant, we will show how scale-invariance can be achieved without a time-consuming scale space analysis. Furthermore, we will present results of successful application of the proposed features within our system for recognition and localization of textured objects. An extensive experimental evaluation proves the practical applicability of our approach.

In this paper, we present efficient solutions for planning motions of dual-arm manipulation and re-grasping tasks. Motion planning for such tasks on humanoid robots with a high number of degrees of freedom (DoF) requires computationally efficient approaches to determine the robot's full joint configuration at a given grasping position, i.e. solving the Inverse Kinematics (IK) problem for one or both hands of the robot. In this context, we investigate solving the inverse kinematics problem and motion planning for dual-arm manipulation and re-grasping tasks by combining a gradient-descent approach in the robot's pre-computed reachability space with random sampling of free parameters. This strategy provides feasible IK solutions at a low computation cost without resorting to iterative methods which could be trapped by joint-limits. We apply this strategy to dual-arm motion planning tasks in which the robot is holding an object with one hand in order to generate whole-body robot configurations suitable for grasping the object with both hands. In addition, we present two probabilistically complete RRT-based motion planning algorithms (J+-RRT and IK-RRT) that interleave the search for an IK solution with the search for a collision-free trajectory and the extension of these planners to solving re-grasping problems. The capabilities of combining IK methods and planners are shown both in simulation and on the humanoid robot ARMAR-III performing dual-arm tasks in a kitchen environment.

For surgical planning in spine surgery, the segmentation of anatomical structures is a prerequisite. Past efforts focussed on the segmentation of vertebrae from tomographic data, but soft tissue structures have, for the most part, been neglected. Only sparse research work has been done for the spinal cord and the trachea. However, as far as the author is aware, there is no work on segmenting intervertebral discs. Therefore, a totally automatic reconstruction algorithm for the most relevant cervical structures is presented. It is implemented as a straightforward process, using anatomical knowledge which is, in concept, transferrable to other tissues of the human body. No seed points are required since the discs, as initial landmarks, are located via an object recognition approach. The spinal musculature is reconstructed by surface analysis on already segmented vertebrae, thus it can be taken into account in a biomechanical simulation. The segmentation results of our approach showed 91% accordance with expert segmentations and the computation time is less than 1 min on a standard PC. Since the presented system follows some general concepts this approach may also be considered as a step towards full body segmentation of the human.

This paper presents a reasoning system for a multi-modal service robot with human-robot interaction. The reasoning system uses partially observable Markov decision processes (POMDPs) for decision making and an intermediate level for bridging the gap of abstraction between multi-modal real world sensors and actuators on the one hand and POMDP reasoning on the other. A filter system handles the abstraction of multi-modal perception while preserving uncertainty and model-soundness. A command sequencer is utilized to control the execution of symbolic POMDP decisions on multiple actuator components. By using POMDP reasoning, the robot is able to deal with uncertainty in both observation and prediction of human behavior and can balance risk and opportunity. The system has been implemented on a multi-modal service robot and is able to let the robot act autonomously in modeled human-robot interaction scenarios. Experiments evaluate the characteristics of the proposed algorithms and architecture.

In this paper, a geometric approach for global self-localization based on a world-model and active stereo vision is introduced. The method uses class specific object recognition algorithms to obtain the location of entities within the surroundings. The perceived entities in recognition trials are simultaneously filtered and fused to provide a robust set of class features. These classified perceptions which simultaneously satisfy geometric and topological constraints are employed for pruning purposes upon the world-model generating the location hypotheses set. Finally, the hypotheses are validated and disambiguated by applying visual recognition algorithms to selected entities of the world-model. The proposed approach has been successfully used with a humanoid robot.

This paper proposes a SLAM algorithm based on FastSLAM 2.0 that maps features representing regions with a semantic type, topological properties, and an approximative geometric extent. The resulting maps enable spatial reasoning on a semantic level and provide abstract information allowing efficient semantic planning and a convenient interface for human-machine interaction. We present novel region features and an algorithm for estimating the feature parameters from uncertain measurements. In particular, we provide a means of estimating parameters even if the region feature is considerably larger than the robot's sensor range. Finally, we adapt the FastSLAM 2.0 algorithm to map the proposed features and show simulation-based results illustrating the capabilities of the proposed algorithm.

Daily life objects reveal natural similarities, which cannot be resolved with the perception of a single view. In this paper, we present an approach for object separation using active methods and multi-view object representations. By actively rotating an object, the coherence between controlled path, inner models, and percept is observed and used to reject implausible object hypotheses. Using the resulting object hypotheses, pose and object correspondence are determined. The proposed approach allows for the separation of different object candidates, which have similar views to the current percept. With the benefit of active methods the perceptual task can be solved using even coarse features, which facilitates a compact multi-view object representation. Furthermore, the approach is independent from a specific visual feature descriptor and thus suitable for multi-modal object recognition.

3D shape reconstruction of objects from tactile exploration data acquired by a multi-fingered robot hand is an important skill for a humanoid robot system. Tactile exploration data captured using current robot technology is naturally sparse and noisy, therefore a satisfying shape estimate is difficult to achieve. In this paper we describe a robust approach for 3D shape recovery using superquadric functions, which makes use of both contact location and normal information. We present two quality measures and compare to other relevant estimation techniques using representative synthetic contact data.

Motion planning for robots with many degrees of freedom (DoF) is a generally unsolved problem in the robotics context. In this work an approach for trajectory planning is presented, which takes account of the different kinematic parts of a humanoid robot. Since not all joints of the robot are important for different planning phases, the RRT-based planner is able to adapt the number of DoF on the fly to improve the performance and the quality of the results. The runtime of the approach is evaluated in comparison to a standard RRT planner. Futhermore several extensions to the algorithm are investigated.

In this paper, we propose a novel method for dexterous manipulation planning problem of rotating object with surface of revolution using a robotic multi-fingered hand. This method finds contact point trajectories from contact points between the robotic hand and the object with task-orientated manipulation quality measurement. Based on the defined manipulation quality, the pose for robotic hand relative to object can also be optimized by random sample. Experiments using Schunk anthropomorphic hand with 13 degrees of freedom screwing a light bulb into holder with screw thread demonstrates the feasibility and efficiency of the introduced method.

Human activity recognition is an essential ability for service robots and other robotic systems which interact with human beings. To be proactive, the system must be able to evaluate the current state of the user it is dealing with. Also future surveillance systems will benefit from robust activity recognition if real time constraints are met, allowing to automate tasks that have to be fulfilled by humans yet. In this paper, a novel approach for the integration of a feature selection in human motion recognition is proposed. Typically, the features are chosen with respect to the relevance of the features for the classification of the activity which shall be recognized. Our new approach extends this process by involving background knowledge about the features and active user engagement. Using taxonomies built on the complete feature set, users can be provided with an interface to guide and refine the selection process. Thereby, certain problems can be avoided which are common if noisy or small amounts of training data are used to train the system.

This paper presents an approach to model multi-modal human-robot interaction as partially observable Markov decision processes (POMDPs) for a service robot in realistic settings. Interaction modalities include spoken dialog and non-verbal human activities like gestures and general body postures. By using POMDPs which can model uncertainties in robot perception as well as human behavior, robustness and flexibility concerning autonomous decision making are improved in real world settings. This paper presents strategies to express perception uncertainties, stochastic human behavior and typical mission objectives in explicit POMDP models. Additionally, a system is presented to compile models from more compact representations. Finally, models are actually evaluated on a physical, autonomous service robot, controlled by POMDP decision making and compared to a classical baseline controller in typical domestic missions.

In this paper we present two approaches for intuitive interactive modelling of special object attributes by use of specific sensoric hardware. After a brief overview over the state of the art in interactive, intuitive object modeling, we motivate the modeling task by deriving the dierent object attributes that shall be modeled from an analysis of important interactions with objects. As an example domain, we chose the setting of a service robot in a kitchen. Tasks from this domain were used to derive important basic actions from which in turn the necessary object attributes were inferred.

In this paper, we present a framework for perception, visualization, reproduction and recognition of human motion. On the perception side, various human motion capture systems exist, all of them having in common to calculate a sequence of configuration vectors for the human model in the core of the system. These human models may be 2D or 3D kinematic models, or on a lower level, 2D or 3D positions of markers. However, for appropriate visualization in terms of a 3D animation, and for reproduction on an actual robot, the acquired motion must be mapped to the target 3D kinematic model. On the understanding side, various action and activity recognition systems exist, which assume input of different kinds. However, given human motion capture data in terms of a high-dimensional 3D kinematic model, it is possible to transform the configurations into the appropriate representation which is specific to the recognition module. We will propose a complete architecture, allowing the replacement of any perception, visualization, reproduction module, or target platform. In the core of our architecture, we define a reference 3D kinematic model, which we intend to become a common standard in the robotics community, to allow sharing different software modules and having common benchmarks.

In this paper, we present an image-based markerless human motion capture system, intended for humanoid robot systems. The restrictions set by this ambitious goal are numerous. The input of the system is a sequence of stereo image pairs only, captured by cameras positioned at approximately eye distance. No artificial markers can be used to simplify the estimation problem. Furthermore, the complexity of all algorithms incorporated must be suitable for real-time application, which is maybe the biggest problem when considering the high dimensionality of the search space. Finally, the system must not depend on a static camera setup and has to find the initial configuration automatically. We present a system, which tackles these problems by combining multiple cues within a particle filter framework, allowing the system to recover from wrong estimations in a natural way. We make extensive use of the benefit of having a calibrated stereo setup. To reduce search space implicitly, we use the 3D positions of the hands and the head, computed by a separate hand and head tracker using a linear motion model for each entity to be tracked. With stereo input image sequences at a resolution of 320 times 240 pixels, the processing rate of our system is 15 Hz on a 3 GHz CPU. Experimental results documenting the performance of our system are available in form of several videos.

Robust vision-based grasping is still a hard problem for humanoid robot systems. When being restricted to using the camera system built-in into the robot's head for object localization, the scenarios get often very simplified in order to allow the robot to grasp autonomously. Within the computer vision community, many object recognition and localization systems exist, but in general, they are not tailored to the application on a humanoid robot. In particular, accurate 6D object localization in the camera coordinate system with respect to a 3D rigid model is crucial for a general framework for grasping. While many approaches try to avoid the use of stereo calibration, we will present a system that makes explicit use of the stereo camera system in order to achieve maximum depth accuracy. Our system can deal with textured objects as well as objects that can be segmented globally and are defined by their shape. Thus, it covers the cases of objects with complex texture and complex shape. Our work is directly linked to a grasping framework being implemented on the humanoid robot ARM AR and serves as its perception module for various grasping and manipulation experiments in a kitchen scenario.

Robots with the capability of learning new tasks from humans need the ability to transform gathered abstract task knowledge into their own representation and dimensionality. New task knowledge that has been acquired e.g. with Programming by Demonstration approaches by observing a human does not a-priori contain any robot-specific knowledge and actions, and is defined in the workspace and action space of the human demonstrator. This paper presents an approach for mapping abstract human-centered task knowledge to a robot execution system based on the target system properties. Therefore the required background knowledge about the target system is examined and defined explicitely. The mapping process is described based on this knowledge, and experiments and an evaluation are given.

We present an approach to motion planning for highly articulated systems that aims to ensure robust execution by augmenting the planning process to reason about the robot's ability to successfully perceive its environment during operation. By simulating the robot's perception system during search, our planner generates a metric, the so-called perceptive capability, that quantifies the 'sensability' of the environment in each state given the task to be accomplished. We have applied our method to the problem of planning robust autonomous manipulations as performed by a humanoid robot in a kitchen environment. Our results indicate that reasoning about the future perceptive capability has the potential to greatly facilitate any task requiring visual feedback during control of the robot manipulator and can thus ensure higher task success rates than perception-unaware planning.

Vehicle guidance in complex scenarios such as inner-city traffic requires an in-depth understanding of the current situation. In order to select the appropriate behavior for an autonomous vehicle, an analysis of the situation is needed. The analysis consists of an estimation of the situation's development with respect to the selected behavior. This can only be done using higher-level reasoning techniques. In this paper, an approach for situation interpretation for autonomous vehicles is presented. The approach relies on case-based reasoning in order to predict the evolvement of the current situation and to select the appropriate behavior. Case-based reasoning allows to utilize prior experiences in the task of situation assessment.

Motion planning for humanoid robotic systems with many degrees of freedom is an important and still generally unsolved problem. To give the robot the ability of acting and navigating in complex environments, the motion planner has to find collision-free paths in a robust manner. The runtime of a planning algorithm is critical, since complex tasks require several planning steps where the collision detection and avoidance should be accomplished in reasonable time. In this paper we present an extension of standard sampling-based techniques using Rapidly Exploring Random Trees (RRT). We extend the free-bubble path validation algorithm from Quinlan, which can be used to guarantee the collision-free status of a C-space path between two samples. By using enlarged robot models it is possible to avoid costly distance calculations and therefore to speed up the planning process. We also present a combined approach based on lazy collision checking that brings together the advantages of fast sampling-based and exact path-validated algorithms. The proposed algorithms have been evaluated by experiments on a humanoid robot in a kitchen environment and by a comparison to a validation based on Quinlan's free bubbles approach.

A cognitive robot system has to acquire and efficiently store vast knowledge about the world it operates in. To cope with every day tasks, a robot needs to learn, classify and recognize a manifold of different objects. Our work focuses on an object representation scheme that allows storing perceived objects in a compact way. This will enable the system to store extensive information about the world and will ease complex recognition tasks. The human visual system deploys several mechanisms to reduce the amount of information. Our goal is to develop an artificial system that mimics these mechanisms to create representations that can be used in cognitive tasks. In particular, in this paper we will present an approach that exploits similarities among different views of objects. The proposed representation scheme allows for reduction of storage required for the representation of objects and preserves the information about the similarity among objects. This is achieved by selecting 'important views' of objects, depending on their stability. Furthermore, by extending the same approach to multiple objects, we are able to exploit similarities between objects to find a common representation and to further reduce the storage requirements.

Human activity recognition is an essential ability for service robots and other robotic systems which are in interaction with human beings. To be proactive, the system must be able to evaluate the current state of the user it is dealing with. Also future surveillance systems will benefit from robust activity recognition if realtime constraints are met, allowing to automate tasks that have to be fulfilled by humans yet. In this paper, a thorough analysis of features and classifiers aimed at human activity recognition is presented. Based on a set of 10 activities, the use of different feature selection algorithms is evaluated, as well as the results different classifiers (SVMs, Neural Networks, Bayesian Classifiers) provide in this context. Also the interdependency between feature selection method and chosen classifier is investigated. Furthermore, the optimal number of features to be used for an activity is examined.

Since many years the robotics community is envisioning robot assistants sharing the same environment with humans. It became obvious that they have to interact with humans and should adapt to individual user needs. Especially the high variety of tasks robot assistants will be facing requires a highly adaptive and user-friendly programming interface. One possible solution to this programming problem is the learning-by-demonstration paradigm, where the robot is supposed to observe the execution of a task, acquire task knowledge, and reproduce it. In this paper, a system to record, interpret, and reason over demonstrations of household tasks is presented. The focus is on the model-based representation of manipulation tasks, which serves as a basis for incremental reasoning over the acquired task knowledge. The aim of the reasoning is to condense and interconnect the data, resulting in more general task knowledge. A measure for the assessment of information content of task features is introduced. This measure for the relevance of certain features relies both on general background knowledge as well as task-specific knowledge gathered from the user demonstrations. Beside the autonomous information estimation of features, speech comments during the execution, pointing out the relevance of features are considered as well. The results of the incremental growth of the task knowledge when more task demonstrations become available and their fusion with relevance information gained from speech comments is demonstrated within the task of laying a table.

Alzheimer’s disease (AD) is a major public health challenge as the median age of the industrialized world’s population is increasing gradually. No cure for this disease has yet been found and the development of new treatments has become a topic of major research interest. This paper aims to propose a sequence of fully automated MRI-based image analysis steps to measure the development stage of atrophy in the brain. The results have been validated on a mixed group of 68 subjects by distinguishing between AD patients, MCIs and health controls using linear classifiers and ANNs. The best classifier identified unseen AD patients correctly in 80% of the cases and control subjects in 85%. Recognizing more than 8 out of 10 MCI subjects, the method also yields an early indication of AD. This simple yet powerful analysis can compete with othermore time-consuming and semi-automaticmethodologies. It could abet an AD diagnosis and provide a tool for measuring the success of therapies.

We propose a novel solution to the problem of inverse kinematics for redundant robotic manipulators for the purposes of goal selection for path planning. We unify the calculation of the goal configuration with searching for a path in order to avoid the uncertainties inherent to selecting goal configurations which may be unreachable because they currently lie in components of the free configuration space disconnected from the initial configuration. We adopt workspace heuristic functions that implicitly define goal regions of the configuration space and guide the extension of rapidly-exploring random trees (RRTs), which are used to search for these regions. The algorithm has successfully been used to efficiently plan reaching and grasping motions for a humanoid robot equipped with redundant manipulator arms

This paper proposes a tracking system called VooDoo for 3D tracking of human body movements based on a 3D body model and the iterative closest point (ICP) algorithm. The proposed approach is able to incorporate raw data from different input sensors, as well as results from feature trackers in 2D or 3D. All input data is processed within the same model fitting step by modeling all input measurements in 3D model space. The system has been implemented and runs in realtime at appr. 10-14 Hz. Experiments with complex human movements exhibit the characteristics and advantages of the proposed approach

Learning tasks from human demonstration is a core feature for household service robots. To increase the utility of future robot servants, the robot should go beyond simply imitating the user's behavior but try to build flexible, extensible and general task knowledge. This knowledge should at the same time encode the constraints of a task while leaving as much flexibility for optimized reproduction at execution time. This raises the question, which features of a task are the constraining or relevant ones both for execution of and reasoning over the task knowledge. In this paper, a system to record and interpret manipulation task demonstrations is presented. A heuristic measure for relevance assessment of task features is introduced. This relevance measure relies both on general background knowledge as well as task-specific knowledge gathered from the user demonstrations and incrementally improves with more task demonstrations becoming available. The utility of this relevance heuristic is evaluated within the problem of recognizing equal operations performed in different demonstrations of the same task in different contexts

A general solution for image-based object recognition and localization is still a goal far away. Therefore, the only way to tackle the problem is to apply the suitable approach for each specific problem. The most common techniques can be classified into global appearance-based, model-based, or histogram-based approaches, and approaches based on local features. In this paper, we concentrate on recognition and full 6D localization of solid colored objects of any geometry for real-time application on a humanoid robot system. State-of-the-art model-based methods can only deal with object geometries which can be broken down into 3D lines and planes, and thus can be efficiently projected into the image plane, which is not the case for most objects in a realistic scenario. In contrast, appearance-based methods have the power to be applicable for any object geometry, but are rarely combined with full 6D localization of objects, which is required for any realistic application in the context of grasping with a humanoid robot. We present a system which combines the benefits of global appearance-based and model-based approaches, resulting in a system which can acquire object representations automatically given its 3D model, and can recognize and localize solid-colored objects in 6D in an arbitrary scene in real-time

This paper proposes a new tracking algorithm within a 3D-SLAM framework that takes segmented range images as observations. The framework has two layers: the local layer tracks the view path and correspondences across the image sequence, using ambiguous landmark surfaces, and provides multiple hypotheses. The global layer conjectures and closes loops. We analyze the effect of different local trackers on the global layer. An interpretation tree (IPT) is compared to a new algorithm, orthogonal surface assignment (OSA), which attempts to track a building coordinate system. OSA is specialized to man-made work spaces. But our indoor experiments, performed with a rotating laser scanner, show clear advantages over the more general IPT: OSA covers the more relevant portion of the solution space, avoids the accumulation of rotation errors, and can estimate a unique translation in cases where IPT fails

In this paper we present a framework for grasp planning with a humanoid robot arm and a five-fingered hand. The aim is to provide the humanoid robot with the ability of grasping objects that appear in a kitchen environment. Our approach is based on the use of an object model database that contains the description of all the objects that can appear in the robot workspace. This database is completed with two modules that make use of this object representation: an exhaustive offline grasp analysis system and a real-time stereo vision system. The offline grasp analysis system determines the best grasp for the objects by employing a simulation system, together with CAD models of the objects and the five-fingered hand. The results of this analysis are added to the object database using a description suited to the requirements of the grasp execution modules. A stereo camera system is used for a real-time object localization using a combination of appearance-based and model-based methods. The different components are integrated in a controller architecture to achieve manipulation task goals for the humanoid robot

Learning tasks from human demonstration is a core feature for household service robots. To increase the utility of future robot servants, the robot should go beyond simply imitating the user's behavior but try to build flexible, extensible and general task knowledge. This requires higher level reasoning methods that allow the robot to consider its task knowledge in a holistic way. In order to cope with the vast datasets of task knowledge databases, these should be structured in a way that reflects the different classes of tasks as well as the specific characteristics of each task. As the complete set of tasks requested by the user can not be built into the robot, this structure should be learned from the database itself. In this paper, a system to record and interpret manipulation task demonstrations is presented. Unsupervised clustering methods on task knowledge are discussed that at the same time find the task class boundaries and the associated characteristics of each task class. These are used to group task demonstrations in the highly anisotropic feature space and to recognize similar demonstrations belonging to the same class of tasks. In each of the found task classes reasoning methods recover the sequential reordering possibilities, representing them in task precedence graphs. This equips the robot with the ability to update and improve its task knowledge without being put in a dedicated learning mode

Robot assistants in the same environment with humans have to interact with humans and learn or at least adapt to individual human needs. One of the core abilities is learning from human demonstrations, were the robot is supposed to observe the execution of a task, acquire task knowledge and reproduce it. In this paper, a system to interpret and reason over demonstrations of household tasks is presented. The focus is on the model based representation of manipulation tasks, which serves as a basis for reasoning over the acquired task knowledge. The aim of the reasoning is to condense and interconnect the knowledge. A measure for the assessment of information content of task features is introduced that relies both on general background knowledge as well as task-specific knowledge gathered from the user demonstrations. Beside the autonomous information estimation of features, speech comments during the execution, pointing out the relevance of features are considered as well

This paper presents an approach for human activity recognition focusing on gestures in a teaching scenario, together with the setup and results of user studies on human gestures exhibited in unconstrained human-robot interaction (HRI). The user studies analyze several aspects: the distribution of gestures, relations, and characteristics of these gestures, and the acceptability of different gesture types in a human-robot teaching scenario. The results are then evaluated with regard to the activity recognition approach. The main effort is to bridge the gap between human activity recognition methods on the one hand and naturally occuring or at least acceptable gestures for HRI on the other. The goal is two-fold: to provide recognition methods with information and requirements on the characteristics and features of human activities in HRI, and to identify human preferences and requirements for the recognition of gestures in human-robot teaching scenarios

Proper navigation of walking machines in unstructured terrain requires the knowledge of the spatial position and orientation of the robot. There are many approaches for localization of mobile robots in outdoor environment, but their application to walking robots is rather rare. In particular, middle sized robots like LAURON III don’t provide the possibility to carry large or heavy sensors. Due to many degrees of freedom of walking robots the localization task becomes a even more complex challenge. This paper discusses the problem, presents a method of resolution and describes the first steps towards a localization system for the six-legged walking robot LAURON III.

In this paper we present a new approach to 3D shape acquisition. This implementation enables a robot arm to move the scan unit over the object without the scan unit being tracked. The chosen structured light approach uses an initially unknown white noise pattern that can easily be projected with any fixed pattern projection system. Object points are acquired on the basis of finding correspondences in the pattern in the camera image of the current scene. This is achieved by using a fast SSD correlation algorithm. In the current test setup we are using a standard video beamer and a standard digital camera, but we have also set up a scan head for fixation on a robots’ wrist. The special requirements of this miniaturized system will be explained as well as our implementation. Our approach reduces hardware complexity to a minimum using only one calibrated camera and one calibrated fixed pattern projector. In this paper we present the whole system also including the calibration procedure. The focus will be on the hardware setup, but we also give an introduction to the software methods used.

This paper deals with building up a knowledge base of manipulation tasks by extracting relevant knowledge from demonstrations of manipulation problems. Hereby the focus of the paper is on modeling and representing manipulation tasks enabling the system to reason and reorganize the gathered knowledge in terms of reusability, scalability and explainability of learned skills and tasks. The goal is to compare the newly acquired skill or task with already existing task knowledge and decide whether to add a new task representation or to expand the existing representation with an alternative. Furthermore, a constraint for the representation is that at execution time the built knowledge base can be integrated and used in a symbolic planner.

This paper presents a method to potentate the multi-segmented inspection robot Kairo-II to navigate in unstructured and dynamic environment. Previous methods for motion planning for such robots come from driving scenarios in highly structured areas. The virtual tube algorithm is introduced which enables a multi-segmented robot to range in such complex environment. Precise force feedback is required. Therefore, we present a sensor system which is based on strain-gauges technology. Information extracted by this sensor enables the trajectory planning algorithm to adapt its curve. Thus, the proposed system provides and evaluates key functions for compliant motion of a multi-segmented robot within unstructured environment.

The Programming by Demonstration (PbD) technique aims at teaching a robot to accomplish a task by learning from a human demonstration. In a manipulation context, recognizing the demonstrator's hand gestures, specifically when and how objects are grasped, plays a significant role. Here, a system is presented that uses both hand shape and contact-point information obtained from a data glove and tactile sensors to recognize continuous human-grasp sequences. The sensor fusion, grasp classification, and task segmentation are made by a hidden Markov model recognizer. Twelve different grasp types from a general, task-independent taxonomy are recognized. An accuracy of up to 95% could be achieved for a multiple-user system.

To allow users without special knowledge to interact with robots, it is desirable to make interaction methods as intuitive as possible. This goal is in many cases difficult to achieve since data flow from the robot to the human is limited, especially if free locomotion of both the human and the robot are required. Therefore, new communication channels need to be created. We propose the use of an augmented reality display together with a wearable, wirelessly networked computer to achieve this goal. This system makes it possible to overlay planning, world model and sensory data provided by the robot over the wearer's field of view. We discuss the system architecture, interaction methods and experimental results. We demonstrate an example application for rapid prototyping of a warehouse transport system using the augmented reality system and a mobile platform. The user can create a topological map in an unknown environment on-the fly by setting and manipulating map nodes. This is done by pointing at the floor with a special interaction device, and issuing voice commands. The map is shown to the user as an augmentation of the real world view. Additionally, the robot's path planning data is visualized.

Efficient navigation of mobile platforms in dynamic, human centered environments is still an open research topic. We have already proposed an architecture (MEPHISTO) for a navigation system that is able to fulfill the main requirements of efficient navigation: fast and reliable sensor processing, extensive global world modeling and distributed path planning. Our architecture uses a distributed system of sensor processing, world modeling and path planning units. In this paper we present some implemented methods in the context of dynamic object detection and global and mobile sensor data fusion for 3D world modeling. Experimental results of the system in the laboratory environment are presented.

This paper presents the distributed robot control software architecture developed for the autonomous service robot Albert2. The development of this architecture is focused on two major issues: modularity and the integration of learning aspects. Each module within the architecture is presented, as well as the underlying event-based communication framework. An approach for integration of learning capabilities is proposed.

Reliable mapping and self-localization in three dimensions while moving is essential to survey inaccessible work spaces or to inspect technical plants autonomously. Our solution to this 3D SLAM problem is novel in several respects. First, a new rotating laser-scanning setup is presented for acquiring point clouds and reducing them to surface patches in real time. Second, the SLAM algorithms work entirely on highly reduced, attributed surface models and in 3D. Third, we propose a novel system architecture of an extended Kalman filter (EKF) for 3D position tracking, cooperating with a 3D range image understanding system for matching, aligning, and integrating overlapping range views. The system is demonstrated by an indoor exploration tour.

In this paper we present a modular and distributed control architecture in order to achieve natural interaction and mobile manipulation task goals for a humanoid robot. We propose a hierarchically organized architecture with three levels and introduce the mapping of the functional features in this architecture into hardware and software modules. We also describe different functional features which have been realized and integrated into the whole control architecture.

This paper deals with easy programming methods of dual-arm manipulation tasks for humanoid robots. Hereby a programming by demonstration system is used in order to observe, learn and generalize tasks performed by humans. A classification for dual-arm manipulations is introduced, enabling a segmentation of tasks into adequate subtasks. Further it is shown how the generated programs are mapped on and executed by a humanoid robot.

In this paper we want to introduce an approach to 3D scanning of dynamic scenes. This implementation enables the user not only to manually move a scan head over the object to scan but also to capture moving objects. Registration from scan to scan is done in real-time. The user interacts with the system and watches the scene assembling. He can immediately respond on shadings that occur due to undercuts in the scene. The chosen structured light scan method uses a primarily unknown speckle image that can be easily etched on a chrome-on-glass slide and projected using a strobe light. In the current large scale implementation we are using a standard video beamer and a standard digital camera. Miniaturization and adaptation for special purposes (i. e. medical applications) are scheduled for next year. Focus in this paper shall be laid on calibration issues regarding camera and projector.

Humanoid robotics is a new challenging field. To cooperate with human beings, humanoid robots not only have to feature human-like form and structure but, more importantly, they must possess human-like characteristics regarding motion, communication and intelligence. In this paper, we propose an algorithm for solving the inverse kinematics problem associated with the redundant robot arm of the humanoid robot ARMAR. The formulation of the problem is based on the decomposition of the workspace of the arm and on the analytical description of the redundancy of the arm. The solution obtained is characterized by its accuracy and low cost of computation. The algorithm is enhanced in order to generate human-like manipulation motions from object trajectories.

Locomotion of walking machines on a well defined path in unstructured terrain requires a model of the environment. But, in particular, middle sized robots like LAURON III don't provide the possibility to carry large or heavy sensors. This paper focuses on generating a 3D map of unstructured environment on the basis of sparse sensory information. In respect of walking robots this covers at first the selection of the next footsteps. For this purpose the advanced inference grid is introduced as a variant of the vector field histogram for the representation of the environment.

This paper presents improvements done to a programming by demonstration (PbD)system in order to handle complex one and two hand manipulations. In order to do this, functional roles were added to the systems knowledge base. According to them a probability density function expressing the relationship between the manipulated objects bas been set up. Since one object can fulfill several functional roles in different contexts multiple hypothesis are considered. This enables the system to detect in a more reliable way the goals and the sub goals of a human demonstrated task. Further it is pointed out how this goals can be reached by setting up a sequence of elemental actions, how these are generated and represented symbolically. Such a representation is important in order to build up complex tasks consisting of several subtasks and skills. Finally an experimental setup is presented in which household task like laying a table, pouring a glass of water, handling work tools can be understood, learned and generalized by the PbD system.

A Web-based system called KaViDo for collaborative research and development is presented. The architecture of the system including its three layers (presentation layer, development layer, persistence layer) is explained. The goals of KaViDo are to record development processes, manage the competences of distributed experts, exchange user experiences, and assist product development. Therefore six different application modules are presented The practical use of KaViDo is demonstrated by an interdisciplinary student contest. Different groups had to develop, build and program a mobile robot by using the KaViDo system and testing its efficiency.

The Programming by Demonstration (PbD) paradigm enable programming of service robots by inexperienced human users. The main goal of these systems is to allow the inexperienced human user to easily integrate motion and perception skills or complex problem solving strategies. Unfortunately, actual PbD systems deal only with manipulation based on Pick & Place operations. For complex service tasks these are insufficient. Therefore, this paper describes how fine manipulations like detecting screw movements can be recognized by a PbD system. In order to do this, finger movements and forces on the fingertips are gathered and analyzed while an object is grasped. This assumes sensory employment like a data glove and integrated tactile sensors. An overview of the used tactile sensors and the gathered signals is given. Furthermore the segmentation of users demonstration and the classification of the recognized dynamic grasp is pointed out. For classifying dynamic grasps a time delay method based on a Support Vector Machine (SVM) is used. Finally the symbolic representation of service tasks is briefly illustrated.

Giving an advice to a mobile robot still requires classical user interfaces. A more intuitive way of commanding can be provided by verbal or gesture commands. In this article, we present new approaches and enhancements for established methods that are in use in our laboratory. Our aim is to direct a robot with simple dynamic gestures. We focus on visual gesture recognition. Based on skin color segmentation algorithms for tracking the user's hand, hidden Markov models are used for gesture type recognition. The filters applied to the recorded trajectory strongly compress the input data. They also mark start and end point of a possible gesture. The hidden Markov models have been enhanced by a threshold model in order to wipe out insignificant movements. Pre-classification of the reference gestures serves for keeping computational effort low.

Easy programming methods following the programming by demonstration (PbD) paradigm have been developed. The main goal of these systems is to allow an inexperienced human user to easily integrate motion and perception skills or complex problem solving strategies. However, describing unconsciously performed actions or motor coordinations is very complex and in general not possible. This paper describes how tactile sensors are integrated in the PbD system which learns from human demonstration. An analysis of the tactile sensor and its characteristics is performed. Furthermore, the integration of tactile information in the systems' cognitive functions is pointed out. Finally, it is concluded that the enhancement of a data glove with tactile sensors improves the analysis of human demonstration. Moreover, the supplied information increases the sub-symbolic and symbolic task knowledge which lead to a more reliable recognition of the user's actions.

Presents methods and experiments of adaptive posture control for a four legged walking machine. Starting from the analysis of the implemented movement behaviour of BISAM we identify adequate tasks for adaptive control components and present adaptive posture control mechanisms for statically stable and dynamically stable movements. The reflex-based posture control is implemented via fuzzy control and reinforcement learning. The integration of the posture control in the control architecture is also described.

To meet the demand for robot to perform complex tasks, it is desirable to develop a methodology for intelligent behavior evolution in which a robot learns behaviors just as a human acquires dexterity, by repeated practice and use. Presented is a method for learning complex and dexterous behaviors through a knowledge array network, i.e., a network of knowledge arrays that play most important role as behavioral building blocks for robot behavior learning and evolution based on the intelligent composite motion control (ICMC). The process to realize a behavior from component element motions is presented. It is shown how a ball shooting behavior by a legged robot in robot soccer is realized according to the proposed method. Component element motions, are optimized. The optimal parameters obtained are then stored as a knowledge array, with which the robot can adaptively execute sub-optimal motions even for inexperienced situations. With the element motions optimized beforehand for a wide range of situations, the desirable shooting is obtained by combining them with additional optimization. The numerical result is given to demonstrate the presented method.

Service robots require interactive programming interfaces that allow users without programming experience to easily instruct the robots. Systems following the programming-by-demonstration (PbD) paradigm are getting closer to this goal. Visual observation of the user and environment is one important aspect for reasoning about goals and actions. With respect to the automatic generation of adaptive programs, a PbD-System should detect, classify and determine the pose of manipulable objects in a fast and stable way. This paper presents a comparison of four established methods proposing a new object classification approach that combines these methods. This gives a means for setting up a world model of a manipulations scene automatically and initializes active contour parameters in order to trade motions of these objects.

In this paper a multi-joint robot for sewer inspection tasks is presented. In order to increase the operating scope the robot has been designed to run round or over obstacles, to follow sewage branches and is operated with no wire attached to it. As a result of the wireless approach the robot has to carry an energy resource and must be abbe to act autonomously. In this paper we give a short description of the mechanical design and the electronic components used. Then we describe the control system and show sequences and results of in-pipe experiments.

The increasing demand for robotic applications in dynamic unstructured environments is motivating the need for dextrous end-effectors which can cope with the wide variety of tasks and objects encountered in these environments. The human hand is a very complex grasping tool that can handle objects of different sizes and shapes. Many research activities have been carried out to develop artificial robot hands with capabilities similar to the human hand. In this paper the mechanism and design of a new humanoid-type hand (called TUAT/Karlsruhe Humanoid Hand) with human-like manipulation abilities is discussed. The new hand is designed for the humanoid robot ARMAR which has to work autonomously or interactively in cooperation with humans and for an artificial lightweight arm for handicapped persons. The arm is developed as close as possible to the human arm and is driven by spherical ultrasonic motors. The ideal end-effector for such an artificial arm or a humanoid would be able to use the tools and objects that a person uses when working in the same environment. Therefore a new hand is designed for anatomical consistency with the human hand. This includes the number of fingers and the placement and motion of the thumb, the proportions of the link lengths and the shape of the palm. It can also perform most part of human grasping types. The TUAT/Karlsruhe Humanoid Hand possesses 20 DOF and is driven by one actuator which can be placed into or around the hand.

Modeling manipulators has been an important part in robotic simulations. There are various types of robot systems used in today's robotic research and application, e.g., the six axis industrial robots, humanoid redundant manipulators and 4 finger grippers. Therefore, the model's structure can be very complex, requiring techniques for both modeling and simulating the system. "Traditional" simulation packages handle each robot respectively. This paper presents a formal model for arbitrary manipulators or grippers and implement the theoretical ideas into a real modeling and simulation tool.

Service robots which should perform human-like operations will penetrate into a great number of applications in the future. Requirements for this is high flexibility, autonomy and the ability to adapt to new situations. The paper describes a design concept and a prototype implementation of an autonomous mobile humanoid service robot, which should mainly support people in their daily life as a personal or an assistance robot. The state of the research is that the general concept is developed and two anthropomorphic arms are built up. In the article the sensor system and the control architecture of the anthropomorphic robot are described. To evaluate the performance and motion abilities of the anthropomorphic arm the human arm kinematics and properties are discussed.

Presents an adaptive control architecture for the four legged walking machine BISAM. This architecture uses coupled neuro-oscillators as representation of periodic behaviours on different control levels such as joint movement, leg control and leg coordination. Coupled neuro-oscillators together with adaptive sensor based reflexes provide a robust and efficient representation for quadrupedal locomotion and support the use of online learning approaches to realize adaptation and optimization of locomotion behaviours.

In this paper a multijoint robot for sewer inspection tasks is presented. In order to increase the operating scope the robot has been made able to run round or over obstacles, to follow sewage branches and is aimed to work wirelessly unlike most other sewer inspection robots. As a result of the wireless approach the robot has to carry an energy resource and must be able to act autonomously. This article is focused on the mechanical design and the control hardware of the system. Additionally, we describe a first approach of a control strategy and some results of first tests.

Service robots require interactive programming interfaces that allow users without programming experience to easily instruct the robots. Systems following the programming by demonstration (PbD) paradigm that were developed within the last years are getting closer to this goal. However, most of these systems lack the possibility for the user to supervise and alter the course of program generation after the initial demonstration was performed. In this paper we present an approach, where the user is able to supervise the entire program generation process and to annotate, and edit system hypotheses. Moreover, the knowledge representation and algorithms presented enable the user to generalize the generated program by annotating conditions and object selection criteria via a 3D simulation and graphical user interface. The resulting PbD-system widens the PbD approach in robotics to the interactive generation of flexible robot programs based on demonstration and annotations.

Environment mapping is a very complex procedure that requires high CPU performance. For the last few years, laser scanners have become more and more important for mobile robots. Using their data requires many transformations between different coordinate systems. This new approach deals with a mapping within the coordinate system of the scanner, therefore it is very fast. The log-Hough transformation performs line finding in scans in a very efficient way, which speeds up mapping.

Since programming by demonstration (PbD) approaches have reached prime importance in interactive robot programming, sensor technology for tracking user actions and user behavior have become more and more important. However, traditional methods based on single sensor system input are already at their limits, since PbD is an application area where real-time requirement do play an important role. Thus, a sensor fusion approach is proposed which serves input data for a finite state automaton. The input sources are on the one hand a data-glove which classifies different grips and on the other hand a movable camera head which tracks the movements of the data glove as well as estimates object positions. Both sensor sources use time efficient algorithms, since sensory data must be processed in real-time. The efficiency of this approach is proven in a PbD environment were flexible infusion bags are handled and respective actions and object positions are recognized.

Stewart platforms are mostly used for simulator applications connected to a large graphic computer. This application is fixed and can not be greatly modified. This paper deals with a new concept of a controller for the Stewart platform that is used in multimedia applications. The controller has to be very flexible, since various and very different applications will run on the Stewart platform. First, the kinematics and dynamics of the platform used are explained. After describing the necessary calibration the new concept for the control architecture is developed. Finally, the filtering technique for the generation of movements in order to simulate acceleration is presented.

In this paper a concept for a wheeled, multijoint robot able to operate in sewage systems is presented. The robot should work in an autonomous way. This concerns power supply, control and information processing. The cable-free navigation and the multijoint redundant construction of the robot enable higher mobility in sewage systems. In contrast to present systems, the robot should be able to avoid and overcome small obstacles, e.g socket displacements, holes or sediments, and to pass junctions and curves. In the following the results of a feasibility study are described in which the state of the art in sewer inspection robot as well as first experiments for the development of such a system is shown.

The mobile service robots described are designed to operate in dynamics and changing environments together with human beings and other static or moving objects. Sensors that are capable of providing the quality of information that is required for the described scenario are optical sensors, like digital cameras and laser scanners. In this paper the sensor integration and fusion for such sensors is described. Complementary sensor information is transformed into a common representation in order to achieve a cooperating sensor system. Sensor fusion is performed by matching the local perception of the laser scanner and camera system with a global model that is being build incrementally. The Mahalanobis distance is used as matching criterion and a Kalman filter is used to fuse matching features. A common representation including the uncertainty and the confidence is used for all scene features. The system's performance is demonstrated for the task of exploring an unknown environment and incrementally building of the geometrical model.

This paper presents an intelligent adaptive systcm for the integration of haptic output in graphical user interfaces.The system observes the user's actions, extracts meaningful features, and generates a user and application specific model.When the model is sufficiently delailled, it is used to predict the widget which is most likely 10 be used next by the user.Upon entering this widget, two magnets in a specialized mouse are activated to stop the movement, so target acquisition becomes easier and more comfortable.Besides the intelligent control system, we will present several methods to generate haptic cues which might be integrated in mttltimodal user interfaces in the future.

This paper presents a general approach to the acquisition of sensor-based robot skills from human demonstrations. Since human-generated examples cannot be assumed to be optimal with respect to the robot, adaptation of the initially acquired skill is explicitly considered. Results for acquiring and refining manipulation skills for a Puma 260 manipulator are given.

Computer vision systems are today an important sensor for intelligent robotic systems. However, the design of a vision system that a robot can use as a fast and robust sensor in a complex, partially unknown and dynamic environment is still difficult. A main reason for this is that the parameters of vision systems are often adjusted by hand and remain static during the operation of the robot. In this paper we present a general architecture that adapts the the parameters of a segment based low-level vision system dynamically to increase its speed and robustness. Adaptation is done to a priori knowledge about the environment or to the sensor data itself. The architecture is implemented on a mobile robot using special hardware that allows real-time operation. Quantitative experimental data on its performance is given.

While distributed control architectures have many advantages over centralized ones, such as their inherent modularity and fault tolerance, a major problem of such architectures is to ensure the goal-oriented behaviour of the controlled system. This paper presents a framework within which the coordination skills required for goal-orientedness are learned from user demonstrations. The framework is based on a state-space model of the single agents building the system and a corresponding model of the coordination mechanism. Our mobile robot PRIAMOS provides an application example.

The exploration and mapping of unknown environments is on important task for the new generation of mobile service robots. These robots are supposed to operate in dynamic and changing environments together with humans and in interaction with other stationary or moving objects. This requires a high flexibility and adaptability of the sensor-system to changing environmental conditions. Sensors that are capable of providing the quality of information that is required for the described scenario are optical sensors like digital cameras and laserscanners. In this paper a sensor system and an architecture for active control of the sensors and adaptive processing of the perceived sensor data are developed for service applications and experimentally evaluated.

A good map of its environment is essential for efficient task execution of a mobile robot. Real time map update, especially in dynamic scenes is a difficult problem due to noisy sensor data and limited observation time. The paper describes a mapping procedure which identifies new obstacles in a scene and constructs a 3D surface model of it. This description is included in the geometrical map which robot navigation relies on. The mapping procedure is based on sonar range information and scenes reconstructed from stereo vision. The combination of sonar and stereo vision is advantageous, due to a complementary error characteristic concerning range and angular resolution. For sonar data integration the idea of local probability grids is proposed. Local grids which only cover areas where new obstacles are expected, reduce the complexity of grid based sonar data integration and can be applied to a dynamic environment. The partial models of an object that has been observed from different viewpoints are fused to a homogeneous description in a later step. A complex example shows the mapping procedure work robustly in dynamic indoor environments.

In this paper an approach to real-time position correction and environmental modeling based on odometry, ultrasonic sensing, structured light sensing and active stereo vision (bin- and trinocular) is presented. Odometry provides the robot with a position estimation and with the help of a model of the environment sensor perceptions can be matched to predictions. Ultrasonic sensing is capable of collision avoidance and obstacle detection and so enables navigation in simply structured environments. Model-based image processing allows detection and classification of natural landmarks in the stereo images uniquely. With only one observation the robot's position and orientation relative to the observed landmark is found precisely. This sensing strategy is used when high precision is necessary for the performance of the navigation task. Finally techniques are described that allow an automatic mapping of an unknown or only partially known environment.

The problem of adapting mobile robot navigation to changes in the environment is usually approached by modifying an internal world model. Descriptions on different levels of abstraction provide the information necessary for navigation and therefore influence the robot's behaviour. The effect of such indirect adaptation is limited. The approach presented in this paper describes a new technique for direct integration of navigation experience in path planning. Thus, not only the world knowledge, but also the planning behaviour is improved over time. Experiments are carried out on a robot which is controlled by a layered architecture. It is integrated in a multirobot control environment which is described. The focus of the article is towards improving the higher navigation levels. The main idea being presented is the realization of adaptive behaviour not only on the level of reflexes, but also with respect to the planning capabilities of the robot. The application of learning techniques allows to continuously improve the estimation of plan costs and therefore the inherent strategy of the topological planner. It is illustrated that a combined learning of world description and navigation allows fast and sophisticated reaction to new environmental conditions.>

In the following report the dynamic control of a robot leg is described. The control algorithm is trained using self-organizing feature maps. This approach belongs to the area of unsupervised learning techniques. The dynamic control is tested using a simulation system. Thereafter, it is used to control the physical robot leg.

F/sub A/TE, an online task-level planning system for an advanced assembly robot consisting of two manipulators and a set of different sensors such as an overhead camera and force-torque sensors, is presented. The planning system takes an implicit description of the assembly task, plans a sequence of explicit robot commands and monitors the execution by the real-time robot control system. Because it runs completely online, the planning process is highly reactive using sensor information about the robot's present environment. This planning integrates as a key feature a dynamic mapping of assembly subtasks ready for execution onto the manipulators available at the moment. The system was implemented in Prolog on a SUN 4/75 SPARCstation running Unix.>

The problem of correctly evaluating noisy and incorrect data for the interpretation of ultrasonic sensor signals is addressed. Neural networks, with their inherent characteristics of adaptivity and high fault and noise tolerance, are well suited for such tasks. A backpropagation algorithm is described for the control of the tracking behavior of an autonomous mobile robot. Input data are provided by three ultrasonic sensors mounted on the front of the vehicle. For more flexibility the behavior and learning capability of the tracking algorithm have been improved using different networks.>