During the last decade(s), few new application fields for industrial robotics have developed although exploiting the state-of-the-art in (industrial) robotics research may unlock a plethora of new applications in industrial manufacturing and other areas. Ongoing structural changes affecting the population of Japan and various European countries, esp. population ageing and accompanying health issues, increase the need for assistive robotics systems in human-centered production environments and require the automation of potentially complex, but unergonomic tasks. Due to various issues - addressed in the paper - current robot control architectures and programming paradigms do not meet these needs. Fortunately, technology leaders in industrial robotics are currently striving to innovate and address these needs. Academia-industry collaboration can successfully accelerate these innovation processes and may thus have significant impact on commercial products and solutions. After reviewing the above mentioned needs and identifying crucial issues in current robot control architectures and programming paradigms, we present four case-studies which contributed to solving some of these issues together with industrial partners. Several factors contributing to the success and failure of academia industry partnerships related to our own project experience are discussed.

We present a method for the detection of dynamic objects in the data of wide angle time-of-flight cameras. A vehicle is equipped with four cameras to incorporate a 360° vision by depth cameras. Depth measurements can be used for environment mapping. Dynamic objects however, lead to errors in the reconstruction of static scenes. Our method detects objects in the surrounding of the vehicle and discriminates between static and dynamic objects. Segments and keypoints are extracted from combinations of depth and amplitude images. After the generation and description of objects the vehicle odometry is incorporated for optimal object matching between consecutive images. Based on the computed movements of the objects a classification into static or dynamic is done. We evaluated our method based on different scenarios and under real world conditions including low lighting conditions and influence of external infrared light.

One major problem, when using statistical shape information in image segmentation problems, is that many training samples are needed in order to obtain a satisfactory shape prior for a particular class, especially when the intra-class variability of the object shapes is high. To cope with this problem, we present an elegant variational formulation that allows local adaptations of the parameters associated with a trained shape prior. This enables us to obtain accurate segmentation results with a limited amount of training shapes. We provide a sound mathematical foundation for our approach and embed it into the well-known level set segmentation framework, which makes our approach applicable to a large class of problems. Moreover, we show how a smooth transition from global to local adaptations of the shape parameters can be achieved. We point out the advantages of our new variational global-to-local approach by comparing it with another level set segmentation approach that includes a global shape prior.

Grasping objects is a well-known problem in robotics. If the objects to be grasped are known, usually they are to be placed at a desired position in a desired orientation. Therefore, the object pose w.r.t the gripper has to be known before placing the object. In this paper we propose a simple and efficient, yet robust approach to this challenge, which can (nearly) eliminate dead times of the employed manipulator - hence speeding up the process significantly. Our approach is based on the observation that the problem of finding a pose at which the object can be grasped and the problem of computing the pose of the object w.r.t. the gripper can be solved separately at different stages. Special attention is paid to the popular bin-picking problem where this strategy shows its full potential. To reduce the overall cycle time, we estimate the grasp pose after the object has been grasped. Our estimation technique relies on the inertial parameters of the object - instead of visual features - which enables us to easily incorporate pose changes due to grasping. Experiments show that our approach is fast and accurate. Furthermore, it can be implemented easily and adapted to diverse pick and place tasks with arbitrary objects.

Manipulation Primitives are state-of-the-art for describing and performing complex manipulation tasks - especially in industrial applications. They combine easy task specification and execution which are key components for the success of this paradigm. However, there are some flaws when describing a complex task by decomposing it into Manipulation Primitives. For instance, the relations between the individual Manipulation Primitives cannot be captured by the original definition but have to be defined in additional glue code. This contribution introduces an extended definition of Manipulation Primitives to describe manipulation tasks as hierarchical structures using Manipulation Primitives at each level of the hierarchy. Additionally, a concept for a generic control architecture is presented which is capable of meeting the requirements of the extended Manipulation Primitive concept and which can be easily plugged“ together using modular code blocks.”

Displacement osteotomy is an operative method for load relieving of pre-damaged joints through the correction of skeletally-based abnormal displacements of leg axes. We present an experimental operating room set-up to demonstrate the applicability of a robot guided water jet osteotomy for human tibial bones. We show that smooth, circular shaped cut surfaces can be obtained with this technology that allow for an easy and precise correction of the leg axis.

In this paper we present a new approach for monocular image based lane border detection in situations where the characteristics of the pavement and the lane border are unknown. To achieve real time performance on standard hardware we analyze statistical characteristics of 1D signals on certain profile lines to find different types of features which belong to the lane border. These features are used for fitting a border model in. The proposed method shows good results in complex situations. It was evaluated in different scenarios with cobblestone pavement, lowered curbs, lane markers, parking cars defining the lane border as well as disturbances on the road like shadows, dirt and asphalt damages. The method was successfully used within the Volkswagen research vehicle ‘eT’ (electronic Transporter).

We present a method that estimates the ego motion of a vehicle based on camera data of high resolution (207x204 pixels) Time-of-Flight cameras using visual odometry techniques. Translation and rotation of camera motion in six degrees of freedom are calculated. Point correspondences in consecutive amplitude image pairs are built. By consideration of the depth image of the camera 3D point correspondences are derived from the two dimensional point correspondences. Camera motion between the two images is then computed by registration of the two resulting point clouds. The process is optimized by incorporation of outlier removal and a multi sensor setup. The presented optimizations raise the precision and robustness of the method and enable visual odometry by Time-of-Flight camera data as an alternative to common odometry systems in low speed scenarios.

The problem of object localization is a well-known problem in industrial robotics. Manufactured parts arrive at factories as bulk goods in boxes. Single parts need to be picked out of the boxes and have to be fed to a machine. The task of automatically isolating single objects is known as the bin-picking problem. Even in modern factories the task of bin-picking is not automated widely yet. The automatization of this task is expensive since state-of-the-art solutions require object-class specific algorithms. In this paper we present an applicable solution for the bin-picking problem which is based on a standard 3d-sensor and is able to handle arbitrary objects. Furthermore, it is robust against noise and object occlusions. Additionally, we propose an approach for optimal grasp pose estimation with collision avoidance that effectively reduces system cycle times.

Rotary encoders and resolvers are by far the most common sensors to measure joint angles in articulated robots. Since various control approaches require angular rates as well, resolvers and encoders are also employed to derive angular rate signals. Due to the involved differentiation operation, however, quantization noise may be augmented significantly. Advanced filtering approaches can only partially overcome this drawback. Therefore, direct measurement of angular rates is desirable. Due to advancements in manufacturing technology and pushed by applications in entertainment devices, MEMS gyroscopes have become an attractive alternative for angular rate measurement. Unfortunately, they are affected by bias and other non-negligible disturbances, which may be a serious problem if consistent joint angle and angular rate measurements are required. The proposed approach fuses encoder and angular rate signals to mitigate their major drawbacks - bias and quantization errors - by exploiting their individual strong points. Bias in the angular rate signal is eliminated by analyzing the deviation between the integrated angular rate signal and the encoder signal. Quantization errors in the encoder signal are reduced by a so-called complementary filter which blends the integrated angular rate output with the encoder signal. Apart from a description of the approach and a theoretical analysis of its characteristics, experimental results demonstrating the effectiveness of the approach in closed-loop control of articulated robots are presented.

Cameras are a commonly used sensor in advanced driver assistance systems (ADAS). They serve to get vast amounts of information about a vehicle's environment. To accurately localize the measured data in relation to the own car, exact camera calibration is a prerequisite. This includes extrinsic as well as intrinsic parameters. While many works in the area of ADAS focus on extrinsic calibration, this work covers the intrinsic calibration. We use a generic camera model which regards the viewing ray of every pixel separately and can therefore be used to describe arbitrary imaging devices even with massive lens distortions. As the calibration procedure works for any camera, only one method has to be implemented, which simplifies the sensor calibration process. Former works have shown the applicability of generic camera models but do not cover important practical aspects which are subpixel ray determination and forward projection of arbitrary 3d points to the image plane. Furthermore, the calibration processes described so far are cumbersome and prone to inaccuracies. We propose to use spline surfaces to simplify the calibration procedure and implement general back and forward projection. The applicability of our approach is proved by showing calibration results for various real cameras.

Online trajectory scaling approaches represent a possible way for handling robotic systems characterized by kinematic or dynamic saturations. Scaling methods are based on the path/velocity decomposition principle: a dynamic filter is used to modify the longitudinal velocity along a planned path in order to satisfy given system constraints. The strategy here proposed enhances the results of a previous work by enlarging the number of considered constraints. In particular, still accounting for the existence of bounds on joint velocities and torques, in this paper, the presence of constraints on joint accelerations is also considered. Furthermore, the nonlinear filter, which represents the core of the scaling system, has been revised in order to devise a new and more compact implementation. Finally, some practical issues that could occur in actual implementations are discussed and solutions are proposed to overcome possible problems.

Electric motors clearly constitute the most common drive principle in robotics and mechatronics. Smart materials, however, offer considerably higher power-to-mass ratios than electric motors. If mechanical energy instead of electrical energy can be distributed through a system, highly dynamic and efficient torque transmission elements based on smart materials, e.g. piezoceramics, can be used to transmit torque from an input to an output element. Just like electric motors, they can thus provide position, velocity, and force-torque control of the output element. This paper introduces machine components, called adaptronic couplers, which can transmit variable torques highly dynamically from an input element to an output element employing static and/or dynamic friction. In the long run, systems (e.g. robots) based on these machine components are envisaged to compete with systems based on classic drive principles - especially electric motors - w.r.t. dynamics and power-to-mass-ratio. Apart from the concept itself, this paper addresses different control approaches and discusses their influence on energy consumption and wear. Moreover, various experimental results proving the basic concept are presented.

Rotary encoders are the most common sensors to measure angles in mechatronic and robotic components, e.g., servo motors or robot joints. Especially, if encoders are not mounted on the motor shaft but on the output side of geared motors, high encoder resolutions are required. Resolution requirements may increase further if velocities are to be derived from the encoder signals, e.g., for motion control purposes. To avoid noise amplification problems when estimating angular rates from encoder signals, angular velocity sensors may be employed instead. However, a significant drawback of angular rate sensors - particularly of cheap MEMS gyroscopes - is their drift.

One of the main tasks of intelligent vehicles is the extraction of information from the vehicle's surroundings and the understanding of the extracted information. The understanding of the environment allows the vehicle to drive autonomously or to support the driver in difficult or dangerous situations. In this paper we propose a vision-based hierarchical interpretation approach. First, we consider one single physical camera as a set of virtual sensors, where each virtual sensor gathers a type of 3d information. Then, the 3d information of this set is converted to high-level information that allows further reasoning. The interpretation is based on a hierarchical scene representation, where objects are recognized using nonparametric belief propagation. To demonstrate this approach we adopted the scene understanding to a parking spot finding application and show that it is real-time applicable and reliable even for multiple camera (on-board) systems.

This contribution introduces a generic framework for sensor-based robot motion control. The key contribution is the introduction of an adaptive selection matrix for sensor-based hybrid switched-system control. The overall control system consists of multiple sensors and open- and closed-loop controllers, in-between which the adaptive selection matrix can switch discretely in order to supply command variables for low-level controllers of robotic manipulators. How control signals are chosen, is specified by Manipulation Primitives, which constitute the interface to higher-level programming. This programming paradigm is briefly specified in order to be able to define and execute sensor-guided and sensor-guarded motion commands simultaneously. The resulting control system is freely adaptable depending on the sensor and control requirements of the desired system and/or application.

This paper presents an algorithm for the global localization of mobile robots. The general idea is to build a local map incrementally which is matched to a global map in each localization step. The matching algorithm is a very time and memory efficient enhancement of the common Random Sample Consensus (RANSAC). It is executed a fixed number of iterations computing a set of hypotheses of the current robot pose. This paper describes how to handle the resulting hypotheses, i.e. a method is introduced deciding when the robot is localized reliably enough. The algorithm has been implemented and its characteristics are evaluated and discussed in an experimental section.

The capability of solving the simultaneous localization and mapping (SLAM) problem is one of the fundamental tasks of mobile robots and many research has focused on this problem over the last decades. In this paper, the SLAM problem is considered as the problem of finding an optimal path through a tree resulting in minimum costs. For this purpose, we apply the Ant Colony Optimization meta-heuristic, which belongs to the class of ant algorithms. It has been successfully employed to solve the well known Traveling Salesman Problem with several thousands of cities. We use a simple scan matching technique for generating a rough pre-solution to the SLAM problem. The (inconsistent) map is partitioned into fragments. A new fragment is initialized as soon as the robot has moved several meters. We draw samples from Gaussian distributions representing alignments of consecutive fragments. The resulting set of samples is interpreted as a tree-like data structure with weights assigned to the edges. We use our own variant of an ant algorithm for finding the optimal path through the tree. Real-world experimental results demonstrate the characteristics of our method.

This paper suggests the idea of a universal method for stabilizing discrete-time hybrid switched-control systems of robot manipulators. The core of this idea is based on an on-line trajectory generation algorithm that is able to generate continuous command variables from any arbitrary state of motion. We define a measurable criterion to on-line detect an instability or a potential instability of the plant, and right after this criterion is fulfilled, we switch to the on-line trajectory generator that acts as an open-loop pose control submodule in the switched-system. The on-line trajectory generation algorithm guides the system under consideration of kinematic motion constraints to a desired target state of motion that can be specified beforehand (e.g., zero-velocity in a pre-defined position). Systems with one and more degrees of freedom are regarded in this paper; finally, real-world experimental results achieved with a six-joint industrial manipulator are presented in order to demonstrate the potential and the high practical relevance of this concept.

In this video we show our current prototype for robot assisted endoscopy. The system requires only few and simple instructions from the surgeon, in order to guide the endoscope in an intelligent, autonomous, and safe way: The surgeon tells what to do and the robot decides how to carry out the task by choosing the best manipulation primitive in every control cycle. Several sensors are integrated into the decision process: The endoscope camera, a stereo camera system, a force/torque sensor, a biomechanical model based on CT data and statistical knowledge, a position and velocity sensor for the manipulator, and interface devices like a foot switch. The executed manipulation primitive is handled by a hybrid controller allowing to switch the control mode (e.g. trajectory following or force control) for each degree of freedom of the task frame individually.

Force guided assembly is attractive but the implementation is challenging when uncertainties are present. In these cases, contact models that guide the assembly process are attractive. This article elaborates the usage of static contact models, which map displacements to force-torque vectors of particular contact situations (force-torque map). This model, a map of discrete points, contains force and torque values that correspond to position errors. The inversion of this map, using forces and torques measured from an assembly attempt, yields correction movements in order to accomplish the assembly iteratively. A hypothesis stating the dependency of the model's quality on its injectivity is investigated. This aspect is studied thoroughly in so-called redundancy maps, which reveal regions of considerable ambiguity of the model. Experimental results are presented, which validate the hypothesis about the dependency of the convergence of the assembly process on the ambiguity of the initial position. In addition to the peg-in-hole problem, which has become a standard scenario to validate force guided assembly, the scope of this article also covers force guided assembly of more complex parts. Here, the analysis gives evidence to believe that it is unlikely that the implementation convergences acceptably, which is validated by experimental results.

This paper introduces a new method for motion-trajectory generation of mechanical systems with multiple degrees of freedom (DOFs). The key feature of this new concept is that motion trajectories are generated online, i.e., within every control cycle, typically every millisecond. This enables systems to react instantaneously to unforeseen and unpredictable (sensor) events at any time instant and in any state of motion. As a consequence, (multi)sensor integration in robotics, in particular the development of control systems enabling sensor-guided and sensor-guarded motions, becomes greatly simplified. We introduce a class of online trajectory-generation algorithms and present the mathematical basics of this new approach. The algorithms presented here consist of three steps: calculation of the minimum synchronization time for all DOFs, synchronization of all DOFs, and calculation of output values. The theory is followed by real-world experimental results indicating new possibilities in robot-motion control.

Today, femoral shaft fractures are usually stabilized by means of intramedullary nails. This video presents the development of a telemanipulator system, which, by supporting the fracture reduction process, aims at achieving reliable operation results with high reduction accuracies. First, we present a system using 2D X-ray images as base information for the surgeon to guide the reduction. We show the advantages but also the limitations of this approach, which finally led to the development of a telemanipulator system that is based on 3D imaging data instead.

This paper introduces a parallelized variant of the Random Sample Matching (RANSAM) approach, which is a very time and memory efficient enhancement of the common Random Sample Consensus (RANSAC). RANSAM exploits the theory of the birthday attack whose mathematical background is known from cryptography. The RANSAM technique can be applied to various fields of application such as mobile robotics, computer vision, and medical robotics. Since standard computers feature multi-core processors nowadays, a considerable speedup can be obtained by distributing selected subtasks of RANSAM among the available cores. First of all this paper addresses the parallelization of the RANSAM approach. Several important characteristics are derived from a probabilistic point of view. Moreover, we apply a fuzzy criterion to compute the matching quality, which is an important step towards real-time capability. The algorithm has been implemented for Windows and for the QNX RTOS. In an experimental section the performance of both implementations is compared and our theoretical results are validated.

In bilateral teleoperation, the operator experiences forces and torques applied to the slave manipulator. These forces and torques, however, consist of two components: on the one hand, forces and torques due to contacts with the environment, and on the other hand, non-contact forces, i.e., inertial forces, centrifugal forces, Coriolis forces, and associated torques. For several reasons, eliminating these non-contact forces and torques from the force-torque measurements of the slave or scaling them can be advantageous. For instance, in highly-dynamic teleoperation tasks, these forces and torques may contribute to operator fatigue or hamper the detection of contacts with the environment. This paper briefly reviews the estimation of inertial parameters of the slave load, e.g., an end-effector or a gripper. Subsequently, a method for eliminating the non-contact forces and torques from the measurements of a wrist-mounted force-torque sensor or scaling them is presented. After a brief overview of our teleoperation system, experimental results are presented which demonstrate the effectiveness of our approach.

Teleoperation has a long history in the robotics community and numerous bilateral teleoperation systems employing manipulators have been proposed in the literature. On the one hand, systems have been designed which employ commercial hardware and hence generally suffer from low update rates and high delays due to restrictions of commercial manipulator controllers and haptic device controllers. On the other hand, bilateral teleoperation systems designed by research institutions often provide only few degrees of freedom. Our 6DoF bilateral teleoperation system, however, combines the amenities of commercial hardware with a high performance distributed control architecture which enables us to achieve update rates of more than 2 kHz and delays in the range of only 100 ¿s. This paper focuses on the architecture of our system and demonstrates how to achieve this performance using commercial hardware. Moreover, we show why update rates of more than 1 kHz are essential for certain teleoperation tasks. Especially with high approach velocities and stiff environments, high update rates and low delays are key requirements for stability and thus for realistic haptic perception. We present experimental results demonstrating the influence of the update rate on system stability. These results not only highlight the benefits of high update rates but also give hints on how to estimate the update rate necessary to achieve stable teleoperation for a given environment stiffness.

The potential of six-axis acceleration sensors in the field of robotic manipulation applications is quite high and most of it has not been used yet - neither in theoretic literature nor in research experiments. When considering six-joint industrial manipulators with six-axis force/torque and six-axis acceleration sensing, many new possibilities arise: all ten inertial parameters of any object can be identified and objects can be recognized based on these parameters, position control behavior can be improved; non-contact forces can be extracted and force control performance can be improved; visual-servoing methods can use acceleration signals to become more robust. The authors made numerous experiments in the mentioned fields and recognized major weaknesses during the realization of prototypic research setups with six-axis acceleration sensors. These problems regard sensor drift, undesired sensor-internal dependencies as the influence of any distal sensor part, noise, and undesired crosstalk behavior. In order to benefit from acceleration signals, it is important to clearly overcome these problems. This paper analyzes typical systematic errors, characterizes them, and suggests important solution methods for a successful usage of acceleration information.

Regarding manipulators with wrist-mounted force/torque sensors a major issue is the high execution time of force-guided and force-guarded motions compared to purely position-controlled tasks. An important factor that aggravates the reduction of the execution time is the influence of non-contact forces, e.g. inertial forces, centrifugal forces, Coriolis forces, and associated torques, which are exerted onto the sensor by a load attached to it. Considering force-guided or force-guarded motions, these non-contact forces may significantly deteriorate contact detection and force control performance when executing dynamic movements. In addition to these disturbance forces/torques, resets of the force/torque sensor consume execution time. This paper presents an approach to eliminating all non-contact forces and associated torques from force/torque sensor measurements thus enabling pure contact force control. Apart from facilitating pure contact force control, the presented approach renders resets of the force/torque sensor unnecessary. To achieve this aim, the ten inertial parameters (mass, coordinates of the center of mass, and the elements of the inertia matrix) of the load attached to the sensor as well as the force/torque sensor offsets are estimated on-line employing a variant of the recursive instrumental variables method. These parameters are used to calculate the non- contact forces/torques acting upon the sensor. The current non-contact forces/torques and sensor offsets are subtracted from the force/torque measurements thus yielding the contact forces/torques. Experimental results show that both contact detection and force control performance are improved significantly by this approach.

This paper describes a new solution of the simultaneous localization and mapping (SLAM) problem. Instead of building one global consistent map, aimed by the most common SLAM techniques, we compute a set of local metrical maps and fuse them to a graph-like structure resulting in a topological map. Thus, our approach does not require a global metrical map consistency. The main contribution of this paper is an algorithm for closing spatial loops. Loop closing means, that a subset of the edges of the graph representing the topological map forms a cycle. To this end we describe a very efficient enhancement of the well-known RANSAC technique for actively recognizing regions explored by the robot previously. This improvement exploits the theory of the birthday attack whose mathematical background is known from cryptography. A fast sample-based scan matcher is employed to compute the local maps. We derive the covariance of the current robot pose from the sample distribution in order to perform a recognition only when loop closing is very likely. Our approach has been implemented and experimental results show its excellent performance.

The estimation of the ten inertial parameters of rigid loads, which are attached to manipulators, may benefit several robotics applications, e.g.: force control, object recognition, and pose estimation. These applications require sufficiently accurate, robust, and fast estimation of the inertial parameters. Existing approaches, however, do not allow for robust on-line estimation, since they use standard batch least-squares techniques, which ignore noise in the data matrix. The proposed approach, however, estimates the inertial parameters on-line and very fast (approx. 1.5s), while explicitly considering noise in the data matrix by a total least-squares approach. Apart from estimation equations and estimation approaches, the design of estimation trajectories is addressed in this paper. The performance of the proposed estimation approach is compared with the recursive ordinary least-squares (RLS) and the recursive instrumental variables (RIV) method. Experimental results clearly recommend the proposed recursive total least-squares approach (RTLS).

In this paper, we present a new method for sensor fusion in robot assembly. In our approach, model information can be derived automatically from CAD-data. We introduce force torque maps, which are either computed automatically exploiting modern graphical processors or are measured by scanning forces and torques during contact motions. Subsequently, force torque maps are applied as model information during execution of real assembly tasks. Also, computer vision is included by comparing relative poses of features in virtual images with their real relative poses given from measured images. For fusion of these two (or more) different sensors we suggest to use particle filters. Experiments with variations of peg in hole tasks in a real work cell demonstrate our new approach to be very useful for the whole process chain from planning to execution.

This paper proposes an object recognition and gripping pose estimation approach based on on-line estimation of the complete set of inertial parameters, i.e. the mass, the coordinates of the center of mass, and the elements of the inertia matrix, of an object gripped by or attached to a manipulator. A multi-sensor fusion approach combining 6D force/torque, 6D acceleration, 3D angular velocity, and joint angle data to estimate these parameters is presented. In order to facilitate practical implementation, approaches to handling force/torque sensor offsets and to compensating the forces/torques caused by the distal mounting plate of the force/torque sensor and the gripper are incorporated. Regarding the joint angle signals, preprocessing steps to derive the angular velocity, linear acceleration and angular acceleration vector w.r.t. the sensor frame are addressed. The estimation of the complete set of inertial parameters employing the recursive instrumental variables (RIV) method is discussed. The extraction of features that are invariant w.r.t. translation and rotation, i.e. the mass and the principal moments of inertia, as well as a recognition approach based on the Kullback-Leibler divergence are presented. Experimental results show very low errors in the estimates of the inertial parameters, good pose estimation accuracy, and the viability of the recognition approach.

This paper deals with force/position control applied to robotic manipulators. It is well-known that force/position control for manipulators is a challenging task, because of strong nonlinearities exhibited by both the manipulator itself and by its environment. Despite the complexity inherent in manipulation processes, the classic single-loop feedback system with its advantages and disadvantages is still preferred in industrial use for force/position control. As interesting alternative we suggest the MFC-p (model-following control) two-loop control structure, which is based on a simple linear 2nd order model. The theoretical as well as the practical properties of this new control scheme are discussed in detail. It is shown that with our approach it is possible to achieve a much better control performance than that provided by a classic PID system. To prove the superiority of the concept, results with industrial manipulators under force control are shown

This paper focusses on sensor fusion in robotic manipulation: 6D force/torque signals and 6D acceleration signals are used to extract forces and torques caused by inertia. As result, only forces and torques established by environmental contact(s) remain. Beside an improvement of hybrid force/pose control behavior, an additional major benefit is that regular resetting/zeroing of force/torque sensors before free space/contact transitions can be omitted. All essential equations, transformations, and calculations that are required for this 6D fusion approach are derived. To highlight the meaning for practical implementations, numerous experiments with a six-joint Staeubli RX60 industrial manipulator are presented, and the achieved results are discussed

This paper proposes a new way of trajectory generation for industrial manipulators. A real-time algorithm for the interpolation of synchronized and time-optimal manipulator trajectories with arbitrary input values is presented. The method has been developed and implemented for multi-sensor systems, where sensor events can abruptly change desired target positions and trajectory constraints from one control cycle to another (i.e. maximum velocities, accelerations, and jerks). This work mainly presents a simple on-line second-order-trajectory generator and gives an outlook to a third-order-trajectory generator. Both algorithms generate time-optimal position progressions and require computational three steps: A. determination of the degree of freedom that requires the longest execution time, B. synchronization of all degrees of freedom by adapting maximum velocities and accelerations, C. calculation of new output values (position, velocity, and acceleration). Experimental results as well as a description of how to integrate this approach into manipulation control architectures are presented

In this paper, we present work about robot control architecture, assembly planning and task planning for manufacturing robots. The interface between an offline planning unit and control systems is handled with skill primitives. Thus, skill primitives and skill primitive nets are explained in detail. Our long term aim is to combine robot control with task and assembly planning, so that with less human interaction manufacturing costs can be reduced. Even parallel kinematic machines provide enormous opportunities to reduce cycle times and thus the benefit should not be wasted by expensive specialized robot programming. Thus, we give an overview of our system and focus on some aspects to implement such a sophisticated system.

Mason's Task Frame Formalism (TFF) is supposed to deliver robot application programmers an intuitive and powerful programming interface. The open literature provides many theoretic approaches, but almost none of them is practicable. To bring these research results into practice is the major aim of this paper. Sets of manipulation primitives specify compliant motion commands, which let us execute complex robot tasks. We introduce an appropriate notation and focus on all significant TFF values, which have to be applied to the transformations, which are required on the control level. All essential calculations are derived; to highlight the meaning for practical implementations, an example, how to embed this knowledge in control architectures, is given.

In this paper, we suggest skill primitive nets for multi-sensor integration in robot task programming. Each skill primitive is either a hybrid motion or a command interpreted by an external vision system. For the integration of different sensors into one general and comprehensive approach, one has to consider different sensing modalities. With an extension of our previously outlined approach of skill primitive nets, multi-sensor integration is possible. Thereby, a large amount of applications can be realized. We also give a precise specification as XML-interface for skill primitive nets. In this paper advantages are outlined, when using this interface between programming and control. Our skill primitive net approach for multi-sensor integration has been evaluated with industrial applications.

The major purpose of this paper is to combine results of current robot force control research with scientific approaches in compliant motion, which are based on Mason's task frame formalism. The embedding of adaptive implicit hybrid force/pose control in a robot control architecture for compliant motion control is described. By the usage of adaptive force control, the practicability of compliant motion applications is improved. The applied control concept is constituted in a theoretical as well as in a practical manner. To highlight the meaning for practical implementations, experimental results with industrial manipulators under adaptive force control in three degrees of freedom are finally shown.

This paper presents a general approach to specify and execute complex robot tasks considering uncertain environments. Robot tasks are defined by a precise definition of so-called skill primitive nets, which are based on Mason's hybrid force/velocity and position control concept, but it is not limited to force/velocity and position control. Two examples are given to illustrate the formally defined skill primitive nets. We evaluated the controller and the trajectory planner by several experiments. Skill primitives suite very well as interface to robot control systems. The presented hybrid control approach provides a modular, flexible, and robust system; stability is guaranteed, particularly at transitions of two skill primitives. With the interface explained here, the results of compliance motion planning become possible to be examined in real work cells. We have implemented an algorithm to search for mating directions in up to three-dimensional configuration-spaces. Thereby, on one hand we have released compliant motion control concepts and on the other hand we can provide solutions for fine motion and assembly planning. This paper shows, how these two fields can be combined by the general concept of skill primitive nets introduced here, in order to establish a powerful system, which is able to automatically execute prior calculated assembly plans based on CAD-data in uncertain environments.

This paper proposes a new notation for kinematic structures which allows a unified description of serial, parallel, and hybrid robots or articulated machine tools. During the past decades, the Denavit-Hartenberg (DH) parameters have been used widely to describe serial kinematics of robots in science and industry. Till now, such a common notation for parallel manipulators has not yet been accepted. This paper tries to fill this gap by presenting a new notation, which is based on the graph representation known from gear trains. In parallel manipulators, spherical and cardan joints are widely used. In order to describe these kinds of joints, the DH-parameter notation has been extended, so that, to each joint as many joint variables can be assigned as degrees of freedom exist. The notation is not only very useful for design, programming, and simulation of parallel robots; it also can be applied as a convention to refer to parallel or hybrid kinematic structures elsewhere.

This paper describes a new system to automatically generate, evaluate and execute assembly sequences, With the commands generated by the system, robots are enabled to assemble complex products without explicit robot programming. Our approach uses CAD-models, symbolic spatial relations and a robot work cell description as input. The system provides a user-friendly interface to define the goal state of the parts to be assembled. The automatically generated and decomposed assembly plans can be executed by robots by means of a set of predefined skill primitives. For this purpose we classify the robot tasks by analyzing the symbolic spatial relations between the objects to be assembled, the depart-spaces and the necessary tools. The tasks are decomposed into suitable elementary robot operations, the skills, automatically. For guiding the robot during assembly we employ internal and external sensors.

This paper presents a new method to decompose complex sequences of assembly operations into skill primitives. This can be realized by analyzing hyper-arcs of the underlying AND/OR graphs representing automatically generated assembly plans. Features like local depart spaces, symbolic spatial relations, and the necessary tools classify the type of assembly operation (peg in hole, placements, alignments, etc.). Skill primitives are robot movements or commands for grippers and tools. The unified modeling language is used to model the robot tasks and skill primitives. A robot control system uses the skill primitives as input to select the desired control scheme (position, force, or hybrid). In addition to this, we use an algorithm to identify assembly process states considering static friction under uniform gravity to execute skill primitives. This enables a robot to select and modify its motion strategies adequately according to the state of the assembly operation.

We propose in this paper an enhanced fuzzy P+ID controller to improve control performance in both dynamic transient and steady-state periods for mechanical manipulators under uncertainty. The fuzzy P+ID controller adds only two additional parameters to be tuned relative to the original PID controller. One of these parameters is mainly used to reduce a steady-state error. The other is used to speed up the dynamic response. A simulation study and experimental results for a two-link manipulator with uncertainty demonstrate the superior control performance of the proposed fuzzy P+ID controllers.

A new approach to automatically generate polyhedral convex cones from contact graphs for identification of assembly process states is presented. The state of contact between subassemblies is changing as operation proceeds in manipulative tasks such as part-mating. Measured contact forces and the resulting torques provide information about the contact geometry which is used to guide the assembly operation. In this paper, we present a method to calculate contact graphs for polyhedral subassemblies. The nodes of the contact graph represent contact formations of the subassemblies and the arcs represent neighborhoods of contact formations. This information is used to generate polyhedral convex cones for each contact formation considering friction. These polyhedral convex cones describe static equilibrium for each contact formation and are used to identify the contact state by measuring forces and torques during a part-mating.

The execution of automatically generated assembly plans by robots is one of the key technologies of modern and flexible manufacturing. During the execution of assembly sequences the robot comes into contact with the environment. Since there are positional and geometrical uncertainties from object representation, robot motion, and sensor information, compliance is used typically to prevent excessive contact forces. The contact forces and the resulting torques provide information about the contact geometry which is used to guide the assembly operation. We present a fast algorithm to identify assembly process states considering static friction under uniform gravity. This identification of assembly process states enables a robot to select and modify its motion strategies adequately according to the state. We give a symbolic representation of contact states and analyze the static properties of assemblies at each contact state by using the theory of polyhedral convex cones.

This paper proposes a general framework to build 3D object recognition systems from a set of CAD object definitions. Reliable features from object corners, edges and 3D rim curves are introduced; they provide sufficient information to allow identification and pose estimation of CAD designed industrial parts. The statistical properties of the data, caused by noise, is modeled by means of Bayesian nets, representing the relations between objects and observable features. This allows to identify objects by a combination of several features considering the significance of each single feature with respect to the object model base. On this basis robust and powerful 3D CAD based object recognition systems can be built.

Mobile robots for advanced applications have to act in environments which contain moving obstacles. Motions of obstacles (e.g. humans) usually are not precisely predictable, but neither they are completely random. Long-term observation of obstacle behavior may yield knowledge about prevailing motion patterns. This paper presents concepts and results of an experimental system. Using cameras mounted at the ceiling, the workspace is observed. The image data is processed and transformed into a compact statistical representation of motion patterns. Mobile robot motion planning benefits from such additional knowledge: trajectories are rated in terms of collision probability and expected time for reaching the goal; planning yields efficient paths which are adapted to obstacle behavior.

We discuss assembly stability as a constraint for assembly sequence planning. An assembly sequence plan is a high-level plan for constructing a mechanical product from its component parts. The analysis of (sub)assembly stability avoids mating parts which tend to disassemble under the influence of gravity. Furthermore, the number of reorientations which gives a good idea on the value of an assembly sequence depends on the stability of the parts to be assembled. We present algorithms to calculate the set of potentially stable orientations of an (sub)assembly considering static friction under uniform gravity. This set is used for the evaluation of assembly sequences and the minimization of the number of reorientations during plan execution. Therefore, a new evaluation function based on the set of potentially stable assembly orientations is proposed and integrated into the assembly cost evaluation of a high level assembly planning system.

The more the robot knows about its environment, the more efficiently it can operate. This is the basic idea of the system presented in this paper. By employing an external camera-based sensing system, a global model of the robot's environment is maintained and updated continuously. Thus, motion planning and collision avoidance in a dynamic environment (i.e. an environment which contains moving obstacles) can be done efficiently. For many indoor applications this concept is inherently superior compared to mobile robots which act completely autonomously. The paper presents concepts and components of an experimental system: Sensing is done with CCD cameras; the processed data is furnished to different modules regarding robot localization, obstacle detection and motion planning.

The execution of generated assembly plans by robots is one of the key technologies of modern and flexible manufacturing. During the execution of assembly sequences the robot comes into contact with environment. Since there are positional and geometrical uncertainties from object representation, robot motion, and sensory information, compliance is used typically to prevent excessive contact forces. These contact forces provide information about the contact geometry which is used to guide the assembly operation. In this paper we present an algorithm to classify and recognize contact states between subassemblies. We analyze the local depart space (LDS) given by the assembly planning system for the calculation, classification, and recognition of the contact states. The LDS describes the direction of motion for the separation of subassemblies involved in the assembly sequence. We calculate valid configurations of the active subassembly in an uncertainty region and use the theory of polyhedral convex cones for the recognition of contact states. Therefore, force-torque sensor information acquired in the assembly process is interpreted using state classifiers which are formulated by polyhedral convex cones.

Introducing general CAD descriptions in object recognition systems has become a major field of research called CAD based vision (CBV). However, the major problem using free-form object descriptions is how to define recognizable features which can be extracted from sensor data. We propose new methods for the extraction of 3D space carves from CAD models and from range data. Object identification is performed by correlating feature vectors from significant subcurves.

Regrasping must be performed whenever a robot's grasp of an object is not compatible with the pick-and-place operation the robot must perform. This paper presents a new approach to the problem of regrasping for a robot arm equipped with a parallel-jaw end-effector. The method employs an evaluated breadth-first search in the space of compatible regrasp operations taking into account several criteria rating about the grasp and placement quality. We pay particular attention to online computational efficiency. The presented system is the first planning system performing as much of offline computation as possible for solving the regrasp problem efficiently online.

In recent papers we (1996, 1997) have proposed a new path planning approach for mobile robots: statistical motion planning with respect to typical obstacle behavior in order to improve pre-planning in dynamic environments. In this paper, we present our experimental system: in a real environment, cameras observe the workspace in order to detect obstacle motions and to derive statistical data. We have developed new techniques based on stochastic trajectories to model obstacle behavior. Collision probabilities are calculated for polygonal objects moving on piecewise linear trajectories. The statistical data can be applied directly, thus the entire chain from raw sensor data to a stochastic assessment of robot trajectories is closed. Finally, some new work regarding different applications of statistical motion planning is outlined, including road-map approaches for pre-planning, expected time to reach the goal, and reactive behaviors.

In this paper we propose a multiobjective decision making based neural-network model and algorithm for image reconstruction from projections. This model combines the Hopfield's model and multiobjective decision making approach. We develop a weighted sum optimization based neural-network algorithm. The dynamical process of the net is based on minimization of a weighted sum energy function and Euler's iteration, and apply this algorithm to image reconstruction from computer-generated noisy projections and Siemens Somatson DR scanner data, respectively. Reconstructions based on this method is shown to be superior to conventional iterative reconstruction algorithms such as the multiplicate algebraic reconstruction technique (MART) and convolution from the point of view of accuracy of reconstruction. Computer simulation using the multiobjective method shows a significant improvement in image quality and convergence behavior over the conventional algorithms.

We discuss the stability of assemblies consisting of arbitrary configurations of rigid bodies considering Coulomb friction under uniform gravity. We give a method to calculate the set of all stable orientations of an assembly considering the frictional properties. The presented algorithm is a new method for automatically determining all stable orientations of assemblies considering friction. The potential stability criterion describes a necessary but not a sufficient condition for the stability of assemblies. To compute the potential stability of an assembly we consider the forces which arise at contact points of objects touching each other. Constraints which must be satisfied for a set of valid contact forces are expressed by linear inequalities; we use linear programming to determine such valid contact forces. Beside this we describe a method based on the concepts of the enumeration of vertices of polyhedra to compute the set of stable orientations considering Coulomb friction even in the case of degeneracy in the linear program, Furthermore, we discuss the space and time complexity of the underlying algorithm.

As recent research is directed towards flexible and autonomous robot behaviour, a priori knowledge is increasingly replaced by data obtained automatically by external sensors of the robot. Much effort already has been made regarding the integration of different types of sensor data into reliable world representations. Yet, the aspect of guiding robot perception in a systematic manner often has been neglected. Closing this gap, we present a framework addressing the efficient exploration of an unknown three-dimensional environment. A natural approach to this problem is a repeated "planning-sensing-updating"-cycle. In this paper special attention is paid on planning: Based on the current, incomplete world model and on constraints respecting the robot's motions, rating functions in configuration space are defined. When planning sensing actions, configurations with high ratings are chosen. Thus, a fast and efficient exploration of the workspace is guaranteed. Our approach works with miscellaneous robot architectures, including mobile systems as well as manipulators with many degrees of freedom and redundancies. The inverse kinematic solution of the robot has not to be known. The sensors to be used have to provide range measurements, like laser scanners, stereo or sonar systems.

To plan collision-free trajectories for mobile robots among moving obstacles most approaches presume the obstacles' trajectories to be known. This is contradictory to practice where motions generally are known-at best-as average behaviours. Thus, we proposed to use statistical information for path planning. This paper focuses on the derivation of formulae to estimate collision probabilities. By generalizing elementary collision patterns we develop a method to calculate a collision probability field. For any given goal position, it estimates the collision probabilities for paths starting at arbitrary points in the workspace. This provides a powerful tool for path planning in dynamic environments. Using an algorithm similar to path planning based on potential fields, we can calculate paths with (approximately) minimum collision probability.

Compares the integral model suggested by Gautier and Khalil (1988, 1989) for identification of robot dynamics with the differential equations of the robot. The advantage of identification by the integral model is, that no measurement of acceleration is required. Although appealing at a first glance, this approach yields problems in conjunction with the necessary least squares algorithm for parameter estimation. The authors even succeed in getting better results by employing the differential equations, without measurement of acceleration. It is shown, how to achieve reasonable good results with low noise by just derivating the position signals. Finally, the authors' results are illustrated by experiments with a direct driven two link robot.>

This paper proposes a new path planning approach based on statistical information, like the occupancy probability of partly or completely dynamic obstacles and/or the statistical flow of dynamic obstacles. With this new approach it is possible to integrate the expected behavior of moving obstacles into the preplanning of paths without knowing the individual movement of these dynamic obstacles. As the preplanned paths coincide with the average movement pattern in the environment, the runtime planning is simplified. The authors propose an appropriate world model to represent the necessary statistical data, which can be obtained automatically. In addition, a path planning approach, suited for point-shaped and arbitrary-shaped vehicles, is sketched. These paths integrate geometrical detours to fit the planned movements into the statistical movement pattern of the dynamic obstacles.>

After a short overview of a monitoring and navigation system for mobile vehicles (called MONAMOVE) consisting of multiple carriers of different size and an environment monitoring system, the navigation principle using one global navigator for gross motion planning and several local ones for fine motion planning is described. The coordination of multiple vehicles with their individual trajectories computed by the local navigators is managed by the global navigator using automatically computed crossings lying on a regular grid. This fine motion planning approach fulfills constraints like right orientation and continuous curvature along the paths; it bases on the computation of right oriented points which lie inside a driving channel (DC). These points are connected to the best right oriented line sequence without being a stupid following of the right border of the DC. Various extensions of this work are proposed.

Presents a new approach to object recognition and pose estimation based on a 3D robot sensor, which produces range images of a scene along problem specific lines of sight. Recognition is realized as a hypothesis generation/hypothesis verification process. Hypothesis generation is based on a minimal number of predominant and connected object parts in one or more range images determining all 6 degrees of freedom of an objects pose. This set of object parts is transformed into a hypotheses set by simple look-up operations in precalculated hash tables. In the subsequent verification step the authors determine an inspection list for the best recognizable, most distinctive and best visible object parts in this hypotheses set and verify the hypotheses by searching these object parts in existing or new range images.>

A hierarchical navigation system which delivers paths with continuous curvature for rectangularly-modeled vehicles is proposed. After a brief overview of the transport system consisting of omni wheel drive carriers, and on-carrier and off-carrier sensor components, the navigation module of this system is described. The multilevel navigation is based on a new polygonal partitioning of the Cartesian free-space, which allows a fast local modification of the partitions if obstacles are added or deleted. An A*-algorithm whose weight function is automatically adjusted by information on lower system levels delivers a coarse route through the environment (driving channel). Subsequently, a sequence of straight lines is generated, which are oriented along the right border of this channel. On the next level of path planning, curves are fitted into the corners of the line segment sequences to generate paths with continuous curvature. The generated, right-oriented paths are collision-free for rectangular modeled vehicles.>

The paper presents a new concept for the flexible integration of a range sensor into a robot work cell. The sensor itself is based on the Coded Light Approach, which is well- known as a fast and robust active triangulation technique. We propose a depth-eye-in-hand configuration, in which the camera is mounted on the robot's hand while one or more projectors are installed rigidly with respect to the robot's base. The initial calibration of the involved system components is discussed in detail. In addition to this, since the configuration of the optical components changes with the movement of the robot, a fast re- calibration of the optical setup as well as an efficient realization of the triangulation is presented. Detailed experimental results reveal the speed of the range data acquisition as well as the accuracy of the calculated range values.

The integration of a three-dimensional sensor into a robot workcell is described. The sensor is based on the well-known coded light approach (CLA). The CCD camera of the CLA is mounted at the robot's hand and one or more projectors are fixed outside the robot's work space. The principle of the CLA and the calibration of th robot-sensor-system are explained. An application in which the tool of the robot follows an unknown path on an arbitrary surface is presented. For evaluation reasons the deviation from the surface coordinates are simultaneously measured by a gage, allowing a quantitative error analysis.>

It is known that the solution of the inverse kinematics problem for robot arms is mathematically very elaborate. A technique is proposed for automatically generating the closed-form inverse kinematics solution for most industrial robots very efficiently. This is achieved by a careful structural analysis of equations resulting from systems like A/sub i-1//sup -1/ A/sub i-2//sup -1/. . . A/sub 1//sup -1R/T/sub H/=A/sub i/A/sub i+1/. . . A/sub n/ with (i=1, . . ., n), where A/sub i/ and /sup R/T/sub H/ denote the homogeneous transformation matrices between the adjacent robot links and the homogeneous robot hand-base transformation, respectively. Features derived from these equations are described, and it is shown how they can be used advantageously to solve the inverse kinematics problem symbolically. The system is implemented in Prolog and needs a few seconds on a PC for a complete inversion.>

It is shown how structural information from Hough space representations of 2-D images from 3-D scenes can be derived and used to match polyhedra-like objects topologically to 3-D wire-frame models. The matching is performed by an attributed subgraph isomorphism detection algorithm. A subsequent algorithm match can be used to verify the topological match, yielding simultaneously pose information. Simulation results show the method to be a practicable approach for 3-D object recognition.>

A new binary image-enhancement operation is proposed to suppress noise in documents, introduced, e.g., by poor-quality copying. As opposed to approaches previously published, the new technique does not erode characters, symbols, drawing elements, etc, and thus improves the intelligibility of documents. In addition, it is shown that the new preprocessing method reduces significantly the data volume necessary to represent documents in compressed form. Finally, a low-cost hardware implementation is proposed, suitable of being integrated, e.g., in future scanning devices.