Path planning in a changing environment is a challenging task in robotics, as moving objects impose time-dependent constraints. Recent planning methods primarily focus on the spatial aspects, lacking the capability to directly incorporate time constraints. In this paper, we propose a method that leverages a generative model to decompose a complex planning problem into small manageable ones by incrementally generating subgoals given the current planning context. Then, we take into account the temporal information and use learned time estimators based on different statistic distributions to examine and select the generated subgoal candidates. Experiments show that planning from the current robot state to the selected subgoal can satisfy the given time-dependent constraints while being goal-oriented.

In this paper, we present an iterative method to quickly traverse multi-dimensional paths considering jerk constraints. As a first step, we analyze the traversal of each individual path dimension. We derive a range of feasible target accelerations for each intermediate waypoint of a one-dimensional path using a binary search algorithm. Computing a trajectory from waypoint to waypoint leads to the fastest progress on the path when selecting the highest feasible target acceleration. Similarly, it is possible to calculate a trajectory that leads to minimum progress along the path. This insight allows us to control the traversal of a one-dimensional path in such a way that a reference path length of a multi-dimensional path is approximately tracked over time. In order to improve the tracking accuracy, we propose an iterative scheme to adjust the temporal course of the selected reference path length. More precisely, the temporal region causing the largest position deviation is identified and updated at each iteration. In our evaluation, we thoroughly analyze the performance of our method using seven-dimensional reference paths with different path characteristics. We show that our method manages to quickly traverse the reference paths and compare the required traversing time and the resulting path accuracy with other state-of-the-art approaches.

Advancements in Industry 4.0 demand sophisticated solutions for automatic robotic assembly sequence planning (RASP), capable of handling the diversity and complexity of modern manufacturing tasks. One approach to RASP is Assembly-by-Disassembly (AbD). It first searches for a disassembly sequence that is then inverted to obtain an assembly sequence. One of the challenges of AbD, however, is the exponential number of potential assembly sequences for any given assembly. To mitigate this challenge, we propose to transfer knowledge obtained during previous planning attempts. Specifically, we present an approach that combines Monte Carlo Tree Search (MCTS) with deep Q-learning to optimize the total length of robotic assembly paths. We use a graph-based representation of disassembly states in combination with a graph neural network to learn the Q-function. We further discuss a principled approach to generate 3D assemblies out of aluminium profiles that a single robot manipulator can assemble. With this approach, we generated two datasets consisting of 14 assemblies with 21 removable parts and 7 assemblies with 30 removable parts. Using leave-one-out cross-validation, we were able to demonstrate how our approach outperformed an unmodified MCTS. Moreover, we successfully transferred knowledge between datasets.

In this paper, we present an approach for quantifying the propagated uncertainty of robot systems in an online and data-driven manner. Especially in Human-Robot Collaboration, keeping track of the safety compliance during run time is essential: Misclassifying dangerous situations as safe might result in severe accidents. According to official regulations (e.g., ISO standards), safety in industrial robot applications depends on critical parameters, such as the distance and relative velocity between humans and robots. However, safety can only be assured given a measure for the reliability of these parameters. While different risk detection and mitigation approaches exist in literature, a measure that can be used to evaluate safety limits online, and succinctly implies whether a situation is safe or dangerous, is missing to date. Motivated by this, we introduce a generalizable method for calculating the propagated measurement uncertainty of arbitrary parameters, that captures the accumulated uncertainty originating from sensory devices and environmental disturbances of the system. To show that our approach delivers correct results, we perform validation experiments in simulation. In addition, we employ our method in two real-world settings and demonstrate how quantifying the propagated uncertainty of critical parameters facilitates assessing safety online in Human-Robot Collaboration.

Industry 4.0 facilitates a high number of product variants, posing significant challenges for modern manufacturing. One of them is the automatic creation of assembly sequences. This can be achieved with the assembly-by-disassembly (AbD) approach, which is currently highly inefficient. We aim at speeding up AbD by leveraging deep learning. AbD relies on iteratively testing parts for removal, which makes the order in which parts are tested highly relevant for its run-time. We optimize this order by training a graph neural network (GNN) based on the shape of parts and the shape of local part connections. For each part, it predicts a removability probability. We use these probabilities to optimize the order in which parts are tested for removal. This reduces the number of parts tested by approximately 64%-90%, depending on the tested product. Further improvements are achieved by combining our approach with bookkeeping, another approach for speeding up AbD. Finally, we separately analyze the impact of the parts and their connections on the removability probabilities predicted by the GNN. We found that most of the important information regarding a part's removability can be derived from its connections alone.

Safety critical systems are typically subjected to hazard analysis before commissioning to identify and analyse potentially hazardous system states that may arise during operation. Currently, hazard analysis is mainly based on human reasoning, past experiences, and simple tools such as checklists and spreadsheets. Increasing system complexity makes such approaches decreasingly suitable. Furthermore, testing-based hazard analysis is often not suitable due to high costs or dangers of physical faults. A remedy for this are model-based hazard analysis methods, which either rely on formal models or on simulation models, each with their own benefits and drawbacks. This paper proposes a two-layer approach that combines the benefits of exhaustive analysis using formal methods with detailed analysis using simulation. Unsafe behaviours that lead to unsafe states are first synthesised from a formal model of the system using Supervisory Control Theory. The result is then input to the simulation where detailed analyses using domain-specific risk metrics are performed. Though the presented approach is generally applicable, this paper demonstrates the benefits of the approach on an industrial human-robot collaboration system.

In this paper, we present a method to engage measurement uncertainties with the probabilistic robustness to one system uncertainty measure. Providing a metric indicating the potential occurrence of dangerous situations is highly essential for safety-critical robot applications. Due to the difficulty of finding a quantifiable, unambiguous representation however, such a metric has not been derived to date. In case of sensory devices, measurement uncertainties are usually provided by manufacturer specifications. Apart from that, several contributions demonstrate that the accuracy of neural networks is verifiable via the robustness. However, state-of-the-art literature is mainly concerned with theoretical investigations such that scarce attention has been devoted to the transfer of the robustness to real-world applications. To fill this gap, we show how the probabilistic robustness can be made useful for evaluating quantitative safety limits. Our key idea is to exploit the analogy between measurement uncertainties and the probabilistic robustness: While measurement uncertainties reflect possible shifts due to technical limitations, the robustness refers to the tolerated amount of distortions in the input data for an unaltered output. Inspired by this analogy, we combine both measures to quantify the system uncertainty online. We validate our method in different settings under real-world conditions. Our findings exemplify that incorporating the novel uncertainty metric effectively prevents the rate of dangerous situations in Human-Robot Collaboration.

Folding garments reliably and efficiently is a long standing challenge in robotic manipulation due to the complex dynamics and high dimensional configuration space of garments. An intuitive approach is to initially manipulate the garment to a canonical smooth configuration before folding. In this work, we develop SpeedFolding, a reliable and efficient bimanual system, which given user-defined instructions as folding lines, manipulates an initially crumpled garment to (1) a smoothed and (2) a folded configuration. Our primary contribution is a novel neural network architecture that is able to predict pairs of gripper poses to parameterize a diverse set of bimanual action primitives. After learning from 4300 human- annotated and self-supervised actions, the robot is able to fold garments from a random initial configuration in under 120 s on average with a success rate of 93 %. Real-world experiments show that the system is able to generalize to unseen garments of different color, shape, and stiffness. While prior work achieved 3–6 Folds Per Hour (FPH), SpeedFolding achieves 30–40 FPH. See https://pantor.github.io/speedfolding for code, videos, and datasets.

With the goal of efficiently computing collisionfree robot motion trajectories in dynamically changing environments, we present results of a novel method for Heuristics Informed Robot Online Path Planning (HIRO). Dividing robot environments into static and dynamic elements, we use the static part for initializing a deterministic roadmap, which provides a lower bound of the final path cost as informed heuristics for fast path-finding. These heuristics guide a search tree to explore the roadmap during runtime. The search tree examines the edges using a fuzzy collision checking concerning the dynamic environment. Finally, the heuristics tree exploits knowledge fed back from the fuzzy collision checking module and updates the lower bound for the path cost. As we demonstrate in real-world experiments, the closed-loop formed by these three components significantly accelerates the planning procedure. An additional backtracking step ensures the feasibility of the resulting paths. Experiments in simulation and the real world show that HIRO can find collisionfree paths considerably faster than baseline methods with and without prior knowledge of the environment.

In this paper, we present a learning-based approach that allows a robot to quickly follow a reference path defined in joint space without exceeding limits on the position, velocity, acceleration and jerk of each robot joint. Contrary to offline methods for time-optimal path parameterization, the reference path can be changed during motion execution. In addition, our approach can utilize sensory feedback, for instance, to follow a reference path with a bipedal robot without losing balance. With our method, the robot is controlled by a neural network that is trained via reinforcement learning using data generated by a physics simulator. From a mathematical perspective, the problem of tracking a reference path in a time-optimized manner is formalized as a Markov decision process. Each state includes a fixed number of waypoints specifying the next part of the reference path. The action space is designed in such a way that all resulting motions comply with the specified kinematic joint limits. The reward function finally reflects the trade-off between the execution time, the deviation from the desired reference path and optional additional objectives like balancing. We evaluate our approach with and without additional objectives and show that time-optimized path tracking can be successfully learned for both industrial and humanoid robots. In addition, we demonstrate that networks trained in simulation can be successfully transferred to a real robot.

Robot learning is often simplified to planar manipulation due to its data consumption. Then, a common approach is to use a fully-convolutional neural network (FCNN) to estimate the reward of grasp primitives. In this work, we extend this approach by parametrizing the two remaining, lateral degrees of freedom (DoFs) of the primitives. We apply this principle to the task of 6 DoF bin picking: We introduce a model-based controller to calculate angles that avoid collisions, maximize the grasp quality while keeping the uncertainty small. As the controller is integrated into the training, our hybrid approach is able to learn about and exploit the model-based controller. After real-world training of 27 000 grasp attempts, the robot is able to grasp known objects with a success rate of over 92 % in dense clutter. Grasp inference takes less than 50 ms. In further real-world experiments, we evaluate grasp rates in a range of scenarios including its ability to generalize to unknown objects. We show that the system is able to avoid collisions, enabling grasps that would not be possible without primitive adaption.

During the planning phase of industrial robot workplaces, hazard analyses are required so that potential hazards for human workers can be identified and appropriate safety measures can be implemented. Existing hazard analysis methods use human reasoning, checklists and/or abstract system models, which limit the level of detail. We propose a new approach that frames hazard analysis as a search problem in a dynamic simulation environment. Our goal is to identify workplace hazards by searching for simulation sequences that result in hazardous situations. We solve this search problem by placing virtual humans into workplace simulation models. These virtual humans act in an adversarial manner: They learn to provoke unsafe situations, and thereby uncover workplace hazards. Although this approach cannot replace a thorough hazard analysis, it can help uncover hazards that otherwise may have been overlooked, especially in early development stages. Thus, it helps to prevent costly re-designs at later development stages. For validation, we performed hazard analyses in six different example scenarios that reflect typical industrial robot workplaces.

We present an approach to learn fast and dynamic robot motions without exceeding limits on the position θ, velocity $\dot \theta $ , acceleration $\ddot \theta $ and jerk $\dddot \theta $ of each robot joint. Movements are generated by mapping the predictions of a neural network to safely executable joint accelerations. The neural network is invoked periodically and trained via reinforcement learning. Our main contribution is an analytical procedure for calculating safe joint accelerations, which considers the prediction frequency fNof the neural network. As a result, the frequency fNcan be freely chosen and treated as a hyperparameter. We show that our approach is preferable to penalizing constraint violations as it provides explicit guarantees and does not distort the desired optimization target. In addition, the influence of the selected prediction frequency on the learning performance and on the computing effort is highlighted by various experiments.

Robot learning of real-world manipulation tasks remains challenging and time consuming, even though actions are often simplified by single-step manipulation primitives. In order to compensate the removed time dependency, we additionally learn an image-to-image transition model that is able to predict a next state including its uncertainty. We apply this approach to bin picking, the task of emptying a bin using grasping as well as pre-grasping manipulation as fast as possible. The transition model is trained with up to 42 000 pairs of real-world images before and after a manipulation action. Our approach enables two important skills: First, for applications with flange-mounted cameras, picks per hours (PPH) can be increased by around 15 % by skipping image measurements. Second, we use the model to plan action sequences ahead of time and optimize time-dependent rewards, e.g. to minimize the number of actions required to empty the bin. We evaluate both improvements with real-robot experiments and achieve over 700 PPH in the YCB Box and Blocks Test.

We present TrueÆdapt, a model-free method to learn online adaptations of robot trajectories based on their effects on the environment. Given sensory feedback and future waypoints of the original trajectory, a neural network is trained to predict joint accelerations at regular intervals. The adapted trajectory is generated by linear interpolation of the predicted accelerations, leading to continuously differentiable joint velocities and positions. Bounded jerks, accelerations and velocities are guaranteed by calculating the range of valid accelerations at each decision step and clipping the network's output accordingly. A deviation penalty during the training process causes the adapted trajectory to follow the original one. Smooth movements are encouraged by penalizing high accelerations and jerks. We evaluate our approach by training a simulated KUKA iiwa robot to balance a ball on a plate while moving and demonstrate that the balancing policy can be directly transferred to a real robot.

Hand guidance of robots has proven to be a useful tool both for programming trajectories and in kinesthetic teaching. However hand guidance is usually relegated to robots possessing joint-torque sensors (JTS). Here we propose to extend hand guidance to robots lacking those sensors through the use of an Augmented Reality (AR) device, namely Microsoft's Hololens. Augmented reality devices have been envisioned as a helpful addition to ease both robot programming and increase situational awareness of humans working in close proximity to robots. We reference the robot by using a registration algorithm to match a robot model to the spatial mesh. The in-built hand tracking capabilities are then used to calculate the position of the hands relative to the robot. By decomposing the hand movements into orthogonal rotations we achieve a completely sensorless hand guidance without any need to build a dynamic model of the robot itself. We did the first tests our approach on a commonly used industrial manipulator, the KUKA KR-5.

Given the task of learning robotic grasping solely based on a depth camera input and gripper force feedback, we derive a learning algorithm from an applied point of view to significantly reduce the amount of required training data. Major improvements in time and data efficiency are achieved by: Firstly, we exploit the geometric consistency between the undistorted depth images and the task space. Using a relative small, fully-convolutional neural network, we predict grasp and gripper parameters with great advantages in training as well as inference performance. Secondly, motivated by the small random grasp success rate of around 3 %, the grasp space was explored in a systematic manner. The final system was learned with 23 000 grasp attempts in around 60 h, improving current solutions by an order of magnitude. For typical bin picking scenarios, we measured a grasp success rate of (96.6 ± 1.0) %. Further experiments showed that the system is able to generalize and transfer knowledge to novel objects and environments.

This paper introduces the KUKA Robot Learning Lab at KIT - a remotely accessible robotics testbed. The motivation behind the laboratory is to make state-of-the-art industrial lightweight robots more accessible for education and research. Such expensive hardware is usually not available to students or less privileged researchers to conduct experiments. This paper describes the design and operation of the Robot Learning Lab and discusses the challenges that one faces when making experimental robot cells remotely accessible. Especially safety and security must be ensured, while giving users as much freedom as possible when developing programs to control the robots. A fully automated and efficient processing pipeline for experiments makes the lab suitable for a large amount of users and allows a high usage rate of the robots.

Robotic grasping in cluttered environments is often infeasible due to obstacles preventing possible grasps. Then, pre-grasping manipulation like shifting or pushing an object becomes necessary. We developed an algorithm that can learn, in addition to grasping, to shift objects in such a way that their grasp probability increases. Our research contribution is threefold: First, we present an algorithm for learning the optimal pose of manipulation primitives like clamping or shifting. Second, we learn non-prehensible actions that explicitly increase the grasping probability. Making one skill (shifting) directly dependent on another (grasping) removes the need of sparse rewards, leading to more data-efficient learning. Third, we apply a real-world solution to the industrial task of bin picking, resulting in the ability to empty bins completely. The system is trained in a self-supervised manner with around 25 000 grasp and 2500 shift actions. Our robot is able to grasp and file objects with 274±3 picks per hour. Furthermore, we demonstrate the system's ability to generalize to novel objects.

Machining applications using robots are still not common in industrial settings. Reasons are the unintuitive programming concepts which typically require expert knowledge and the inflexibility regarding small alterations of the workpieces. We present a prototypical solution for an intuitive and flexible robotic machining concept for unmodeled work pieces. For this we use a high resolution laser scanner to record very dense point clouds. Algorithms to detect linear edges with obtuse angled corners, linear edges with acute angled corners, linear inner edges and circular edges were developed, demonstrated and validated. To accurately execute generated trajectories in practice, an algorithm to directly calibrate the transformation between the sensor and the milling tool was developed. For the algorithms and the calibration process a repeatability tolerance of 0.2 mm is achieved.

Hand guidance emerged from the safety requirements for collaborative robots, namely possessing joint-torque sensors. Since then it has proven to be a powerful tool for easy trajectory programming, allowing lay-users to reprogram robots intuitively. Going beyond, a robot can learn tasks by user demonstrations through kinesthetic teaching, enabling robots to generalise tasks and further reducing the need for reprogramming. However, hand guidance is still mostly relegated to collaborative robots. Here we propose a method that does not require any sensors on the robot or in the robot cell, by using a Microsoft HoloLens augmented reality head mounted display. We reference the robot using a registration algorithm to match the robot model to the spatial mesh. The in-built hand tracking and localisation capabilities are then used to calculate the position of the hands relative to the robot. By decomposing the hand movements into orthogonal rotations and propagating it down through the kinematic chain, we achieve a generalised hand guidance without the need to build a dynamic model of the robot itself. We tested our approach on a commonly used industrial manipulator, the KUKA KR-5.

A concept consisting of a new configurable vacuum gripper system and a corresponding method for determining optimal grasp configurations solely based on 3D vision is introduced. The robot system consists of a dynamically configurable vacuum gripper, a visual sensor, and a robot arm that are used in combination with a new grasp planner to robustly grasp unknown objects in arbitrary positions. For this purpose, formalized aspects of selecting contact surfaces for arbitrary suction cups are described; the concept involves visual detection of the objects, segmentation, iterative grasp planning, and action execution. The approach allows for a fast and efficient, yet precise execution of grasps. The core idea is a two-step 3D data acquisition approach and grasp point computation that takes advantage of the fact that the suction cups of the gripper can all be aligned axis-parallel. Therefore, an adequate sensor-based surface acquisition is done from a single viewpoint with respect to the gripper. Results of realworld experiments show that the proposed concept is suitable for a wide range of different and unknown objects in our setup.

This paper presents the Dexterity Network (Dex-Net) 1.0, a dataset of 3D object models and a sampling-based planning algorithm to explore how Cloud Robotics can be used for robust grasp planning. The algorithm uses a Multi- Armed Bandit model with correlated rewards to leverage prior grasps and 3D object models in a growing dataset that currently includes over 10,000 unique 3D object models and 2.5 million parallel-jaw grasps. Each grasp includes an estimate of the probability of force closure under uncertainty in object and gripper pose and friction. Dex-Net 1.0 uses Multi-View Convolutional Neural Networks (MV-CNNs), a new deep learning method for 3D object classification, to provide a similarity metric between objects, and the Google Cloud Platform to simultaneously run up to 1,500 virtual cores, reducing experiment runtime by up to three orders of magnitude. Experiments suggest that correlated bandit techniques can use a cloud-based network of object models to significantly reduce the number of samples required for robust grasp planning. We report on system sensitivity to variations in similarity metrics and in uncertainty in pose and friction. Code and updated information is available at http://berkeleyautomation.github.io/dex-net/.

Generating robot motion trajectories instantaneously in the moment unforeseen sensor events happen is very essential for many real-world robot applications. Using a previous work on online trajectory generation as a basis, this paper proposes an alternative approach that also considers dynamic models. The former class of algorithms does not take into account dynamically changing acceleration capabilities based on maximum actuator forces/torques. This paper extends target velocity-based algorithms of the previous approach by taking into consideration the entire system dynamics when generating trajectories online within one control cycle (typically 1 ms or less). The extension includes the acceleration capabilities of a robot at every discrete time step assuming constant values for the maximum actuator forces/torques, thus allowing the generation of adaptive trajectory profiles during the motion of the robot. Several real-world experimental results using a seven-degree-of-freedom lightweight robot arm underline the relevance of this extension.

All mammals but humans use whiskers in order to rapidly acquire information about objects in the vicinity of the head. Collisions of the head and objects can be avoided as the contact point is moved from the body surface to the whiskers. Such a behavior is also highly desirable during many robot tasks such as for human-robot interaction. Using novel capacitive proximity sensors, robots sense when they approach a human (or an object) and react before they actually collide with it. We propose a sensor and control concept that mimics the behavior of whiskers by means of capacitive sensors. Major advantages are the absence of physical whiskers, the absence of blind spots and a very short response time. The sensors are flexible and thin so that they feature skin-like properties and can be attached to various robotic link and joint shapes. In comparison to capacitive proximity sensors, the proposed virtual whiskers offer better sensitivity towards small conductive as well as non conductive objects. Equipped with the new proximity sensors, a seven-joint robot for humanrobot interaction tasks shows the efficiency and responsiveness of our concept.

In this paper a real-time collision avoidance approach is presented for safe human-robot coexistence. The main contribution is a fast method to evaluate distances between the robot and possibly moving obstacles (including humans), based on the concept of depth space. The distances are used to generate repulsive vectors that are used to control the robot while executing a generic motion task. The repulsive vectors can also take advantage of an estimation of the obstacle velocity. In order to preserve the execution of a Cartesian task with a redundant manipulator, a simple collision avoidance algorithm has been implemented where different reaction behaviors are set up for the end-effector and for other control points along the robot structure. The complete collision avoidance framework, from perception of the environment to joint-level robot control, is presented for a 7-dof KUKA Light-Weight-Robot IV using the Microsoft Kinect sensor. Experimental results are reported for dynamic environments with obstacles and a human.

A concept of on-line trajectory generation for robot motion control systems enabling instantaneous reactions to unforeseen sensor events was introduced in a former publication. This previously proposed class of algorithms requires constant kinematic motion constraints, and this paper extends the approach by the usage of time-variant motion constraints, such that low-level trajectory parameters can now abruptly be changed, and the system can react instantaneously within the same control cycle (typically one millisecond or less). This feature is important for instantaneous switchings between state spaces and reference frames at sensor-dependent instants of time, and for the usage of the algorithm as a control submodule in a hybrid switched robot motion control system. Real-world experimental results of two sample use-cases highlight the practical relevance of this extension.

Common visual servoing methods use image features to define a signal error in the feedback loops of robot motion controllers. This paper suggests a new visual servo control scheme that uses an on-line trajectory generator as an intermediate layer between image processing algorithms and robot motion controllers. The motion generation algorithm is capable of computing an entire trajectory from an arbitrary initial state of motion within one servo control cycle (typically one millisecond or less). This algorithm is fed with desired pose and velocity signals that are generated by an image processing algorithm. The advantages of this new architecture are: (a) jerk-limited and continuous motions are guaranteed independently of image processing signals, (b) kinematic motion constraints as well as physical and/or artificial workspace limits can be directly considered, and (c) the system can instantaneously and safely react to sensor failures (e.g., if cameras are covered or image processing fails). Real-world experimental results using a seven-joint robot arm are presented to underline the relevance for the field of robust sensor-guided robot motion control.

This paper introduces the Reflexxes Motion Libraries and describes, how they open doors for next generation robot motion controllers. When robots become capable to perform sensor-guided and sensor-guarded motions, there is no predefined path anymore, and motions have to be calculated online, that is, during the motion. The Reflexxes Motion Libraries calculate jerk-limited motions within one control cycle only (typically 1 ms or less). This way, robots can instantaneously react to unforeseen sensor events, which opens the door to a huge number of new robot capabilities and fundamentally new motion control features. For instance: unforeseen switchings of coordinate frames, unforeseen switchings of control state spaces, deterministic and instantaneous reactions to sensor signals, safe and stable reactions to sensor failures, simple visual servo control, and stable switched-system control. All these features are important for the execution of sensor-based robot motions and to realize new applications as will be outlined in this paper.

A concept of online trajectory generation for robot motion control systems that enables instantaneous reactions to unforeseen sensor events was introduced in a former publication. This concept is now extended with the important feature of homothety. Homothetic trajectories are 1-D straight lines in a multidimensional space and are relevant for all straight-line motion operations in robotics. This paper clarifies 1) how online concepts can be used to generate homothetic trajectories and 2) how we can instantaneously react to (sensor) events with homothetic trajectories. To underline the practical relevance, real-world experimental results with a seven-degree-of-freedom (DOF) robot arm are shown.

This contribution introduces a generic framework for sensor-based robot motion control. The key contribution is the introduction of an adaptive selection matrix for sensor-based hybrid switched-system control. The overall control system consists of multiple sensors and open- and closed-loop controllers, in-between which the adaptive selection matrix can switch discretely in order to supply command variables for low-level controllers of robotic manipulators. How control signals are chosen, is specified by Manipulation Primitives, which constitute the interface to higher-level programming. This programming paradigm is briefly specified in order to be able to define and execute sensor-guided and sensor-guarded motion commands simultaneously. The resulting control system is freely adaptable depending on the sensor and control requirements of the desired system and/or application.

This paper suggests the idea of a universal method for stabilizing discrete-time hybrid switched-control systems of robot manipulators. The core of this idea is based on an on-line trajectory generation algorithm that is able to generate continuous command variables from any arbitrary state of motion. We define a measurable criterion to on-line detect an instability or a potential instability of the plant, and right after this criterion is fulfilled, we switch to the on-line trajectory generator that acts as an open-loop pose control submodule in the switched-system. The on-line trajectory generation algorithm guides the system under consideration of kinematic motion constraints to a desired target state of motion that can be specified beforehand (e.g., zero-velocity in a pre-defined position). Systems with one and more degrees of freedom are regarded in this paper; finally, real-world experimental results achieved with a six-joint industrial manipulator are presented in order to demonstrate the potential and the high practical relevance of this concept.

This paper introduces a new method for motion-trajectory generation of mechanical systems with multiple degrees of freedom (DOFs). The key feature of this new concept is that motion trajectories are generated online, i.e., within every control cycle, typically every millisecond. This enables systems to react instantaneously to unforeseen and unpredictable (sensor) events at any time instant and in any state of motion. As a consequence, (multi)sensor integration in robotics, in particular the development of control systems enabling sensor-guided and sensor-guarded motions, becomes greatly simplified. We introduce a class of online trajectory-generation algorithms and present the mathematical basics of this new approach. The algorithms presented here consist of three steps: calculation of the minimum synchronization time for all DOFs, synchronization of all DOFs, and calculation of output values. The theory is followed by real-world experimental results indicating new possibilities in robot-motion control.

The potential of six-axis acceleration sensors in the field of robotic manipulation applications is quite high and most of it has not been used yet - neither in theoretic literature nor in research experiments. When considering six-joint industrial manipulators with six-axis force/torque and six-axis acceleration sensing, many new possibilities arise: all ten inertial parameters of any object can be identified and objects can be recognized based on these parameters, position control behavior can be improved; non-contact forces can be extracted and force control performance can be improved; visual-servoing methods can use acceleration signals to become more robust. The authors made numerous experiments in the mentioned fields and recognized major weaknesses during the realization of prototypic research setups with six-axis acceleration sensors. These problems regard sensor drift, undesired sensor-internal dependencies as the influence of any distal sensor part, noise, and undesired crosstalk behavior. In order to benefit from acceleration signals, it is important to clearly overcome these problems. This paper analyzes typical systematic errors, characterizes them, and suggests important solution methods for a successful usage of acceleration information.

Regarding manipulators with wrist-mounted force/torque sensors a major issue is the high execution time of force-guided and force-guarded motions compared to purely position-controlled tasks. An important factor that aggravates the reduction of the execution time is the influence of non-contact forces, e.g. inertial forces, centrifugal forces, Coriolis forces, and associated torques, which are exerted onto the sensor by a load attached to it. Considering force-guided or force-guarded motions, these non-contact forces may significantly deteriorate contact detection and force control performance when executing dynamic movements. In addition to these disturbance forces/torques, resets of the force/torque sensor consume execution time. This paper presents an approach to eliminating all non-contact forces and associated torques from force/torque sensor measurements thus enabling pure contact force control. Apart from facilitating pure contact force control, the presented approach renders resets of the force/torque sensor unnecessary. To achieve this aim, the ten inertial parameters (mass, coordinates of the center of mass, and the elements of the inertia matrix) of the load attached to the sensor as well as the force/torque sensor offsets are estimated on-line employing a variant of the recursive instrumental variables method. These parameters are used to calculate the non- contact forces/torques acting upon the sensor. The current non-contact forces/torques and sensor offsets are subtracted from the force/torque measurements thus yielding the contact forces/torques. Experimental results show that both contact detection and force control performance are improved significantly by this approach.

The estimation of the ten inertial parameters of rigid loads, which are attached to manipulators, may benefit several robotics applications, e.g.: force control, object recognition, and pose estimation. These applications require sufficiently accurate, robust, and fast estimation of the inertial parameters. Existing approaches, however, do not allow for robust on-line estimation, since they use standard batch least-squares techniques, which ignore noise in the data matrix. The proposed approach, however, estimates the inertial parameters on-line and very fast (approx. 1.5s), while explicitly considering noise in the data matrix by a total least-squares approach. Apart from estimation equations and estimation approaches, the design of estimation trajectories is addressed in this paper. The performance of the proposed estimation approach is compared with the recursive ordinary least-squares (RLS) and the recursive instrumental variables (RIV) method. Experimental results clearly recommend the proposed recursive total least-squares approach (RTLS).

This paper proposes an object recognition and gripping pose estimation approach based on on-line estimation of the complete set of inertial parameters, i.e. the mass, the coordinates of the center of mass, and the elements of the inertia matrix, of an object gripped by or attached to a manipulator. A multi-sensor fusion approach combining 6D force/torque, 6D acceleration, 3D angular velocity, and joint angle data to estimate these parameters is presented. In order to facilitate practical implementation, approaches to handling force/torque sensor offsets and to compensating the forces/torques caused by the distal mounting plate of the force/torque sensor and the gripper are incorporated. Regarding the joint angle signals, preprocessing steps to derive the angular velocity, linear acceleration and angular acceleration vector w.r.t. the sensor frame are addressed. The estimation of the complete set of inertial parameters employing the recursive instrumental variables (RIV) method is discussed. The extraction of features that are invariant w.r.t. translation and rotation, i.e. the mass and the principal moments of inertia, as well as a recognition approach based on the Kullback-Leibler divergence are presented. Experimental results show very low errors in the estimates of the inertial parameters, good pose estimation accuracy, and the viability of the recognition approach.

This paper deals with force/position control applied to robotic manipulators. It is well-known that force/position control for manipulators is a challenging task, because of strong nonlinearities exhibited by both the manipulator itself and by its environment. Despite the complexity inherent in manipulation processes, the classic single-loop feedback system with its advantages and disadvantages is still preferred in industrial use for force/position control. As interesting alternative we suggest the MFC-p (model-following control) two-loop control structure, which is based on a simple linear 2nd order model. The theoretical as well as the practical properties of this new control scheme are discussed in detail. It is shown that with our approach it is possible to achieve a much better control performance than that provided by a classic PID system. To prove the superiority of the concept, results with industrial manipulators under force control are shown

This paper focusses on sensor fusion in robotic manipulation: 6D force/torque signals and 6D acceleration signals are used to extract forces and torques caused by inertia. As result, only forces and torques established by environmental contact(s) remain. Beside an improvement of hybrid force/pose control behavior, an additional major benefit is that regular resetting/zeroing of force/torque sensors before free space/contact transitions can be omitted. All essential equations, transformations, and calculations that are required for this 6D fusion approach are derived. To highlight the meaning for practical implementations, numerous experiments with a six-joint Staeubli RX60 industrial manipulator are presented, and the achieved results are discussed

This paper proposes a new way of trajectory generation for industrial manipulators. A real-time algorithm for the interpolation of synchronized and time-optimal manipulator trajectories with arbitrary input values is presented. The method has been developed and implemented for multi-sensor systems, where sensor events can abruptly change desired target positions and trajectory constraints from one control cycle to another (i.e. maximum velocities, accelerations, and jerks). This work mainly presents a simple on-line second-order-trajectory generator and gives an outlook to a third-order-trajectory generator. Both algorithms generate time-optimal position progressions and require computational three steps: A. determination of the degree of freedom that requires the longest execution time, B. synchronization of all degrees of freedom by adapting maximum velocities and accelerations, C. calculation of new output values (position, velocity, and acceleration). Experimental results as well as a description of how to integrate this approach into manipulation control architectures are presented

This work recommends an architecture and its fundamental components for motion planning for mobile robots in dynamic environments. An adaptive behavior to typical motion patterns of people is essential for robots to be accepted in crowded environments. This adaptation is achieved by a global and distributed sensing system to detect obstacle motions and model motion behavior. This article focuses on the detection, the modeling and the prediction of obstacle motion behavior. The model consists of a dynamic Bayesian network. The topology of the network is a geometric graph

Mason's Task Frame Formalism (TFF) is supposed to deliver robot application programmers an intuitive and powerful programming interface. The open literature provides many theoretic approaches, but almost none of them is practicable. To bring these research results into practice is the major aim of this paper. Sets of manipulation primitives specify compliant motion commands, which let us execute complex robot tasks. We introduce an appropriate notation and focus on all significant TFF values, which have to be applied to the transformations, which are required on the control level. All essential calculations are derived; to highlight the meaning for practical implementations, an example, how to embed this knowledge in control architectures, is given.

The major purpose of this paper is to combine results of current robot force control research with scientific approaches in compliant motion, which are based on Mason's task frame formalism. The embedding of adaptive implicit hybrid force/pose control in a robot control architecture for compliant motion control is described. By the usage of adaptive force control, the practicability of compliant motion applications is improved. The applied control concept is constituted in a theoretical as well as in a practical manner. To highlight the meaning for practical implementations, experimental results with industrial manipulators under adaptive force control in three degrees of freedom are finally shown.

This paper presents a general approach to specify and execute complex robot tasks considering uncertain environments. Robot tasks are defined by a precise definition of so-called skill primitive nets, which are based on Mason's hybrid force/velocity and position control concept, but it is not limited to force/velocity and position control. Two examples are given to illustrate the formally defined skill primitive nets. We evaluated the controller and the trajectory planner by several experiments. Skill primitives suite very well as interface to robot control systems. The presented hybrid control approach provides a modular, flexible, and robust system; stability is guaranteed, particularly at transitions of two skill primitives. With the interface explained here, the results of compliance motion planning become possible to be examined in real work cells. We have implemented an algorithm to search for mating directions in up to three-dimensional configuration-spaces. Thereby, on one hand we have released compliant motion control concepts and on the other hand we can provide solutions for fine motion and assembly planning. This paper shows, how these two fields can be combined by the general concept of skill primitive nets introduced here, in order to establish a powerful system, which is able to automatically execute prior calculated assembly plans based on CAD-data in uncertain environments.