Trajectory segmentation refers to dividing a trajectory into meaningful consecutive sub-trajectories. This paper focuses on trajectory segmentation for 3D rigid-body motions. Most segmentation approaches in the literature represent the body’s trajectory as a point trajectory, considering only its translation and neglecting its rotation. We propose a novel trajectory representation for rigid-body motions that incorporates both translation and rotation, and additionally exhibits several invariant properties. This representation consists of a geometric progress rate and a third-order trajectory-shape descriptor. Concepts from screw theory were used to make this representation time-invariant and also invariant to the choice of body reference point. This new representation is validated for a self-supervised segmentation approach, both in simulation and using real recordings of human-demonstrated pouring motions. The results show a more robust detection of consecutive sub-motions with distinct features and a more consistent segmentation compared to conventional representations. We believe that other existing segmentation methods may benefit from using this trajectory representation to improve their invariance.

Trajectory optimization is a powerful tool for robot motion planning and control. State-of-the-art general-purpose nonlinear programming solvers are versatile, handle constraints effectively and provide a high numerical robustness, but they are slow because they do not fully exploit the optimal control problem structure at hand. Existing structure-exploiting solvers are fast, but they often lack techniques to deal with nonlinearity or rely on penalty methods to enforce (equality or inequality) path constraints. This work presents FATROP: a trajectory optimization solver that is fast and benefits from the salient features of general-purpose nonlinear optimization solvers. The speed-up is mainly achieved through the integration of a specialized linear solver, based on a Riccati recursion that is generalized to also support stagewise equality constraints. To demonstrate the algorithm's potential, it is bench-marked on a set of robot problems that are challenging from a numerical perspective, including problems with a minimum-time objective and no-collision constraints. The solver is shown to solve problems for trajectory generation of a quadrotor, a robot manipulator and a truck-trailer problem in a few tens of milliseconds. The algorithm's C++-code implementation accompanies this work as open source software, released under the GNU Lesser General Public License (LGPL). This software framework may encourage and enable the robotics community to use trajectory optimization in more challenging applications.

Invariant descriptors of point and rigid-body motion trajectories have been proposed in the past as representative task models for motion recognition and generalization. Currently, no invariant descriptor exists for representing force trajectories, which appear in contact tasks. This article introduces invariant descriptors for force trajectories by exploiting the duality between motion and force. Two types of invariant descriptors are presented depending on whether the trajectories consist of screw or vector coordinates. Methods and software are provided for robustly calculating the invariant descriptors from noisy measurements using optimal control. Using experimental human demonstrations of 3-D contour following and peg-on-hole alignment tasks, invariant descriptors are shown to result in task representations that do not depend on the calibration of reference frames or sensor locations. The tuning process for the optimal control problems is shown to be fast and intuitive. Similar to motions in free space, the proposed invariant descriptors for motion and force trajectories may prove useful for the recognition and generalization of constrained motions, such as during object manipulation in contact.

In this work we present our results on dynamic visual servoing for the case of moving targets while also exploring the possibility of using such a controller for interaction with the environment. We illustrate the derivation of a feature space impedance controller for tracking a moving object as well as an Extended Kalman Filter based on the visual servoing kinematics for increasing the rate of the visual information and estimating the target velocity for both the cases of PBVS and IBVS with image point features. Simulations are carried out to validate the estimator performance during a Peg-in-Hole insertion task with a moving part. Experiments are also conducted on a real redundant manipulator with a low-cost wrist-mounted camera. Details on several implementation issues encountered during implementation are also discussed.

Learning from Demonstration (LfD) requires methodologies able to generalize tasks in new situations. This paper studies the use of virtual demonstrations to extend the extrapolation capabilities of probabilistic motion models such as the traPPCA method. Similarly to other LfD methods, traPPCA is able to calculate new trajectories very fast, but does not generalize well outside the area covered by the demonstrations. Another approach, the invariants method, shows outstanding generalization capabilities thanks to its shape-preserving prop-erties, while being limited by long computation times. The pro-posed methodology combines the advantages of the two methods by learning traPPCA models using virtual demonstrations generated by the invariants method. The proposed approach is analyzed in three case studies. Furthermore, a comparison is made between learning with virtual demonstrations and learning with only real demonstrations. The results encourage the use of virtual demonstrations to extend the extrapolation capabilities of probabilistic motion models and hence reduce the required number of real demonstrations. The latter has the potential of reducing the cost of commissioning robot tasks.

Force sensing is highly desirable in minimally invasive medical applications, since this feature shows great potential for reducing tissue damage and enhancing manipulation safety. However, embedding force sensors in medical devices is challenging and costly. This article explores the possibility to use shape sensing as a measure to extract force information. In this work, a model-based approach that allows simultaneous shape and force sensing is proposed. Shape information is reconstructed employing a multicore fiber with fiber Bragg grating sensors spaced over the fiber length. This fiber is capable of distributed 3D shape sensing. It is shown how by making use of extended Kalman filter and a mechanics model of the flexible instrument, it becomes possible to estimate both the magnitudes and locations of externally applied forces. Experiments were carried out to validate the proposed method for both one and two external forces applied at arbitrary locations in different directions on a flexible instrument. Results show that one-directional force magnitude and location can be estimated with an average error of 23.08 mN (15.39%) and 11.06 mm (6.51%), respectively. For two-directional forces, results of the load near the base show an average error of 52.01 mN (30.59%) for the magnitude and 29.24 mm (17.20%) for the location. For the load applied simultaneously near the tip, the mean magnitude error is 16.79 mN (11.19%) and the average location error is 10.18 mm (5.99%). The force sensing algorithm can run in real time with an approximate frequency of 59 Hz. In these experiments, it can be observed that the force-sensing accuracy, which depends on the sensitivity of the shape of flexible instruments with respect to the external force, can vary drastically in function of the force application point and force direction.

This paper introduces a constraint-based skill framework for programming robot applications. Existing skill frameworks allow application developers to reuse skills and compose them sequentially or in parallel. However, they typically assume that the skills are running independently and in a nominal condition. This limitation hinders their applications for more involved and realistic scenarios e.g. when the skills need to run synchronously and in the presence of disturbances. This paper addresses this problem in two steps. First, we revisit how constraint-based skills are modeled. We classify different skill types based on how their progress can be evaluated over time. Our skill model separates the constraints that impose task-consistency and the constraints that make the skills progress i.e. reaching their end conditions. Second, this paper introduces composition patterns that couple skills in parallel such that they are executed in a synchronized manner and reactive to disturbances. The effectiveness of our framework is evaluated on a dual-arm robotics setup that performs an industrial assembly task in the presence of disturbance.

In this paper, we aim to expedite the deployment of challenging manipulation tasks involving both motion and contact wrenches (forces and moments). To this end, we acquire motion and wrench signals from a small set of demonstrations using passive observation. To learn these tasks, we introduce Trajectory parameterized Probabilistic Principal Component Analysis (traPPCA) which compactly re-parameterizes the acquired signals using trajectory information and encodes the signal correlations using Probabilistic Principal Component Analysis (PPCA). Finally, the task is transferred to a robot setup by specifying the robot behavior using a constraint-based task specification and control approach. This framework results in increased robustness of the system against different sources of uncertainty: imprecise sensors, adaptation of the tool, and changes in the execution speed.

There is a high cost associated to the time and expertise required to program complex robot applications with high variability. This is one of the main barriers that inhibit the entry of robotic automation in small and medium-sized enterprises. To tackle the high level of task uncertainty associated with changing conditions of the environment, we propose a framework that leverages a combination between learning from demonstration (LfD) and constraint-based task specification and control. This synergy enables our framework to use LfD to generalize reactive approach motions (RAMo) towards not only a single pose but towards an allowable manifold defined with respect to the object to interact with. As a result, the robot executes the task by following a feasible approach motion gen-eralized from the learned information. This approach motion is generated based on an initial representation of the environment, and it can be reactively adapted in function of current updates of the environment using sensor information. The proposed framework enables the system to deal with applications that involve a high level of uncertainty, increasing the flexibility and robustness, compared to traditional sense-plan-act paradigms.

This paper presents a study of haptic perception of virtual stiffness and the influence of visual feedback in virtual reality. In this study, a novel and lightweight haptic glove (ExoTen-Glove) based on Twisted String Actuation (TSA) system is presented and evaluated. This system has two independent TSA modules with integrated force sensors and small-size DC motors. ExoTen-Glove provides force feedback to the users during the execution of grasping virtual objects. The overall design, the controller and the preliminary experimental evaluation of the ExoTen-Glove have been shown in this paper. Different experiments have been performed in virtual reality environment using HTC VIVE headset with 2 degrees of freedom grasping tasks, squeezing a pair of virtual springs with the participant's thumb and index fingers. The aim of this study is to illustrate the benefit of using ExoTen-Glove to distinguish stiffness of a pair of virtual springs and the role of the visual feedback. The results show that the users use not only haptic cues but also use visual cues in detecting spring stiffness difference.

Invariant representations of demonstrated motion trajectories provide context-independent motion models that can be used in motion recognition and generalization applications such as robot programming by demonstration. In practice, the use of invariant representations is still limited because their numerical calculation from a demonstrated trajectory is complicated by sensitivity to measurement noise and singularities, yielding inaccurate invariant functions that do not correspond well with the original trajectory. This paper improves the calculation of invariant representations for point and rigid-body motions by reformulating their calculation as an optimization problem that minimizes the error between the trajectory reconstructed from the invariant representation and the measured trajectory. Robustness against noise and singularities is ensured through the addition of regularization terms on the invariants. Simulations and real motion experiments show that the accuracy of the calculated invariant representations greatly improves with respect to standard smoothing methods. These results encourage future developments of motion recognition and generalization applications based on invariant trajectory representations.

This paper presents a novel lightweight and simple TSA-based (twisted string actuation) wearable haptic glove (ExoTen-Glove). This system is using two independent twisted string actuators with integrated force sensors and small-size DC motors. The proposed system can provide users force feedback during the execution of grasping virtual objects. The design of the TSA-based ExoTen-Glove, description of the TSA system, the controller and the preliminary experimental evaluation of the proposed system has been presented in this paper. This device has been evaluated by an experiment in virtual reality environment using HTC VIVE headset with 2 degrees of freedom grasping tasks, where the participants were squeezing a real spring with their thumb and index finger and compare it with a virtual spring stiffness. The results prove the applicability of the ExoTen-Glove for rehabilitation and haptic purposes.

In programming by demonstration, generalization is necessary to apply demonstrated motions in novel situations. Many existing trajectory representations have poor generalization capabilities since they are built on trajectory coordinates that depend on the context in which the motion is recorded. In order to generalize, the user is typically required to perform a large number of varied demonstrations. This paper instead emphasizes the usefulness of an invariant trajectory representation to separate essential motion information from context-specific information of the recorded demonstrations. The invariants are interpreted as the control inputs of a dynamical system describing the evolution of the trajectory. New trajectories are generated for novel situations as the solution of a constrained optimal control problem in which context-specific information of the novel situation is encoded in the constraints. Results indicate how, starting from only a single demonstration, new trajectories can be generated in novel situations while maintaining similarity with the original demonstration. Invariance in trajectory representations therefore proves useful to reduce the number of necessary demonstrations to learn and apply new motions.

An increasing number of robotic systems are using compliant actuators in which springs are placed in series with the actuator. The need for identification procedures tailored to these systems is consequently rising. When measurements of both the link side and the motor side of the spring are available the dynamic parameters can be identified independent of the spring model. The excitation of the identification procedure is optimized for the identification of the dynamical parameters to provide a rich data set by maximizing a reduced information matrix. Adopting this reduced matrix leads to a near-optimal excitation. A Fourier series is chosen as parametrization of the excitation, resulting in a periodic movement of the system under identification. This periodicity is exploited to develop an alternative weighting of the parameter estimation. This proposed weighting uses the motor torque variance at each time step of the trajectory, instead of one global torque variance. This identification procedure is demonstrated and validated on one leg of a lower limb exoskeleton. Only the dynamic model in the sagital plane (2D) is identified.

This paper presents an overview and comparison of minimal and complete rigid body motion trajectory descriptors, usable in applications like motion recognition and programming by demonstration. Motion trajectory descriptors are able to deal with potentially unwanted variations acting on the motion trajectory such as changes in the execution time, the motion's starting position, or the viewpoint from which the motion is observed. A suitable rigid body motion trajectory descriptor retains only the trajectory information relevant to the application. This paper compares different trajectory descriptors for rigid body motion and validates their usefulness for dealing with motion variation in a motion recognition experiment. Furthermore, a new type of invariant trajectory descriptor is introduced based on the Frenet-Serret formulas.

This paper introduces a framework for constraint-based force/position control of robots that exhibit large nonlinear structural compliance and that undergo large deformations. Controller synthesis follows hereto the principles of the Task Frame and instantaneous Task Specification using Constraints (iTaSC) formalisms. iTaSC is found particularly suitable due to its ability to express and combine control tasks in a natural way. Control tasks can be formulated as combinations of target positions, velocities, or forces expressed in an arbitrary number and type of coordinate frames. The proposed framework is applied to a mixed mechatronic system composed of a traditional rigid-link robot whose end-effector is a continuum (flexible) link. A selection of different position/force control tasks is prepared to demonstrate the validity and general nature of the proposed framework.

This work aims to extend the application field of the constraint-based control framework called iTaSC (instantaneous task specification using constraints) toward manipulation tasks. iTaSC offers two advantages with respect to other methods: the ability to specify tasks in different spaces (and not only in Cartesian coordinates as for the Task Frame Formalism), and the treatment of geometric uncertainties. These properties may be very useful within a manipulation context, where tasks are executed by robots with many degrees of freedom, which calls for some degree of abstraction; by choosing a suitable set of coordinates, it is possible to reduce the complexity and the number of constraints that fully describe such tasks; in addition, controlling only the subspace that is needed to fulfil a task allows us to use the remaining degrees of freedom of the robot system to achieve secondary objectives. This paper discusses the instruments and techniques that can be employed in manipulation scenarios; in particular it focuses on aspects like the specification of a grasp and control of the stance of the robotic arm. iTaSC offers the possibility of specifying a grasp. While this approach allows for very fine control of a grasping task, in most cases a less fine-grain specification suffices to guarantee a successful execution of the grasping action. To this end synergy-based grasp specification is formulated within iTaSC. We also show how to take into account secondary objectives for the arm stance. In particular we consider, as an example, the manipulability index along a given direction. Such indexes are maximised by exploring the null space of the other tasks. The proposed approach is demonstrated by means of simulations, where a robotic hand grasps a cylindrical object.

This work aims to extend the application field of the constraint-based control framework called iTaSC (instantaneous task specification using constraints) toward tasks where physical interaction between the robot and the environment, or a human, is contemplated. iTaSC, in its original formulation, allows for a systematic derivation of control schemes from task descriptions; tasks are defined as constraints enforced on outputs (e.g. distances, angles), and the iTaSC control takes care to fulfil such constraints by computing desired velocities to be commanded to the robot(s) joints. This approach, being based on a velocity resolution scheme, principally addresses tasks where positioning is the main issue. However, tasks that involve contacts with the environment or with the user, either desired or accidental, can be considered as well, taking advantage of impedance control, when position is controlled, or with force control. This paper describes the implementation of force tasks, and, by the combination of conflicting force and position tasks, impedance control, within the iTaSC formalism. This result is achieved by taking advantage of an approximate physical modelling of the robotic system and the environment. The proposed control scheme is tested by means of experiments where constraints on forces and/or positions described in cylindrical coordinates are imposed on a Kuka LWR arm.

This paper presents a new framework for constraint-based task specification of robot controllers. A task specification language (eTaSL) is defined as well as a corresponding implementation of a controller (eTC). This new framework is based on feature variables and a new concept referred to as expression graphs. It avoids some of the common pitfalls in previous frameworks, and provides a flexible and composable way to define robot control tasks. An architecture for a robot controller is proposed, as well as an implementation that can execute tasks described in the new specification language. Typical usage patterns for the new framework are explained on an example consisting of a kinematically redundant, bi-manual task on a PR2 robot. A comparison with existing frameworks shows the advantages of the new approach.

Path following deals with the problem of following a geometric path with no predefined timing information and constitutes an important step in solving the motion-planning problem. For differentially flat systems, it has been shown that the projection of the dynamics along the geometric path onto a linear single-input system leads to a small dimensional optimal control problem. Although the projection simplifies the problem to great extent, the resulting problem remains difficult to solve, in particular in the case of nonlinear system dynamics and time-optimal problems. This paper proposes a nonlinear change of variables, using a time transformation, to arrive at a fixed end-time optimal control problem. Numerical simulations on a robotic manipulator and a quadrotor reveal that the proposed problem formulation is solved efficiently without requiring an accurate initial guess.

Time-optimal path following considers the problem of moving along a predetermined geometric path in minimum time. In the case of a robotic manipulator a convex reformulation of this optimal control problem has been derived previously [1]. However, the bang-bang nature of the time-optimal trajectories results in near-infinite jerks in joint space and operational (Cartesian) space. For systems with un-modeled flexibilities, this usually results in excitation of the resonant frequencies, hence in unwanted vibrations and acceleration peaks, contributing to a tracking error. These vibrations can be reduced by imposing jerk constraints on the trajectory [2]. However, these jerk constraints destroy the convexity of the time-optimal control problem. The present paper proposes an efficient sequential convex programming (SCP) approach to solve the corresponding non-convex optimal control problem by writing the non-convex jerk constraints as a difference of convex (DC) functions. We illustrate the developed approach by means of experiments with a seven d.o.f. robot. Furthermore, numerical simulations illustrate the fast convergence of the proposed method in only a few SCP iterations, confirming the efficiency and practicality of the proposed framework.

In constraint-based programming, robot tasks are specified and solved as optimization problems with sets of constraints and one or multiple objective functions. In our previous work, we presented (i) a generic modeling approach for geometrically complex robot tasks, including the modeling of parametric uncertainty, in order to allow the robot task programmer to specify the optimization problem without explicitly writing down the different (possibly numerous and involved) constraint equations, and (ii) methods for solving these optimization problem online in the instantaneous case (reactive control), and offline in the non-instantaneous case (trajectory planning). This paper has two contributions. First, it extends our framework to include task constraints (e.g. tracking a curve) that are not given as explicit functions of time. These constraints are highly relevant in practice, for example to facilitate time-optimal path planning combined with other constraints. Second, it extends our framework to user-configurable task horizons when solving the optimization problem, to allow task programmers to make a trade-off between computational speed and (global) task optimality. Both of these novel framework extensions are illustrated by a time-optimal laser tracing experiment.

Navigating an electrical wheelchair can be very challenging due to its large size and limited maneuverability. Additionally, target users often suffer from cognitive or physical disabilities, which interfere with safe navigation. Therefore, a robotic wheelchair that helps to drive can prove invaluable. Such a wheelchair shares the control with its human operator. Typically, robots excel in fine-motion control whereas users want to remain in charge. Hence, the robot should focus its help locally and let the user decide about global behavior. Further, an effective robot should understand the navigation plans of its user. It needs to consider the user's abilities to avoid frustrating the user with wrong assistance. In order to address these requirements, we propose a probabilistic framework to recognize local navigation plans in a user-specific way. The framework infers navigation plans online and provides a method to calibrate all model parameters from real driving data. It fuses past local information with a user-specific model to reason about how and where the user intends to navigate. We illustrate the validity of our approach by recognizing the local navigation plans of a spastic user driving in a daily environment.

This paper presents a scene graph for geometric relations between rigid bodies that keeps track of poses and twists of rigid bodies in the scene. The scene graph relies on semantic pose and twist representation, making it invariant to the actual coordinate representation at hand. This makes the scene graph more general and interoperable than most scene graphs currently available. The presented scene graph takes into account constraints imposed by particular coordinate representations, allows for constant poses, answers semantic pose and twist queries, and provides built-in semantic consistency checks. Since the scene graph also keeps track of the twist, it allows native twist calculations, as opposed to deriving the velocities from the poses in the graph. This paper comes with software released under a dual BSD/LGPLv2.1 license.

The advent of inexpensive RGB-D cameras brings new opportunities to capture a 3D environment. This paper presents a method to create a modular setup for generating a large 3D point cloud, with attention to the study of interference, the influence of a USB extension cable, and the calibration procedure. The study of interference includes the influence of the distance between the cameras, the orientation of the cameras, and the illumination. Furthermore, this paper proposes a number of evaluation metrics for similar setups.

Current state-of-the-art robot program development needs expert programmers. Moreover, most robot programs developed today are robot hardware and software specific, and therefore little reusable without modifications. This paper realizes easier robot (re-)programming, by software framework independent models that can be executed using different hard- and software platforms. First, the paper focuses on the formalization of the tasks to be fulfilled by a robot, more specifically constraint-based programming tasks using a Domain Specific Language (DSL). Second, it gives a reference implementation in Lua [1]. The presented DSL makes it easy to develop applications, yet is powerful to execute. It enables automatic model verification and code generation for different hard- and software platforms, diminishing code debugging efforts. Experimental validation shows the ease of creating an application and adapting it, the reduction of the amount of hand-written code, and the debugging aid offered through meaningful errors returned by model verification.

Time-optimal path following considers the problem of moving along a predetermined geometric path in minimum time. In the case of a robotic manipulator with simplified constraints, a convex reformulation of this optimal control problem has been derived previously. However, many applications in robotics feature constraints such as velocity-dependent torque constraints or torque rate constraints that destroy the convexity. The present paper proposes an efficient sequential convex programming (SCP) approach to solve the corresponding nonconvex optimal control problems by writing the nonconvex constraints as a difference of convex (DC) functions, resulting in convex-concave constraints. We consider seven practical applications that fit into the proposed framework even when mutually combined, illustrating the flexibility and practicality of the proposed framework. Furthermore, numerical simulations for some typical applications illustrate the fast convergence of the proposed method in only a few SCP iterations, confirming the efficiency of the proposed framework.

This paper presents a hybrid pose / wrench control framework for quadrotor helicopters, allowing direct contact of the quadrotor with its environment and stable motion while in contact. The presented work explicitly takes into account the quadrotor's underactuation and utilizes a wrench estimator, able to estimate the wrench generated by the quadrotor using only quadrotor inputs and pose measurements. Experimental results of the quadrotor in pose and hybrid pose-wrench control mode are presented, showing stable behaviour while moving in contact. Whereas a lot of research nowadays is focused mainly on free-flight applications, stable and robust wrench control of quadrotors in direct contact with their environment, opens up new application areas for this class of flying robots.

This work aims to extend the constraint-based formalism iTaSC for scenarios where physical human-robot interaction plays a central role, which is the case for e.g. surgical robotics, rehabilitation robotics and household robotics. To really exploit the potential of robots in these scenarios, it should be possible to enforce force and geometrical constraints in an easy and flexible way. iTaSC allows to express such constraints in different frames expressed in arbitrary spaces and to obtain control setpoints in a systematic way. In previous implementations of iTaSC, industrial velocity-controlled robots were considered. This work presents an extension of the iTaSC-framework that allows to take advantage of the back-drivability of a robot thus avoiding the use of force sensors. Then, as a casestudy, the iTaSC-framework is used to formulate a (positionposition) teleoperation scheme. The theoretical findings are experimentally validated using a PR2 robot.

This article introduces a set of novel haptic guidance algorithms intended to provide intuitive and reliable assistance for electric wheelchair navigation through narrow or crowded spaces. The proposed schemes take hereto the non-holonomic nature and a detailed geometry of the wheelchair into consideration. The methods encode the environment as a set of collision-free circular paths and, making use of a model-free impedance controller, `haptically' guide the user along collision-free paths or away from obstructed paths or paths that simply do not coincide with the motion intended by the user. The haptic feedback plays a central role as it establishes a fast bilateral communication channel between user and wheelchair controller and allows a direct negotiation about wheelchair motion. If found unsatisfactory, suggested trajectories can always be overruled by the user. Relying on inputs from user modeling and intention recognition schemes, the system can reduce forces needed to move along intended directions, thereby avoiding unnecessary fatigue of the user. A commercial powered wheelchair was upgraded and feasability tests were conducted to validate the proposed methods. The potential of the proposed approaches was hereby demonstrated.

In this paper, the use of a coordinate-free representation for recognizing six DOF rigid body motion trajectories is experimentally validated. In the recognition part of this approach, the three-dimensional measured position trajectories of arbitrary and uncalibrated points attached to the rigid body are transformed to an invariant, coordinate-free representation of the rigid body motion trajectory. This representation is theoretically independent of the reference frame in which the motion is observed, the chosen marker positions, the linear scale (magnitude) of the motion, the time scale, and the motion profile. During the experiments, a person manipulated an object. The camera viewpoints, time scales, motions profiles, and linear or angular scales were changed between different motion recordings. The experimental results validate that not only in similar but also in different recording conditions, through using the invariant representation, the dependency on the parameters mentioned above are eliminated, and therefore better recognition results are obtained.

This paper presents an approach to recognize 6 DOF rigid body motion trajectories (3D translation + rotation), such as the 6 DOF motion trajectory of an object manipulated by a human. As a first step in the recognition process, 3D measured position trajectories of arbitrary and uncalibrated points attached to the rigid body are transformed to an invariant, coordinate-free representation of the rigid body motion trajectory. This invariant representation is independent of the reference frame in which the motion is observed, the chosen marker positions, the linear scale (magnitude) of the motion, the time scale and the velocity profile along the trajectory. Two classification algorithms which use the invariant representation as input are developed and tested experimentally: one approach based on a Dynamic TimeWarping algorithm, and one based on Hidden Markov Models. Both approaches yield high recognition rates (up to 95 % and 91 %, respectively). The advantage of the invariant approach is that motion trajectories observed in different contexts (with different reference frames, marker positions, time scales, linear scales, velocity profiles) can be compared and averaged, which allows us to build models from multiple demonstrations observed in different contexts, and use these models to recognize similar motion trajectories in still different contexts.

This paper discusses the theoretical background and practical implementation of a large-scale, low-performance haptic remote control setup. The experimental system consists of a pair of KUKA Light Weight Robots (LWR) coupled to a Willow Garage Personal Robot (PR2) via two different robotic frameworks. The haptic “performance” is, of course, not comparable to dedicated haptic applications, but has its use as a test-bed for interaction between “legacy” service robot systems, that have not been especially designed for mutual haptic interaction. We discuss some major application problems, and the future work needed for nonuniform robot coupling. Beside haptic coupling, we provide the human operator with visual feedback. To this end, the head movements of the human operator are coupled to the head movement of the PR2 and the images of the eye cameras are displayed to the human operator using a wearable display. The presented teleoperation application is furthermore an example of the integration of two component-based robotic frameworks namely OROCOS (Open Robot Control Software)and ROS (Robot Operating System) Experimental results regarding the haptic coupling are presented using an “artistic” painting task for qualitative results, and a hard contact at the slave side for quantitative results.

Most of today's robotic task descriptions are designed for a single software and hardware platform and thus can not be reused without modifications. This work follows the meta-model approach of Model Driven Engineering (MDE) to introduce the concepts of Domain Specific Languages (DSL) and of Model Transformations to the domain of hybrid force-velocity controlled robot tasks, as expressed in (i) the Task Frame formalism (TFF), and (ii) a Statechart model representing the discrete coordination between TFF tasks. The result is a representation in MDE's M0, M1, M2 and M3 form, with increasingly robot and software independent representations, that do remain instantaneously executable, except obviously for the M3 metametamodel. The Platform Specific Model information can be added in three steps: (i) the type of the hybrid force-velocity controlled task, (ii) the hardware properties of the robot, tool and sensor, and (iii) the software properties of the applied execution framework. We demonstrate the presented approach by means of an alignment task executed on a Willow Garage PR2 and a KUKA Light Weight Robot (LWR) arm.

This paper proposes a novel online two-level multitarget tracking and detection (MTTD) algorithm. The algorithm focuses on multitarget detection and tracking for the case of multiple measurements per target and for an unknown and varying number of targets. Information is continuously exchanged in both directions between the two levels. Using the high level target position and shape information, the low level clusters the measurements. Furthermore, the low level features automatic relevance detection (ARD), as it automatically determines the optimal number of clusters from the measurements taking into account the expected target shapes. The high level's data association allows for a varying number of targets. A joint probabilistic data association algorithm looks for associations between clusters of measurements and targets. These associations are used to update the target trackers and the target shapes with the individual measurements. No information is lost in the two-level approach since the measurement information is not summarized into features. The target trackers are based on an underlying motion model, while the high level is supplemented with a filter estimating the number of targets. The algorithm is verified using both simulations and experiments using two sensor modalities, video and laser scanner, for detection and tracking of people and ants.

We presented our constraint-based programming approach, iTaSC, that formulates instantaneous sensor-based robot tasks as constraint sets, and subsequently solves a corresponding least-squares problem to obtain control set points, such as desired joint velocities or joint torques. This paper further extends this approach, (i) by explicitly supporting the inclusion of inequality constraints in the task and (ii) by supporting a broader class of objective functions for translating the task constraints into robot motion. These extensions are made while retaining a tractable mathematical problem structure (a convex program). Furthermore, first results on extending the approach to non-instantaneous tasks are presented. As illustrated in the paper, the power of the approach lies (i) at its versatility to specify a wide range of robot behaviors and the ease of making task adjustments, and (ii) at its generic nature, that permits using systematic procedures to derive the underlying control equations.

This paper focuses on time-optimal path tracking, which involves planning of robot motions along prescribed geometric paths. Starting from a discretized convex reformulation of time-optimal path tracking problems, a log-barrier based batch solution method is presented which allows to rapidly obtain an approximate solution with smooth actuator torques. Based on this batch method, a recursive variant is derived for on-line path tracking. By means of an experimental test case in which the path data is generated on-line by human demonstration, the results and trade-offs in calculation time, delay and path duration are compared for the batch and recursive variant of the log-barrier method as well as for an exact solution method.

This paper investigates and demonstrates the feasibility of identifying contact dynamics parameters forstiffroboticpayloadsusing a robotic system. The contact dynamics model for stiff payloads is motivated, and theoretical parameter values and bounds are provided. Then, the effect of nonidealities such as surface roughness and plastic deformation on the theoretical values is demonstrated. A row-wise-scaled total least-squares parameter estimation algorithm is proposed and applied to experimental data measured using the special purpose dexterous manipulator task verification facility manipulator at the Canadian Space Agency. The experimental results are compared to a separate set of experiments with a material testing machine as well as finite-element modeling results. Finally, the experimental findings are generalized by providing guidelines for the maximum identifiable payload stiffness as a function of the position resolution, the maximum exertable force, and the structural stiffness of the robotic system.

This paper proposes and experimentally validates a Bayesian network model of a range finder adapted to dynamic environments. The modeling rigorously explains all model assumptions and parameters, improving the physical interpretation of all parameters and the intuition behind the model choices. With respect to the state of the art model [1], this paper proposes: (i) a different functional form for the probability of range measurements caused by unexpected objects, (ii) an intuitive explanation for the discontinuity encountered in the cited paper, and (iii) a reduction in the number of model parameters, while maintaining the same representational power for experimentally obtained data. The proposed beam model is called RBBM, short for rigorously Bayesian beam model. A maximum-likelihood estimation and a variational Bayesian estimation algorithm (both based on expectation-maximization) are proposed to learn the model parameters.

Robots are increasingly used to perform complex tasks, which often involve interaction and contact with unstructured environments. By identifying geometric uncertainties and the dynamic behavior of the environment on-line, the autonomy of intelligent robot systems can be considerably improved. This paper considers the 2D case of an industrial robot equipped with a probe to explore an unknown environment. The goal is to estimate from the measured end-effector position, velocity and forces not only the environmental contact dynamics parameters, but also geometric parameters such as the environment position and orientation, and the position of the probe end-point with respect to the robot end-effector. To this end, a Kalman filter based algorithm is proposed, which enforces physical constraints and which is executed in an event-triggered way to improve convergence and robustness. Experimental results illustrate the viability of the proposed algorithm.

This paper proposes and experimentally validates a Bayesian network model of a range finder adapted to dynamic environments. All modeling assumptions are rigorously explained, and all model parameters have a physical interpretation. This approach results in a transparent and intuitive model. With respect to the state of the art beam model this paper: (i) proposes a different functional form for the probability of range measurements caused by unmodeled objects, (ii) intuitively explains the discontinuity encountered in te state of the art beam model, and (iii) reduces the number of model parameters, while maintaining the same representational power for experimental data. The proposed beam model is called RBBM, short for Rigorously Bayesian Beam Model. A maximum likelihood and a variational Bayesian estimator (both based on expectation-maximization) are proposed to learn the model parameters. Furthermore, the RBBM is extended to a full scan model in two steps: first, to a full scan model for static environments and next, to a full scan model for general, dynamic environments. The full scan model accounts for the dependency between beams and adapts to the local sample density when using a particle filter. In contrast to Gaussian-based state of the art models, the proposed full scan model uses a sample-based approximation. This sample-based approximation enables handling dynamic environments and capturing multi-modality, which occurs even in simple static environments.

A first step towards truly versatile robot assistants consists of building up experience with simple tasks such as the cooperative manipulation of objects. This paper extends the state-of-the-art by developing an assistant which actively cooperates during the point-to-point transportation of an object. Besides using admittance control to react to interaction forces generated by its operator, the robot estimates the intended human motion and uses this identified motion to move along with the operator. The offered level of assistance can be scaled, which is vital to give the operator the opportunity to gradually learn how to interact with the system. Experiments revealed that, while the robot is programmed to adapt to the human motion, the operator also adapts to the offered assistance. When using the robot assistant the required forces to move the load are greatly reduced and the operators report that the assistance feels comfortable and natural.

This paper shows the application of a systematic approach for constraint-based task specification for sensorbased robot systems [1] to a laser tracing example. This approach integrates both task specification and estimation of geometric uncertainty in a unified framework. The framework consists of an application independent control and estimation scheme. An automatic derivation of controller and estimator equations is achieved, based on a geometric task model that is obtained using a systematic task modeling procedure. The paper details the systematic modeling procedure for the laser tracing task and elaborates on the task specific choice of two types of task coordinates: feature coordinates, defined with respect to object and feature frames, which facilitate the task specification, and uncertainty coordinates to model geometric uncertainty. Furthermore, the control and estimation scheme for this specific task is studied. Simulation and real world experimental results are presented for the laser tracing example.

This paper presents a contribution to programming by human demonstration, in the context of compliant-motion task specification for sensor-controlled robot systems that physically interact with the environment. One wants to learn about the geometric parameters of the task and segment the total motion executed by the human into subtasks for the robot, that can each be executed with simple compliant-motion task specifications. The motion of the human demonstration tool is sensed with a 3-D camera, and the interaction with the environment is sensed with a force sensor in the human demonstration tool. Both measurements are uncertain, and do not give direct information about the geometric parameters of the contacting surfaces, or about the contact formations (CFs) encountered during the human demonstration. The paper uses a Bayesian sequential Monte Carlo method (also known as a particle filter) to do the simultaneous estimation of the CF (discrete information) and the geometric parameters (continuous information). The simultaneous CF segmentation and the geometric parameter estimation are helped by the availability of a contact state graph of all possible CFs. The presented approach applies to all compliant-motion tasks involving polyhedral objects with a known geometry, where the uncertain geometric parameters are the poses of the objects. This work improves the state of the art by scaling the contact estimation to all possible contacts, by presenting a prediction step based on the topological information of a contact state graph, and by presenting efficient algorithms that allow the estimation to operate in real time. In real-world experiments, it is shown that the approach is able to discriminate in real time between some 250 different CFs in the graph

This paper describes the construction of a local polyhedral 3-D feature model derived from pose and wrench sensor measurements collected during a force-controlled execution of a sequence of polyhedral contact formations. The procedure consists of two steps: 1) the sensor measurements are filtered, resulting in a (nonminimal) representation of a 3-D feature model; and 2) identification of a reduced set of geometric parameters (vertices and faces) of the rigid polyhedral objects in the environment is performed. The following improvements with respect to the state of the art are made: 1) creation of a polyhedral 3-D feature model of a previously unknown polyhedral environment; 2) estimation of a force decomposition useful for feedback in a force controller or for monitoring the contact forces; and 3) reduction of the number of model parameters, leading to a computational reduction, a better precision of the geometric parameters, and a higher level description of the contact topology

This paper presents a unified task specification formalism and a unified control scheme for the lowest control level of sensor-based robot tasks. The formalism is based on: (i) the integration of any sensor that provides (direct or indirect) distance (and time derivatives) and force information; (ii) the possibility to use multiple "Tool Centre Points", e.g. defined relative to the robot end effector, other links or the environment; (iii) the integration of optimization functions for underconstrained as well as overconstrained specifications with linear constraints; (iv) the integration of on-line estimators; and (v) compatibility with all major low level control approaches. The unified formalism applies to the whole range from industrial manipulators over cooperating robots to humanoid robots, and from pure position control tasks over industrial processes to interaction between a humanoid robot and its environment.

This paper presents the compliant task generator: a new approach for the automatic conversion of a geometric path generated by a compliant path planner to a force based task specification for a compliant robot controller. Based on the geometric model of a moving object and its environment, a compliant path planner generates a set of six-dimensional positions x/sub 1...m/ and their corresponding contact formations CF/sub 1...n/. The compliant force controller, which executes a planned path under force feedback using the hybrid control paradigm, expects a desired force w/sub d/, velocity t/sub d/ and position x/sub d/ at each time-step, together with their force and velocity controlled subspaces W and T. To specify these controller primitives, we add information about the desired dynamic interaction between the moving object and its environment, in the form of the desired kinetic energy E/sub kin/ of the moving object and the potential energy E/sub pot/ in the contacts with the environment, together with the inertia and stiffness matrix M and S. We fully automated the conversion process of the compliant planner output together with the added information about the dynamic interaction, to a force based task specification. This eliminates the requirement of human intervention between the planning and execution phase. The presented approach applies to all compliant motions between polyhedral objects, and is verified in a real world experiment.

This paper presents a modular demonstration tool for robot programming by human demonstration and an approach for the calibration of the tool's sensors. The tool is equipped with a wrench sensor, twelve LED markers for fast and accurate six dimensional position tracking with the Krypton K600 camera system, a compact camera and a laser distance sensor. A gripper mechanism is mounted on the tool for grasping and manipulating objects. The design of the tool specifically focused on the demonstration of compliant motion task, with applications in manipulation and assembly tasks. The calibration approach first uses an extended Kalman Filter to convert the measured positions of three to twelve visible LED's into the pose of the tool frame relative to the Krypton camera frame. Then, using a non minimal state Kalman filter, the force sensor calibration parameters are calculated, and the orientation of the Krypton camera frame relative to the world frame is defined. This calibration approach is verified in a real world experiment.

This work presents new experimental results for the estimation of large position and orientation inaccuracies of contacting objects during force-controlled compliant motion. The estimation is based on position, velocity, and force measurements. The authors have described the contact modeling and presented some Kalman filter identification results for small inaccuracies. However, when dealing with larger inaccuracies, the nonlinear estimation problem remained unsolved. This problem has now been solved satisfactorily by applying a new Bayesian estimator. The Bayesian filter is valid for static systems (parameter estimation) with any kind of nonlinear measurement equation, subject to Gaussian measurement uncertainty and for a limited class of dynamic systems. Experimental results of this new filter are given for the estimation of the positions and orientations of contacting objects during the cube-in-corner assembly described in the first reference.

Autonomous execution of robot tasks requires the ability to deal online with uncertainties such as partially unknown environments, inaccurate models, and measurement noise. This is especially true for the execution of motions maintaining stiff contacts ("compliant motions"), as contact forces become very high even for small position errors. The autonomy during compliant motion tasks is based on i) a force controller, dealing with small misalignments and keeping the contact forces within safe limits, and ii) an estimator, which recognizes the model (e.g., the type of contact) and estimates the system state (e.g., the relative position of the contacting objects). This paper focuses on Bayesian model-based solutions to the model recognition problem. We discuss Bayesian hypothesis testing and practical approximations. Experimental results are provided for two autonomous-compliant motion tasks by applying consistency testing and likelihood ratio testing. The system state is estimated simultaneously with the model recognition. This estimation is performed by the Iterated Extended Kalman filter for (approximate) linear problems and by the nonminimal state Kalman filter for nonlinear problems.

This paper presents a new filter developed for exact nonlinear Bayesian state estimation and model recognition. The filter is able to deal (i) with static systems with any kind of nonlinear measurement function subject to additive Gaussian measurement uncertainty, and (ii) with a limited class of dynamic systems. Experimental position estimation and shape recognition results during a contouring task show the strength of the new nonlinear filtering approach.

Compliant motion is required or desirable in many robotic tasks, especially assembly tasks. Both planning and execution of autonomous compliant motion requires the knowledge of contact states between parts in contact beforehand. Previous research has addressed automatic generation of contact states between rigid objects. However, not all such contact states can be possibly reached if a rigid object is attached to and moved by a manipulator due to the manipulator constraints. In this paper, we study the problem of finding feasible contact states between a polyhedral part A held by a manipulator with a fixed base and a fixed polyhedral part B. Given a contact state graph between the unattached part A and the fixed part B, our approach then attaches A to the manipulator model and checks the reachability of each contact state and the connection between two neighboring contact states by applying a virtual compliant controller to the manipulator to test possible compliant motions of A. Implementation results validate the effectiveness of our method.

This paper describes the optimization of a compliant motion task plan which guarantees the estimation of the inaccurately known position and orientation of the contacting objects to the required accuracy ("active sensing"). The task plan consists of: (1) the desired sequence of contact formations; and (2) the compliant path to execute during each of these contact formations. The plan is optimized to minimize the expected execution time, an important criterion in industrial applications. The theory is implemented for contacts between polyhedral objects.

This paper describes the contact formation modeling for the identification of geometrical parameters (positions, orientations, and dimensions) of rigid polyhedral objects during the force-controlled execution of contact formation sequences. The following improvements with respect to the state of the art are made: 1) the modeling effort is reduced considerably; 2) the generation of the measurement equations for the online estimators can be automated more easily; 3) the propagation of the geometrical parameter estimates over sequences of contact formations becomes straightforward; and 4) the measurement equations are valid for large uncertainties on the geometrical parameter estimates.

In contrast to most hybrid vision/force research, this work uses eye-in-hand vision and force control. Mounting both sensors on the same end effector gives rise to new constraints, control issues and advantages, which are discussed in the paper. Four meaningful tool/camera configurations, being parallel or non-parallel endpoint closed-loop and fixed or variable endpoint open-loop are suggested. Several task examples (in 3D space), specified in the task frame formalism, illustrate the use of these four configurations. Experimental results for the fixed EOL configuration are presented.

Virtual decomposition control approach is applied to two coordinated industrial manipulators KUKA361 and KUKA160 subject to a rigid unilateral constraint. In order to accommodate motion/force control with the rigid unilateral constraint, the required velocity is re-designed by introducing filtered contact forces in full control space. This provides the controlled robots with a hybrid control behavior in case of a known contact geometry and with an impedance control behavior in case of an unknown contact geometry. The experiment demonstrates bouncing-free rigid contact control stability and filtered force tracking capability, in addition to asymptotic motion tracking.

Hybrid position/force control can be extended to incorporate visual servoing. Together with the task frame formalism (TFF), it forms the basis to easily model, implement and execute robotic tasks in an uncalibrated workspace, using traded, hybrid or shared vision/force control. This paper emphasizes shared control in which both vision and force simultaneously control a given task frame direction. Our work shows the usefulness and feasibility of a range of tasks using shared control. Moreover, it offers a framework based on the TFF to distinguish between different (and partly new) forms of shared control. Each basic form is illustrated by a robotic task with shared control in one direction. In addition, an extension to classify multi-dimensional shared control tasks is presented. As shown by the experimental results, the tasks at hand benefit from the shared control approach.

This paper presents a multisine approach for trajectory optimization based on information gain, with distance and orientation sensing to known beacons. It addresses the problem of active sensing, i.e. the selection of a robot motion or sequence of motions, which make the robot arrive in its desired goal configuration (position and orientation) with maximum accuracy, given the available sensor information. The optimal trajectory is parameterized as a linear combination of sine functions. An appropriate optimality criterion is selected which takes into account various requirements (e.g. maximum accuracy and minimum time). Several constraints can be formulated, e.g. with respect to collision avoidance. The optimal trajectory is then determined by numerical optimization techniques. The approach is applicable to both nonholonomic and holonomic robots. Its effectiveness is illustrated here for a nonholonomic wheeled mobile robot (WMR) in an environment with and without obstacles.

A virtual-decomposition-based control approach has been developed, which can accomplish a variety of control objectives without any restriction regarding the type of mechanical system that is being controlled. This paper presents experimental verifications involving two six-joint industrial robots (KUKA361 and KUKA160) performing hybrid tasks with rigid constraints. In order to achieve stable motion/force control in the presence of rigid unilateral constraints, the required velocities are redesigned by introducing filtered contact forces in all control dimensions. Based on a L/sub 2//L/sub /spl infin// stable tracking controller, the required velocities automatically provide the controlled robots with a hybrid control behavior in the case of a known contact geometry, and with an impedance control behavior in the case of an unknown contact geometry. The experiments yield consistent control results between the case of a single-arm constrained robot (KUKA361) and the case of a coordinated dual-robot system (KUKA361 + KUKA160). Furthermore, the experiments demonstrate the control stability in the presence of rigid contact, as well as very good force regulation. The transition phases from free motion to rigid contact are very smooth without any bouncing.

This paper gives an overview of the different levels of sensor processing complexity found in force controlled tasks, and explains which techniques from Bayesian probability theory are appropriate to cope with uncertainties and missing information at each of the different levels. The paper reduces all approaches for "intelligent" compliant, motion sensor processing to a basis set of just four classes. Some of these algorithms have already been tested experimentally, while others are still beyond the current state-of-the-art. The major contribution of this paper is to bring a clear structure to the field, which should eventually result in an easier integration of different research results, and a more precise discussion about their relative merits and innovations.

The paper above by Yang and Geng (1998) presents a class of parallel manipulators that has an efficient closed form solution to its forward kinematics. This Correspondence points to some missing insights in that paper (and in many others), especially with respect to singular configurations.

A modular approach, called virtual decomposition, is applied to adaptive control of robot manipulators with mixed rigid/flexible joints. The system is virtually decomposed into the held object, the links, and the rigid/flexible joints. Each subsystem is controlled independently with decentralized parameter adaptation, while the dynamic interactions between the subsystems are completely represented by virtual power flows (VPFs) which are independent of the dynamics of the subsystems. The equivalence between a rigid joint control and a flexible joint control is established, since they dedicate the same dynamic interactions with their connecting links. Each subsystem is assigned a nonnegative accompanying function which takes care of its dynamics. The system Lyapunov function is formed by considering the VPFs without considering the dynamics of the subsystems. Lyapunov asymptotic stability is guaranteed. Finally, simulation results are presented.

This paper describes the modelling and motion specification of compliant motion tasks with two or three contact points. These tasks cannot be done with Mason's (1981) classical "taskframe" (TF) or "compliance frame" approach. Hence, a more flexible and versatile motion constraint model is introduced, that maintains most of the intuitiveness of the TF approach.

An approach for real-time robot programming by human demonstration for 6D force controlled actions is presented. A human operator utilises a joystick to guide a robot with a force sensor to execute a task including continuous contact between a manipulated object and an unmodelled environment. During the demonstration, the position, velocity and force of the manipulated object as well as the human commands via the joystick are recorded. In real-time, the recorded information is translated into a textual robot program providing more robust execution in the presence of uncertainties. This approach has three main features (1) online control type adjustment; (2) automatic subtask termination; (3) real-time program generation. Experiments show the potential industrial applicability.

An experiment with two industrial robots KUKA 361/KUKA 160 holding an egg with two hard points contact without using any flexibility is reported in this paper. This experiment illustrates the force control result based on the framework of general constrained robots. Decentralized full-model parameter adaptation is performed by using virtual decomposition. This experiment indicates that the robots are able to perform very delicate operations with the aid of advanced control algorithms and force feedback. It also highlights the potential applications to human-centred robotics.

The choice of "where to look next" is a special case of an optimal experiment design. The paper proposes the tolerance-optimal experiment design, which is a special instance of the well-known L-optimal design, that minimises the weighted trace of the covariance matrix of the estimated state under Gaussian assumptions. The weighting matrix is chosen such that the design is invariant to transformations with nonsingular Jacobians, and such that the emerging sensing sequence reflects the information needs of the task. This tolerance-optimal design does not require more calculations than existing optimal experiment designs. Existing optimal experiment designs do not reflect the information needs of the task. In addition, some of them physically do not make sense if the estimated state has inconsistent units. The tolerance-optimal design is illustrated with two examples.

This paper finds an analytical expression for the Jacobian matrix of a parallel manipulator as well as for its derivatives (with respect to time or to a change in joint angles) of any order. The most important part in the presented procedures is the closed-form expression for the velocity closure equations of the manipulator.

A modular approach, named virtual decomposition, is applied to adaptive control of robot manipulators with mixed rigid/flexible joints. First, the held object (payload) is decomposed from the manipulator. Then the manipulator is further decomposed into a series of rigid links and rigid/flexible joints. For each subsystem (rigid body or rigid/flexible joint), independent control design and parameter adaptation are performed, and a non-negative accompanying function is assigned. The dynamic interactions between the subsystems are completely represented by virtual power flows. Both a rigid joint and a flexible joint have the same dynamic interactions with their connected links. Lyapunov asymptotic stability is guaranteed. Finally, simulation verification is conducted.

This paper extends previous work on (suboptimal) Kalman filter based identification of geometric uncertainties that occur during the manipulation of an object in contact with its environment. The setup consists of a force-controlled robot which has grasped a cylindrical peg; the relative position and orientation of the peg with respect to the robot end effector are only approximately known. The peg is moved over a surface, whose position and orientation are also uncertain. This paper describes how to estimate the above-mentioned uncertainties, even if some of the geometric uncertainties are instantaneously indeterminate, i.e., their influences on the force and/or velocity errors are instantaneously indistinguishable. Experimental results illustrate the theoretical background of the estimation problem.

The presented local world modelling tool assists a human operator of a teleoperated robot in a nuclear environment in locating and recognising stationary objects with an ultrasonic sensor. The interpretation of the measurements of this type of sensor is not trivial for a human operator. In addition, not only the uncertainty on the measurement value is important, but also the uncertainty on the origin of the measurement needs to be dealt with, i.e. from which feature of which object the measurement originates. The Bayesian multiple-hypothesis tree allows different hypotheses on the origin of a measurement to coexist if the ambiguity cannot be resolved at once. For each of these hypotheses, rigorous Bayesian methods are used to update the estimate of the object location, and to calculate its probability. The presented algorithm is easier to apply than other existing multiple hypothesis algorithms. The probability of each local scene model is calculated as the probability of this residual error of the estimate. It is shown that this residual error can be calculated in a trivial way from the results of the Kalman filter based estimator. In addition, the assumption of a static world allows to do without complex probability calculus.

This paper presents the Smoothly Constrained Kalman Filter (SCKF) for nonlinear constraints. A constraint is any relation that exists between the state variables. Constraints can be treated as perfect observations. But, linearization errors can prevent the estimate from converging to the true value. Therefore, the SCKF iteratively applies nonlinear constraints as nearly perfect observations, or, equivalently, weakened constraints. Integration of new measurements is interlaced with these iterations, which reduces linearization errors and, hence, improves convergence compared to other iterative methods. The weakening is achieved by artificially increasing the variance of the nonlinear constraint. The paper explains how to choose the weakening values, and when to start and stop the iterative application of the constraint.

This paper discusses experimental robot identification based on a statistical framework. It presents a new approach toward the design of optimal robot excitation trajectories, and formulates the maximum-likelihood estimation of dynamic robot model parameters. The differences between the new design approach and the existing approaches lie in the parameterization of the excitation trajectory and in the optimization criterion. The excitation trajectory for each joint is a finite Fourier series. This approach guarantees periodic excitation which is advantageous because it allows: 1) time-domain data averaging; 2) estimation of the characteristics of the measurement noise, which is valuable in the case of maximum-likelihood parameter estimation. In addition, the use of finite Fourier series allows calculation of the joint velocities and acceleration in an analytic way from the measured position response, and allows specification of the bandwidth of the excitation trajectories. The optimization criterion is the uncertainty on the estimated parameters or a lower bound for it, instead of the often used condition of the parameter estimation problem. Simulations show that this criterion yields parameter estimates with smaller uncertainty bounds than trajectories optimized according to the classical criterion. Experiments on an industrial robot show that the presented trajectory design and maximum-likelihood parameter estimation approaches complement each other to make a practicable robot identification technique which yields accurate robot models.

This paper presents a systematic adaptive control strategy which can accomplish a variety of control objectives (position control, internal force control, constraints,and optimizations) for the generalized high-dimensional robotic systems (GHDRS) without restriction on target systems. Based on the concept of virtual decomposition by which a GHDRS is virtually decomposed into several objects and base-floating open chains, the motion control problem of the original system is converted into that of each object and that of each open chain, individually, while the internal force control as well as the constraint force control may be performed with respect to each object only. This feature makes it possible to implement the control algorithm of each subsystem in modularly structured hardware which can be integrated to form any specific robot controller dedicated to a specific application. In the sense of Lyapunov, it is declared that the dynamic coupling between every two physically connected subsystems can be completely represented by the so-called virtual power flows (VPFs) at the cutting points between them. Asymptotic stability of the complete system can be ensured by choosing the system Lyapunov function as the sum of all nonnegative accompanying functions assigned for the subsystems. Some possible applications based on the proposed approach are discussed. Finally, computer simulations of two PUMA 560 arms transporting a common object along a prespecified trajectory are carried out to verify the stability and robustness issues of the system.

This article presents a systematic and fully general model-based approach to identify geometrical uncertainties and to detect topological transitions in contact situations during compliant motion robot tasks. The emphasis is not on control but on modelling, online uncertainty identification and model validation (monitoring). The identification and monitoring methods presented are based on energy considerations and thus are invariant with respect to changes in the mathematical representations. Only one signal has to be monitored in order to validate the correctness of the contact model. The theoretical developments are illustrated by a real world experiment: a complete assembly of a peg into a hole. Due to the computational complexity, only off-line identification and monitoring are executed.

This paper presents a recognition and location method for objects, composed of primitives with a simple analytical description, using local sensors such as an ultrasonic and an infra-red sensor. As these sensors only return local data, several measurements are needed to obtain global information. To allow effective exploration, on-line estimation is needed. Estimation is based on a constrained Kalman filter, the constraints defining relations between the primitives in the object model. Linearisation errors are reduced with a careful choice of the parametrisation of the primitives and by using a novel version of the constrained Kalman filter. Special features of the presented approach are the easy extendibility to objects composed of a different number and type of primitives, the fast convergence due to the effective use of geometric constraints, the ability to handle incomplete models, and the robustness to linearisation errors. The performance of the estimator has been thoroughly evaluated with simulations and experiments, showing that 10 measurements with an infrared sensor on the edges of a 800/spl times/550 mm/sup 2/ rectangular plate are sufficient to locate a point on the plate with standard deviation /spl sigma/=3 mm.

An approach to force controlled robot programming by human demonstration is presented. A human operator utilises a joystick to guide a robot with a force sensor to do a task including continuous contact between a manipulated object and an un-modelled environment. During the demonstration, the position, velocity and force of the object manipulated are acquired. Then the recorded data are processed, analysed, and translated into a textual robot program, which provides more robust execution in the presence of uncertainties. This approach consists of three key techniques-data processing, subtask segmentation and termination condition identification. A software package is developed to generate the programs automatically. Experiments show the industrial applicability.

This paper shows how to identify the local second order shape of an unknown surface with a force controlled robot. Dedicated tools and paths are designed in order to make the identification easier. Two basic methods are presented, based on measurements of contact forces and of positions and velocities of the end effector. The first method fits a surface through a set of contact points while the second method uses the shape of surface curves. Experiments show that following two curves of limited length is sufficient to identify second order surface shape with an accuracy which is in general better than 10%. The most important result however is that this accuracy is sufficient to increase the performance of subsequent surface following tasks.

An integrated system for programming by demonstration, visualizing, and executing compliant motion programs is described. A human operator utilises a joystick to guide a robot with a force sensor to do a task including continuous contact between manipulator and environment. The demonstration may be executed either on an actual robot, or in a graphically simulated environment. During the demonstration, the position, velocity and force of the object manipulated are acquired. Then the recorded data are processed, analysed, and translated into a textual robot program, which provides more robust execution in the presence of uncertainties. The system is composed of a model-based reaction force simulator, a visualization package, a rule-based translator, and an interpreter for compliant motion programs. Experiments show the industrial applicability.

This paper presents a general approach for the identification of contact and grasping uncertainties, and the monitoring of contact situation changes in force controlled assembly operations. The identification problem is solved using virtual contact manipulators and Kalman filter techniques. Monitoring is solved by carrying out a statistical test on the sum of normalized and squared innovations of the Kalman filter, within a moving window, identification and monitoring are verified by experimental results. The paper explains how the error covariance matrix of the Kalman filter is interpreted to analyse the observability of the Kalman filter's states. Preliminary simulation results are presented for an ad hoc active sensing strategy to achieve complete state observability.

Autonomous robot tasks involving contacts with the environment must be performed under active force control if the geometric uncertainties in the task models are too large to cope with by means of passive compliance only. In practice, task specification of force-controlled actions is closely linked to the task frame formalism (TFF), also known as the compliance frame formalism. The TFF is a very intuitive and controller-independent approach to model a motion constraint, and to specify the desired forces and motions compatible with this constraint. However, it has never been defined clearly and unambiguously, and it cannot cope with all possible constrained motion tasks. This paper provides, for the first time, a formal definition of what makes up a TFF task specification. It gives also a synthesis of which tasks the TFF can cope with, and proposes a generic textual task specification formalism. Finally, it describes an example constrained motion task that the TFF cannot handle.

Most of the literature on the classical "peg-in-hole" problem concentrates on the insertion phase. In unstructured environments, however, the preceding phases of finding the hole and aligning the axes of peg and hole are equally important. This paper describes how to model the "peg-on-hole" contact situation, and how to specify the alignment motion for arbitrarily large alignment errors between the axes of peg and hole. The results are given for any radius of peg and hole. They are applied in a real world experiment, in which some uncertainties in the location of the hole are also resolved online by active force sensing.

This paper shows how to improve the following of a planar contour with a force controlled robot using a model of the contour (i.e. how to make the task execution faster, of higher quality and more robust to large position uncertainties of the contour). The contour description is differential (it consists of curvature as a function of arc length) and Euclidean invariant as it does not change if the location of the contour is changed. A contour description is also identified from sensor measurements and matched online with the modelled contour description in order to identify the point in the model corresponding to the real contact point. The modelled curvature in this point is used for trajectory generation. Experiments have proved the usefulness of this approach.

This paper discusses the position control of a velocity controlled robot with compliant end effector using modal decoupling. Recent research has shown promising positioning and tracking results if a flexible one link robot is equipped with a velocity controlled actuator, together with an independent state feedback (ISF) of the rigid body mode and the flexible mode. The controller proposed extends the ISF controller to the case of a robot with multiple degrees of freedom (DOF) end-effector compliance. This multi-DOF system is transformed into a set of SISO-systems using modal decoupling. The ISF-approach is then applied to each of the SISO-system.

This paper deals with the combination of model-based programming and sensor-based execution of surface following tasks. It shows how differential workpiece models, in combinations with proper matching strategies, allow a robust and high quality (i.e. fast and accurate) task execution in the presence of positioning and modelling inaccuracies. A geometric description of the local shape of curves and surfaces in the neighbourhood of the current "contact-point" is given in terms of local curvature and torsion, as used in differential geometry. Matching strategies try to find a good correspondence between the real contact point on the physical workpiece and its counterpart in the model. For this purpose, local characteristics of the surface are derived from sensor and control signals. Experiments show how the introduction of heuristic rules can make the application of model information less sensitive to matching errors.>

The authors derive a theoretical framework, based on the feature frame formalism, in which the tracking equations for any general motion constraint configuration are derived in a systematic way. The resulting equations are compared to previous research, which applied intuitively derived tracking equations to simple constrained motion tasks. Their applicability to complex tasks is shown. The authors describe the elements of a kinestatic model for ideal motion constraints, and introduce one possible representation: The Jacobian, or virtual manipulator representation. The modeling of uncertainties and their influence on the Jacobian representation are discussed. Observation and identification are considered. The enhanced observation and identification potential of the feature frame formalism with respect to the task frame formalism is discussed.>

The authors describe a system that synthesizes robot programs for two-dimensional assemblies. Starting from a high-level problem definition, a number of algorithmic and heuristic modules gradually build a network of augmented compliant motion specification, determining how the robot must execute the successive assembly stages. The system has been implemented on a workstation yielding a satisfactory performance.>

The authors report on the stability characteristics of force-controlled manipulators with a discrete-time implementation. For a rigid arm, three conclusions are derived. Firstly, the system has linear damping, the amount of damping plays a key role in stability. Typically, a rise in contact stiffness can make damping insufficient, causing instability even in the case of a low feedback gain and sampling frequency well above the Nyquist frequency. Secondly, the role of Coulomb friction and stiction in closed-loop stability is clarified. For the case in which Coulomb friction is the dominant source of damping, a sufficient condition for maintaining stability is given. Thirdly, low-pass filtering can eliminate the instability caused by an insufficient sampling frequency, provided there is linear or nonlinear damping in the system. Experiments on a single-degree-of-freedom arm confirmed the conclusions.>

Most force control algorithms become unstable in case of stiff contact between the end-effector and the environment. To cope with this problem, the authors introduce active damping based on force sensor data. Linear active damping is introduced using the force derivative which is obtained either by means of filtered differentiation or by means of a state-space estimator, and both enable stable stiff contact. Nonlinear active damping through a feedforward method is introduced. The novel algorithm is simple and does not require knowledge of the contact stiffness. It enables satisfactory force response which is fast and without overshoot in the case of a high feedback gain with stiff contact. This algorithm has better potential for rejecting positional disturbances than many current algorithms.>

An approach is presented to improve the performance of existing control laws when the end effector is in contact with a moving object or when it moves along the surface of the object. The approach consists of feeding forward the object motion parameters such as velocity and acceleration, in the force control law. If the motion of the object is not known in advance, its motion parameters are estimated using an observer. The approach is applied to external force control. Simulation results demonstrate the improved performance and are confirmed experimentally.>

A generic control scheme is proposed to study the properties of a class of active compliant motion control methods that have been presented in literature. The paper concentrates on active force control methods. However, the similarity with other types of external sensor feedback is quite straightforward. It is shown that "type 1"-systems (i.e. one integration in the forward path) exhibit the most interesting properties. In addition it is shown that all active force control methods require a comparable degree of passive compliance in order to yield a comparable execution speed and disturbance rejection capability.