sponsorship: This work was supported by Flanders Make (Flanders Make is the Flemish Strategic Research Center for the manufacturing industry), through the Flanders Make SBO project AUTOCRAFT. This article was recommended by Associate Editor C. P. Hung. (Flanders Make (Flanders Make is the Flemish Strategic Research Center for the manufacturing industry) through Flanders Make SBO project AUTOCRAFT)

This paper describes a solution for pick-and-place tasks that do not require high precision throughout the execution, using a robotised gantry crane. Two common issues that occur when using a crane are addressed, and solutions are provided. First, the lack of rotational controllability is overcome by designing a gripper that passively aligns itself with the handle of the payload using a simple, robust control strategy. Second, the active workspace is expanded by using controlled, dynamic motions, based on a variable-length pendulum model. Thus, the workspace is no longer limited to positions directly accessible from above, as is the case with quasi-static control methods. The robustness and effectiveness of the proposed solution was validated by a grasping, and a shelf insertion experiment. The robot was able to grasp the payload during all 48 trials. Failures were detected and recovery strategies were implemented. The handle could also be reliably ungrasped. During the shelf insertion, imperfect trajectory tracking caused a significant error during the unobservable part of the trajectory. Nevertheless, the actual placement position was always close to the desired position.

We address the problem of (a) predicting the trajectory of an arm reaching motion, based on a few seconds of the motion’s onset, and (b) leveraging this predictor to facilitate shared-control manipulation tasks, by reducing the operator’s cognitive load through assistance in their anticipated direction of motion. Our novel intent estimator, dubbed the Robot Trajectron (RT), produces a probabilistic representation of the robot’s anticipated trajectory based on its recent position, velocity and acceleration history. By taking arm dynamics into account, RT can capture the operator’s intent better than other SOTA models that only use the arm’s position, making it particularly well-suited to assist in tasks where the operator’s intent is susceptible to change. We derive a novel shared-control solution that combines RT’s predictive capacity to a representation of the locations of potential reaching targets. Our experiments demonstrate RT’s effectiveness in both intent estimation and shared-control tasks. We will make the code and data supporting our experiments publicly available at https://gitlab.kuleuven.be/detry-lab/public/robot-trajectron

Invariant descriptors of point and rigid-body motion trajectories have been proposed in the past as representative task models for motion recognition and generalization. Currently, no invariant descriptor exists for representing force trajectories, which appear in contact tasks. This article introduces invariant descriptors for force trajectories by exploiting the duality between motion and force. Two types of invariant descriptors are presented depending on whether the trajectories consist of screw or vector coordinates. Methods and software are provided for robustly calculating the invariant descriptors from noisy measurements using optimal control. Using experimental human demonstrations of 3-D contour following and peg-on-hole alignment tasks, invariant descriptors are shown to result in task representations that do not depend on the calibration of reference frames or sensor locations. The tuning process for the optimal control problems is shown to be fast and intuitive. Similar to motions in free space, the proposed invariant descriptors for motion and force trajectories may prove useful for the recognition and generalization of constrained motions, such as during object manipulation in contact.

In this work we present our results on dynamic visual servoing for the case of moving targets while also exploring the possibility of using such a controller for interaction with the environment. We illustrate the derivation of a feature space impedance controller for tracking a moving object as well as an Extended Kalman Filter based on the visual servoing kinematics for increasing the rate of the visual information and estimating the target velocity for both the cases of PBVS and IBVS with image point features. Simulations are carried out to validate the estimator performance during a Peg-in-Hole insertion task with a moving part. Experiments are also conducted on a real redundant manipulator with a low-cost wrist-mounted camera. Details on several implementation issues encountered during implementation are also discussed.

Learning from Demonstration (LfD) requires methodologies able to generalize tasks in new situations. This paper studies the use of virtual demonstrations to extend the extrapolation capabilities of probabilistic motion models such as the traPPCA method. Similarly to other LfD methods, traPPCA is able to calculate new trajectories very fast, but does not generalize well outside the area covered by the demonstrations. Another approach, the invariants method, shows outstanding generalization capabilities thanks to its shape-preserving prop-erties, while being limited by long computation times. The pro-posed methodology combines the advantages of the two methods by learning traPPCA models using virtual demonstrations generated by the invariants method. The proposed approach is analyzed in three case studies. Furthermore, a comparison is made between learning with virtual demonstrations and learning with only real demonstrations. The results encourage the use of virtual demonstrations to extend the extrapolation capabilities of probabilistic motion models and hence reduce the required number of real demonstrations. The latter has the potential of reducing the cost of commissioning robot tasks.

This paper introduces a constraint-based skill framework for programming robot applications. Existing skill frameworks allow application developers to reuse skills and compose them sequentially or in parallel. However, they typically assume that the skills are running independently and in a nominal condition. This limitation hinders their applications for more involved and realistic scenarios e.g. when the skills need to run synchronously and in the presence of disturbances. This paper addresses this problem in two steps. First, we revisit how constraint-based skills are modeled. We classify different skill types based on how their progress can be evaluated over time. Our skill model separates the constraints that impose task-consistency and the constraints that make the skills progress i.e. reaching their end conditions. Second, this paper introduces composition patterns that couple skills in parallel such that they are executed in a synchronized manner and reactive to disturbances. The effectiveness of our framework is evaluated on a dual-arm robotics setup that performs an industrial assembly task in the presence of disturbance.

In this paper, we aim to expedite the deployment of challenging manipulation tasks involving both motion and contact wrenches (forces and moments). To this end, we acquire motion and wrench signals from a small set of demonstrations using passive observation. To learn these tasks, we introduce Trajectory parameterized Probabilistic Principal Component Analysis (traPPCA) which compactly re-parameterizes the acquired signals using trajectory information and encodes the signal correlations using Probabilistic Principal Component Analysis (PPCA). Finally, the task is transferred to a robot setup by specifying the robot behavior using a constraint-based task specification and control approach. This framework results in increased robustness of the system against different sources of uncertainty: imprecise sensors, adaptation of the tool, and changes in the execution speed.

There is a high cost associated to the time and expertise required to program complex robot applications with high variability. This is one of the main barriers that inhibit the entry of robotic automation in small and medium-sized enterprises. To tackle the high level of task uncertainty associated with changing conditions of the environment, we propose a framework that leverages a combination between learning from demonstration (LfD) and constraint-based task specification and control. This synergy enables our framework to use LfD to generalize reactive approach motions (RAMo) towards not only a single pose but towards an allowable manifold defined with respect to the object to interact with. As a result, the robot executes the task by following a feasible approach motion gen-eralized from the learned information. This approach motion is generated based on an initial representation of the environment, and it can be reactively adapted in function of current updates of the environment using sensor information. The proposed framework enables the system to deal with applications that involve a high level of uncertainty, increasing the flexibility and robustness, compared to traditional sense-plan-act paradigms.

Specifying assembly tasks in computer-aided design (CAD) level is a promising approach to intuitively program complex robot skills. In this article, a three-layered system architecture is presented to generate sensor-based robot skills from an assembly task instance. The architecture consists of an application layer where the user instantiates assembly tasks by specifying CAD constraints between geometric primitives pairs. A process layer infers the most suitable robot skills and their appropriate parameters. This inference is made possible by reasoning on a knowledge database represented as an ontology. The ontology contains semantic models of relevant classes such as tasks, skills, and geometric primitives as well as the relations between them. A control layer executes the sensor-based skills in real time using the eTaSL programming framework. A software implementation for the three layers is presented. The application layer is implemented in FreeCAD, whereas the process layer consists of a Web ontology language (OWL) ontology, a Prolog-based reasoner, and fuzzy inference to correctly select the skill and generate its parameters. In the control layer, the instantiated eTaSL skills execute the assembly tasks by sending an optimized control command to the robot. The system is validated on two challenging assembly cases with two distinct robot types, thus demonstrating the system's capability across different scenarios. Note to Practitioners-The widespread use of computer-aided design (CAD) models for describing parts assembly has motivated the research community to create systems that automatically generate robot programs satisfying the assembly goal. While most of the existing literature focuses on generating the assembly sequence, this article deals with the aspect of translation from CAD-level assembly specification to executable robot motion, also called skills. This article systematically addresses the problem by dividing it into different layers and solving them separately. Parameters that influence the successful execution of an assembly task are identified and categorized into application- and process-related parameters. Different inference techniques are employed to address each parameter category. Experimental results show that the proposed system can successfully generate and execute robot skills for assembly scenarios of an air compressor and an electric motor.

An increasing number of robotic systems are using compliant actuators in which springs are placed in series with the actuator. The need for identification procedures tailored to these systems is consequently rising. When measurements of both the link side and the motor side of the spring are available the dynamic parameters can be identified independent of the spring model. The excitation of the identification procedure is optimized for the identification of the dynamical parameters to provide a rich data set by maximizing a reduced information matrix. Adopting this reduced matrix leads to a near-optimal excitation. A Fourier series is chosen as parametrization of the excitation, resulting in a periodic movement of the system under identification. This periodicity is exploited to develop an alternative weighting of the parameter estimation. This proposed weighting uses the motor torque variance at each time step of the trajectory, instead of one global torque variance. This identification procedure is demonstrated and validated on one leg of a lower limb exoskeleton. Only the dynamic model in the sagital plane (2D) is identified.

This work aims to extend the application field of the constraint-based control framework called iTaSC (instantaneous task specification using constraints) toward manipulation tasks. iTaSC offers two advantages with respect to other methods: the ability to specify tasks in different spaces (and not only in Cartesian coordinates as for the Task Frame Formalism), and the treatment of geometric uncertainties. These properties may be very useful within a manipulation context, where tasks are executed by robots with many degrees of freedom, which calls for some degree of abstraction; by choosing a suitable set of coordinates, it is possible to reduce the complexity and the number of constraints that fully describe such tasks; in addition, controlling only the subspace that is needed to fulfil a task allows us to use the remaining degrees of freedom of the robot system to achieve secondary objectives. This paper discusses the instruments and techniques that can be employed in manipulation scenarios; in particular it focuses on aspects like the specification of a grasp and control of the stance of the robotic arm. iTaSC offers the possibility of specifying a grasp. While this approach allows for very fine control of a grasping task, in most cases a less fine-grain specification suffices to guarantee a successful execution of the grasping action. To this end synergy-based grasp specification is formulated within iTaSC. We also show how to take into account secondary objectives for the arm stance. In particular we consider, as an example, the manipulability index along a given direction. Such indexes are maximised by exploring the null space of the other tasks. The proposed approach is demonstrated by means of simulations, where a robotic hand grasps a cylindrical object.

This paper presents a new framework for constraint-based task specification of robot controllers. A task specification language (eTaSL) is defined as well as a corresponding implementation of a controller (eTC). This new framework is based on feature variables and a new concept referred to as expression graphs. It avoids some of the common pitfalls in previous frameworks, and provides a flexible and composable way to define robot control tasks. An architecture for a robot controller is proposed, as well as an implementation that can execute tasks described in the new specification language. Typical usage patterns for the new framework are explained on an example consisting of a kinematically redundant, bi-manual task on a PR2 robot. A comparison with existing frameworks shows the advantages of the new approach.

A first step towards truly versatile robot assistants consists of building up experience with simple tasks such as the cooperative manipulation of objects. This paper extends the state-of-the-art by developing an assistant which actively cooperates during the point-to-point transportation of an object. Besides using admittance control to react to interaction forces generated by its operator, the robot estimates the intended human motion and uses this identified motion to move along with the operator. The offered level of assistance can be scaled, which is vital to give the operator the opportunity to gradually learn how to interact with the system. Experiments revealed that, while the robot is programmed to adapt to the human motion, the operator also adapts to the offered assistance. When using the robot assistant the required forces to move the load are greatly reduced and the operators report that the assistance feels comfortable and natural.

This paper presents a unified task specification formalism and a unified control scheme for the lowest control level of sensor-based robot tasks. The formalism is based on: (i) the integration of any sensor that provides (direct or indirect) distance (and time derivatives) and force information; (ii) the possibility to use multiple "Tool Centre Points", e.g. defined relative to the robot end effector, other links or the environment; (iii) the integration of optimization functions for underconstrained as well as overconstrained specifications with linear constraints; (iv) the integration of on-line estimators; and (v) compatibility with all major low level control approaches. The unified formalism applies to the whole range from industrial manipulators over cooperating robots to humanoid robots, and from pure position control tasks over industrial processes to interaction between a humanoid robot and its environment.