A decentralized strategy for object transportation is presented, assuming that the object is grasped by a team of N cooperative manipulators. The proposed strategy consists of two steps. First, each robot estimates the wrenches applied to the object by all the others robots, even without all-to-all communication. Second, an admittance control scheme is used to limit internal wrenches, preventing excessive stresses that could affect manipulation stability and object integrity. Stability is proven under the assumption of a spring connection between each robot end-effector and its grasping point on the object. A work cell with two 7-degree-of-freedom (DOF) and one 6-DOF robotic manipulators was used to validate the strategy. Experimental results show that the controller effectively reduces internal wrenches, confirming the feasibility and robustness of the decentralized approach in cooperative manipulation.

This paper addresses the path-tracking problem of time-optimal trajectories under model uncertainties, by proposing a real-time predictive scaling algorithm. The algorithm is formulated as a convex optimization problem, designed to balance the trade-off between improving feasibility and time optimality of a trajectory. The predicted trajectory is scaled based on the presence of path segments that are particularly sensitive to model uncertainties within the prediction horizon. Numerical simulations and experiments demonstrate that the proposed scaling algorithm reduces the path traversal time, while preserving similar path-tracking accuracy compared to an existing non-predictive method.

This work demonstrates the utility of dual-arm robots with dual-wrist force-torque sensors in manipulating a Deformable Linear Object (DLO) within an unknown environment that imposes constraints on the DLO’s movement through contacts and fixtures. We propose a strategy to estimate the pose of unknown environmental contacts encountered during the manipulation of a DLO, classifying the induced constraints as unilateral, bilateral and fully constrained, exploiting the redundancy of force sensors. A semantic approach to define environmental constraints is introduced and incorporated into a graph-based model of the DLO. This model remains accurate as long as the DLO is under tension and is dynamically updated throughout the manipulation process, built by sequencing a set of primitives. The estimation strategy is validated through simulations and real-world experiments, demonstrating its potential in handling DLOs under various, possibly uncertain, constraints.

During planar motion, contact surfaces exhibit a coupling between tangential and rotational friction forces. This article proposes planar friction models grounded in the LuGre model and limit surface theory. First, distributed planar extended state models are proposed, and the elastoplastic model is extended for multidimensional friction. Subsequently, we derive a reduced planar friction model coupled with a precalculated limit surface, which offers the reduced computational cost. The limit surface approximation through an ellipsoid is discussed. The properties of the planar friction models are assessed in various simulations, demonstrating that the reduced planar friction model achieves comparable performance to the distributed model while exhibiting$\sim\! 80$times the lower computational cost.

In this paper, a novel decentralized leader-follower control scheme for multi-agent systems is devised, where each agent communicates only with a subset of neighboring mates. The goal is to track assigned trajectories for the centroid and the formation of the system. The desired trajectories are known only by a subset of agents, named leaders: the other agents, the followers, are required to estimate the desired trajectories based on a dynamic consensus scheme. Then, the desired trajectories to be tracked by each agent are computed from the estimated trajectories for the centroid and the formation and a simple local control loop is adopted to track the former. Stability and performance are analyzed and experiments are run on Robotarium platform to show the effectiveness of the approach and the effect of different parameters on the achieved performance.

Closed-loop motion planning is suitable for obstacle avoidance in dynamically changing environments due to its reactive nature, and various methods have been presented to provide (almost) global convergence. A common assumption in the control design is that the robot operates in a disjoint star world, i.e., all obstacles are strictly starshaped and mutually disjoint. However, in real-life scenarios obstacles may intersect due to expanded obstacle regions corresponding to robot radius or safety margins. To broaden the applicability of closed-loop motion planning methods, such as harmonic potential fields, we propose a method to reshape a workspace of intersecting obstacles into a disjoint star world. The algorithm is based on two novel concepts presented here, namely, admissible kernel and starshaped hull with specified kernel, which are closely related to the notion of starshaped hull. The utilization of the proposed method is illustrated with examples of a robot operating in a 2-D workspace using a harmonic potential field approach in combination with the developed algorithm.

Trajectory scaling has long been used to address velocity and acceleration constraints in robotic motion planning. In later years, reactive motion planning based on dynamical systems has become popular. The traditional scaling techniques are not always suitable to adopt directly when online modifications of the trajectories are made leading to feasibility problems. In this paper, we propose an approach which scales trajectories modelled as dynamical systems for improved feasibility. This is achieved by proactively scaling the trajectory as the acceleration limits are approached. Performance is illustrated by means of simulations and experiments on a UR10 robot.

In this paper, we propose a new framework for solving cable-routing problems with a dual-arm robot, where the objective is to clip a Deformable Linear Object (DLO) into several arbitrarily placed fixtures. The core of the framework is a task-space planner, which builds a roadmap from predefined tasks and employs a replanning strategy based on a genetic algorithm, if problems occur. The manipulation tasks are executed with either individual or coordinated control of the arms. Moreover, hierarchical quadratic programming is used to solve the inverse differential kinematics together with extra feasibility objectives. A vision system first identifies the desired fixture route and structure preserved registration estimates the state of the DLO in real-time. The framework is tested on real-world experiments with a YuMi robot, demonstrating a 90% success rate for 3 fixture problems.

Humans are able to manipulate Deformable Linear Objects (DLOs) such as cables and wires, with little or no visual information, relying mostly on force sensing. In this work, we propose a reduced DLO model which enables such blind manipulation by keeping the object under tension. Further, an online model estimation procedure is also proposed. A set of elementary sliding and clipping manipulation primitives are defined based on our model. The combination of these primitives allows for more complex motions such as winding of a DLO. The model estimation and manipulation primitives are tested individually but also together in a real-world cable harness production task, using a dual-arm YuMi, thus demonstrating that force-based perception can be sufficient even for such a complex scenario.

Recent advances in machine learning have triggered an enormous interest in using learning-based approaches for robot control and object manipulation. While the majority of existing algorithms are evaluated under the assumption that the involved bodies are rigid, a large number of practical applications contain deformable objects. In this work we focus on Deformable Linear Objects (DLOs) which can be used to model cables, tubes or wires. They are present in many applications such as manufacturing, agriculture and medicine. New methods in robotic manipulation research are often demonstrated in custom environments impeding reproducibility and comparisons of algorithms. We introduce ReForm, a simulation sandbox and a tool for benchmarking manipulation of DLOs. We offer six distinct environments representing important characteristics of deformable objects such as elasticity, plasticity or self-collisions and occlusions. A modular framework is used, enabling design parameters such as the end-effector degrees of freedom, reward function and type of observation. ReForm is a novel robot learning sandbox with which we intend to facilitate testing and reproducibility in manipulation research for DLOs.

Deformable object manipulation tasks have long been regarded as challenging robotic problems. However, until recently very little work has been done on the subject, with most robotic manipulation methods being developed for rigid objects. Deformable objects are more difficult to model and simulate, which has limited the use of model-free Reinforcement Learning (RL) strategies, due to their need for large amounts of data that can only be satisfied in simulation. This paper proposes a new shape control task for Deformable Linear Objects (DLOs). More notably, we present the first study on the effects of elastoplastic properties on this type of problem. Objects with elastoplasticity such as metal wires, are found in various applications and are challenging to manipulate due to their nonlinear behavior. We first highlight the challenges of solving such a manipulation task from an RL perspective, particularly in defining the reward. Then, based on concepts from differential geometry, we propose an intrinsic shape representation using discrete curvature and torsion. Finally, we show through an empirical study that in order to successfully solve the proposed task using Deep Deterministic Policy Gradient (DDPG), the reward needs to include intrinsic information about the shape of the DLO.

Deep Neural Networks (NNs) have been widely utilized in contact-rich manipulation tasks to model the complicated contact dynamics. However, NN-based models are often difficult to decipher which can lead to seemingly inexplicable behaviors and unidentifiable failure cases. In this work, we address the interpretability of NN-based models by introducing the kinodynamic images. We propose a methodology that creates images from kinematic and dynamic data of contact-rich manipulation tasks. By using images as the state representation, we enable the application of interpretability modules that were previously limited to vision-based tasks. We use this representation to train a Convolutional Neural Network (CNN) and we extract interpretations with Grad-CAM to produce visual explanations. Our method is versatile and can be applied to any classification problem in manipulation tasks to visually interpret which parts of the input drive the model’s decisions and distinguish its failure modes, regardless of the features used. Our experiments demonstrate that our method enables detailed visual inspections of sequences in a task, and high-level evaluations of a model’s behavior. Code for this work is available at [1].

In this work, the problem of human-robot collaborative object transfer to unknown target poses is addressed. The desired pattern of the end-effector pose trajectory to a known target pose is encoded using DMPs (Dynamic Movement Primitives). During transportation of the object to new unknown targets, a DMP-based reference model and an EKF (Extended Kalman Filter) for estimating the target pose and time duration of the human's intended motion is proposed. A stability analysis of the overall scheme is provided. Experiments using a Kuka LWR4+ robot equipped with an ATI sensor at its end-effector validate its efficacy with respect to the required human effort and compare it with an admittance control scheme.

Dual-arm robots can overcome grasping force and payload limitations of a single arm by jointly grasping an object. However, if the distribution of mass of the grasped object is not even, each arm will experience different wrenches that can exceed its payload limits. In this work, we consider the problem of balancing the wrenches experienced by a dual-arm robot grasping a rigid tray. The distribution of wrenches among the robot arms changes due to objects being placed on the tray. We present an approach to reduce the wrench imbalance among arms through discrete bimanual manipulation. Our approach is based on sequential sliding motions of the grasp points on the surface of the object, to attain a more balanced configuration. We validate our modeling approach and system design through a set of robot experiments.

Road friction estimation concerns inference of the coefficient between the tire and road surface to facilitate active safety features. Current state-of-the-art methods lack generalization capability to cope with different tire characteristics and models are restricted when using Bayesian inference in estimation while recent supervised learning methods lack uncertainty prediction on estimates. This paper introduces variational inference to approximate intractable posterior of friction estimates and learns an amortized variational inference model from tire measurement data to facilitate probabilistic estimation while sustaining the flexibility of tire models. As a by-product, a probabilistic tire model can be learned jointly with friction estimator model. Experiments on simulated and field test data show that the learned friction estimator provides accurate estimates with robust uncertainty measures in a wide range of tire excitation levels. Meanwhile, the learned tire model reflects well-studied tire characteristics from field test data.

We present an integrated, open-source platform for the control of assistive vehicles. The system is vehicle-agnostic and can be controlled using a myoelectric interface to translate muscle contractions into vehicular commands. A modular shared-control system was used to enhance safety and ease of use, and three collision avoidance systems were included and verified in both an included test platform and on a quadcopter operating in a simulated environment. Seven subjects performed the experiments and rated the user experience of the system under each of the provided collision avoidance systems with positive results. Qualitative tests with the quadcopter validated the proposed system and shared-control techniques. This open-source platform for shared control between humans and machines integrates decoding of motor volition with control engineering to expedite further investigation into the operation of mobile robots.

In this work, we address the dual-arm manipulation of a two degrees-of-freedom articulated object that consists of two rigid links. This can include a linkage constrained along two motion directions, or two objects in contact, where the contact imposes motion constraints. We formulate the problem as a cooperative task, which allows the employment of coordinated task space frameworks, thus enabling redundancy exploitation by adjusting how the task is shared by the robot arms. In addition, we propose a method that can estimate the joint location and the direction of the degrees-of-freedom, based on the contact forces and the motion constraints imposed by the object. Experimental results demonstrate the performance of the system in its ability to estimate the two degrees of freedom independently or simultaneously.

This paper deals with the problem of human-robot cooperative object manipulation for cases where the grasp position of the operator can change during task execution similar to human-human collaborative scenarios. In state of the art algorithms for cooperative object handling, a constant grasping position is considered for the operator. In order to accommodate the changes of the human grasping point in the control design, we do not depend on sensors on the operator's hand or on the object but we employ estimates obtained through a recursive least-squares estimator. The estimation algorithm uses only the measured wrenches obtained by a force/torque sensor located at the end-effector of the manipulator. We also propose a switching strategy for a damping controller based on the online estimates. Simulation results are provided in order to demonstrate the proposed method.

We propose a method that allows for dexterous manipulation of an object by exploiting contact with an external surface. The technique requires a compliant grasp, enabling the motion of the object in the robot hand while allowing for significant contact forces to be present on the external surface. We show that under this type of grasp it is possible to estimate and control the pose of the object with respect to the surface, leveraging the trade-off between force control and manipulative dexterity. The method is independent of the object geometry, relying only on the assumptions of type of grasp and the existence of a contact with a known surface. Furthermore, by adapting the estimated grasp compliance, the method can handle unmodelled effects. The approach is demonstrated and evaluated with experiments on object pose regulation and pivoting against a rigid surface, where a mechanical spring provides the required compliance.

In this paper, we consider folding assembly as an assembly primitive suitable for dual-arm robotic assembly, that can be integrated in a higher level assembly strategy. The system composed by two pieces in contact is modelled as an articulated object, connected by a prismatic-revolute joint. Different grasping scenarios were considered in order to model the system, and a simple controller based on feedback linearisation is proposed, using force torque measurements to compute the contact point kinematics. The folding assembly controller has been experimentally tested with two sample parts, in order to showcase folding assembly as a viable assembly primitive.

In this work we present an adaptive control approach for pivoting, which is an in-hand manipulation maneuver that consists of rotating a grasped object to a desired orientation relative to the robot's hand. We perform pivoting by means of gravity, allowing the object to rotate between the fingers of a one degree of freedom gripper and controlling the gripping force to ensure that the object follows a reference trajectory and arrives at the desired angular position. We use a visual pose estimation system to track the pose of the object and force measurements from tactile sensors to control the gripping force. The adaptive controller employs an update law that accommodates for errors in the friction coefficient, which is one of the most common sources of uncertainty in manipulation. Our experiments confirm that the proposed adaptive controller successfully pivots a grasped object in the presence of uncertainty in the object's friction parameters.

We study the problem of robot interaction with mechanisms that afford one degree of freedom motion, e.g., doors and drawers. We propose a methodology for simultaneous compliant interaction and estimation of constraints imposed by the joint. Our method requires no prior knowledge of the mechanisms' kinematics, including the type of joint, prismatic or revolute. The method consists of a velocity controller that relies on force/torque measurements and estimation of the motion direction, the distance, and the orientation of the rotational axis. It is suitable for velocity-controlled manipulators with force/torque sensor capabilities at the end-effector. Forces and torques are regulated within given constraints, while the velocity controller ensures that the end-effector of the robot moves with a task-related desired velocity. We give proof that the estimates converge to the true values under valid assumptions on the grasp, and error bounds for setups with inaccuracies in control, measurements, or modeling. The method is evaluated in different scenarios involving opening a representative set of door and drawer mechanisms found in household environments.

In this work we propose a sliding mode controller for in-hand manipulation that repositions a tool in the robot's hand by using gravity and controlling the slippage of the tool. In our approach, the robot holds the tool with a pinch grasp and we model the system as a link attached to the gripper via a passive revolute joint with friction, i.e., the grasp only affords rotational motions of the tool around a given axis of rotation. The robot controls the slippage by varying the opening between the fingers in order to allow the tool to move to the desired angular position following a reference trajectory. We show experimentally how the proposed controller achieves convergence to the desired tool orientation under variations of the tool's inertial parameters.

The rolling motion of a soft robotic fingertip is in this paper explicitly included in the control objectives together with the force/position regulation targets. A model based control law is proposed to linearize and decouple the system with respect to the force/position and sliding dynamics based on an appropriately defined task Jacobian. The controller is validated by simulations including rolling on a stationary surface and graspless manipulation of a flat object.

In the future mobile dual-arm robots are expected to perform many tasks. Kinematically, the configuration of two manipulators that branch from the same common mobile base results in a serial-to-parallel kinematic structure, which makes inverse kinematic computations non-trivial. The motion of the base has to be decided in a trade-off, taking the needs of both arms into account. We propose to use a Virtual Kinematic Chain (VKC) to specify the common motion of the parallel manipulators, instead of using the two manipulators kinematics directly. With this VKC, we formulate a constraint based programming solution for the robot to respond to external disturbances during task execution. The proposed approach is experimentally verified both in a noise-free illustrative simulation and a real human robot co-manipulation task.

One of the big challenges for robots working outside of traditional industrial settings is the ability to robustly and flexibly grasp and manipulate tools for various tasks. When a tool is interacting with another object during task execution, several problems arise: a tool can be partially or completely occluded from the robot's view, it can slip or shift in the robot's hand - thus, the robot may lose the information about the exact position of the tool in the hand. Thus, there is a need for online calibration and/or recalibration of the tool. In this paper, we present a model-free online tool-tip calibration method that uses force/torque measurements and an adaptive estimation scheme to estimate the point of contact between a tool and the environment. An adaptive force control component guarantees that interaction forces are limited even before the contact point estimate has converged. We also show how to simultaneously estimate the location and normal direction of the surface being touched by the tool-tip as the contact point is estimated. The stability of the the overall scheme and the convergence of the estimated parameters are theoretically proven and the performance is evaluated in experiments on a real robot.

In this paper we consider the problem of human-robot collaborative manipulation of an object, where the human is active in controlling the motion, and the robot is passively following the human's lead. Assuming that the human grasp of the object only allows for transfer of forces and not torques, there is a disambiguity as to whether the human desires translation or rotation. In this paper, we analyze different approaches to this problem both theoretically and in experiment. This leads to the proposal of a control methodology that uses switching between two different admittance control modes based on the magnitude of measured force to achieve disambiguation of the rotation/translation problem.

This paper addresses the problem of robot interaction with objects attached to the environment through joints such as doors or drawers. We propose a methodology that requires no prior knowledge of the objects' kinematics, including the type of joint - either prismatic or revolute. The method consists of a velocity controller which relies on force/torque measurements and estimation of the motion direction, rotational axis and the distance from the center of rotation. The method is suitable for any velocity controlled manipulator with a force/torque sensor at the end-effector. The force/torque control regulates the applied forces and torques within given constraints, while the velocity controller ensures that the end-effector moves with a task-related desired tangential velocity. The paper also provides a proof that the estimates converge to the actual values. The method is evaluated in different scenarios typically met in a household environment.

This paper introduces a method for estimating the constraints imposed by a human agent on a jointly manipulated object. These estimates can be used to infer knowledge of where the human is grasping an object, enabling the robot to plan trajectories for manipulating the object while subject to the constraints. We describe the method in detail, motivate its validity theoretically, and demonstrate its use in co-manipulation tasks with a real robot.

This paper proposes a local information based control law in order to solve the planar manipulation problem of rotating a grasped rigid object to a desired orientation using multiple mobile manipulators. We adopt a multi-agent systems theory approach and assume that: (i) the manipulators (agents) are capable of sensing the relative position to their neighbors at discrete time instances, (ii) neighboring agents may exchange information at discrete time instances, and (iii) the communication topology is connected. Control of the manipulators is carried out at a kinematic level in continuous time and utilizes inverse kinematics. The mobile platforms are assigned trajectory tracking tasks that adjust the positions of the manipulator bases in order to avoid singular arm configurations. Our main result concerns the stability of the proposed control law.

This paper proposes a PID type regulator that achieves not only the global asymptotic convergence of the robot joint velocities and position errors to zero but it also guarantees a prescribed performance for the position error transient that is independent of system constants and control parameters. The proportional term of the control input uses a transformed error (TP) which incorporates the desired performance function; given sufficiently high proportional and damping gains, the proposed TPID controller ensures the position error's prescribed performance irrespective of constant disturbances and choice of control gains. Control parameter selection is merely confined in achieving admissible input torques. Simulation results for a three dof spatial robot confirm the theoretical analysis and illustrate the robustness of the prescribed performance regulator in case of time-variant bounded disturbances.

This work deals with the problem of force/position trajectory tracking under uncertainties arising from surface position and orientation. A robotic finger with a soft hemispherical tip of uncertain compliance parameter is considered in contact with a rigid flat surface. A novel adaptive controller is designed using online estimates of the unknown parameters and is proved to achieve force and position tracking by ensuring the convergence of the estimated normal to the surface direction to its actual value. The performance of the proposed controller is demonstrated by a simulation example.

This work proposes an adaptive control law for the force position regulation problem under surface kinematic uncertainties. A compliant contact with friction is considered. The control law achieves exact regulation of force and position along the surface tangent by identifying the surface slope. The asymptotic stability of the closed loop system equilibrium point is proved in a local sense and is demonstrated by a simulation example

This paper refers to the problem of force regulation for a robot finger with soft tip in contact with a rigid surface with unknown geometrical characteristics. A simple adaptive controller is employed in order to cope with surface kinematic uncertainties and the asymptotic stability of the force error is shown for the spatial case. Simulation results demonstrate the controller performance.