This paper addresses the robotic manipulation of deformable linear objects (DLOs) in 3D space, which is a complex problem with relevant applications in, e.g., industrial, agricultural, or medical domains. We propose a simple geometric model of elastic deformation tailored to this problem, and exploit this model to derive a quasi-static deformation Jacobian mapping robot motions to changes in the DLO’s shape. We then propose a control law, based on this Jacobian, to drive multiple points on the DLO toward target positions. The described approach extends prior work restricted to 2D scenarios, and is capable of controlling the DLO’s shape in 3D and with six-degrees-of-freedom gripper motions. The main advantage of this approach is its simplicity, as it does not require training, simulation, or full perception of the object’s shape. We present validation results from both simulations and real-world experiments.

In this article, we propose a general unified tracking-servoing approach for controlling the shape of elastic deformable objects using robotic arms. Our approach works by forming a lattice around the object, binding the object to the lattice, and tracking and servoing the lattice instead of the object. This makes our approach have full control over the deformation of elastic deformable objects of any general form (linear, thin-shell, and volumetric) in 3-D space. Furthermore, it decouples the runtime complexity of the approach from the objects' geometric complexity. Our approach is based on the as-rigid-as-possible deformation model. It requires no mechanical parameter of the object to be known and can drive the object toward desired shapes through large deformations. The inputs to our approach are the point cloud of the object's surface in its rest shape and the point cloud captured by a 3-D camera in each frame. Overall, our approach is more broadly applicable than existing approaches. We validate the efficiency of our approach through numerous experiments with elastic deformable objects of various shapes and materials (paper, rubber, plastic, and foam).

In this paper, we propose a formation control system for deforming and transporting simultaneously a de-formable object with a team of robots, modeled with double-integrator dynamics. The goal is to reach a target configuration, defined as a combination of shape, scale, orientation and position of the formation. We augment this controller with a set of control barrier functions (CBFs). The CBFs allow us to satisfy fundamental constraints for the success of the task: avoidance of agent-to-agent, agent-to-obstacle and object-to-obstacle collisions, and of excessive stretching. We test the performance of our proposal in different simulation scenarios.

We propose a new approach to control the shape of deformable objects with robots. Specifically, we consider a fixed-length elastic linear object lying on a 2D workspace. Our main idea is to encode the object's deformation behavior in an offline constant Jacobian matrix. To derive this Jacobian, we use geometric deformation modeling and combine recent work from the fields of deformable object control and multirobot systems. Based on this Jacobian, we then propose a robotic control law that is capable of driving a set of shape features on the object toward prescribed values. Our contribution relative to existing approaches is that at run-time we do not need to measure the full shape of the object or to estimate/simulate a deformation model. This simplification is achieved thanks to having abstracted the deformation behavior as an offline model. We illustrate the proposed approach in simulation and in experiments with real deformable linear objects.

We address the monocular visual shape servoing problem. This pushes the challenging visual servoing problem one step further from rigid object manipulation towards deformable object manipulation. Explicitly, it implies deforming the object towards a desired shape in 3D space by robots using monocular 2D vision. We specifically concentrate on a scheme capable of controlling large isometric deformations. Two important open subproblems arise for implementing such a scheme. (P1) Since it is concerned with large deformations, perception requires tracking the deformable object's 3D shape from monocular 2D images which is a severely underconstrained problem. (P2) Since rigid robots have fewer degrees of freedom than a deformable object, the shape control becomes underactuated. We propose a template-based shape servoing scheme in which we solve these two problems. The template allows us to both infer the object's shape using an improved Shape-from-Template algorithm and steer the object's deformation by means of the robots' movements. We validate the scheme via simulations and real experiments.

Addressing the problem of enclosing and tracking a target requires multiple agents with adequate motion strategies. We consider a team of unicycle robots with a standard camera on board. The robots must maintain the desired enclosing formation while dealing with their nonholonomic motion constraints. The reference formation trajectories must also guarantee permanent visibility of the target by overcoming the limited field of view of the cameras. In this article, we present a novel approach to characterize the conditions on the robots' trajectories taking into account the motion and visual constraints. We also propose online and offline motion planning strategies to address the constraints involved in the task of enclosing and tracking the target. These strategies are based on maintaining the formation shape with variable size or, alternatively, on maintaining the size of the formation with flexible shape.

We present a novel method to control the relative positions of the members of a robotic team. The application scenario we consider is the cooperative manipulation of a deformable object in 2D space. A typical goal in this kind of scenario is to minimize the deformation of the object with respect to a desired state. Our contribution, then, is to use a global measure of deformation directly in the feedback loop. In particular, the robot motions are based on the descent along the gradient of a metric that expresses the difference between the team's current configuration and its desired shape. Crucially, the resulting multirobot controller has a simple expression and is inexpensive to compute, and the approach lends itself to analysis of both the transient and asymptotic dynamics of the system. This analysis reveals a number of properties that are interesting for a manipulation task: fundamental geometric parameters of the team (size, orientation, centroid, and distances between robots) can be suitably steered or bounded. We describe different policies within the proposed deformation-based control framework that produce useful team behaviors. We illustrate the methodology with computer simulations.

An emerging application of multirobot systems is the monitoring of a dynamic event. Here, the goal is to enclose and track a moving target by attaining a desired geometric formation around it. By considering a circular pattern configuration for the target enclosing, the multirobot system is able to perform full perception of the target along its motion. In the proposed system, the robots rely only on their onboard vision sensor without external input to complete the task. The key problem resides in overcoming the motion and visual constraints of the agents. In particular, differential-drive robots with limited sensing, that must maintain visibility of the moving target as it navigates in the environment, are considered. A novel approach to characterize the motion of the robots in the formation that allows to enclose and track the target while overcoming their limited field of view (FOV) is presented. The proposed approach is illustrated through simulations.

This paper describes a new vision-based control method to drive a set of robots moving on the ground plane to a desired formation. As the main contribution, we propose to use multiple camera-equipped unmanned aerial vehicles (UAVs) as control units. Each camera views, and is used to control, a subset of the ground team. Thus, the method is partially distributed, combining the simplicity of centralized schemes with the scalability and robustness of distributed strategies. Relying on a homography computed for each UAV-mounted camera, our approach is purely image-based and has low computational cost. In the control strategy we propose, if a robot is seen by multiple cameras, it computes its motion by combining the commands it receives. Then, if the intersections between the sets of robots viewed by the different cameras satisfy certain conditions, we formally guarantee the stabilization of the formation, considering unicycle robots. We also propose a distributed algorithm to control the camera motions that preserves these required overlaps, using communications. The effectiveness of the presented control scheme is illustrated via simulations and experiments with real robots.

This paper presents a novel method that enables a team of aerial robots to enclose a target in 3D space by attaining a desired geometric formation around it. We propose an approach in which each robot obtains its motion commands using measurements of the relative position of the other agents and of the target, without the need for a central coordinator. As contribution, our method permits any desired 3D target enclosing configuration to be defined, in contrast with the planar circular patterns commonly encountered in the literature. The proposed control strategy relies on the minimization of a cost function that captures the collective motion objective. In our method, the robots do not need to use a common reference frame. This coordinate independence is achieved through the introduction in the cost function of a rotation matrix computed locally by each robot. We prove that our motion controller is exponentially stable, and illustrate its performance through simulations.

We present a method for visual control of a set of robots moving on the ground plane. The goal of the control task is for the team to reach a desired geometric configuration. Each robot carries an omnidirectional camera and can communicate with a number of the other robots. The approach relies on the computation of the planar motion between two views, by means of 1D homographies. This knowledge, obtained by each robot from its own images and the visual information received from neighboring robots, allows it to define a desired position on the plane. Then, we propose a novel control scheme based on computing a particular 2D transformation to drive each robot towards its goal position. A contribution of this work is the use of 1D homography in a multirobot control framework. This tool allows to deal with purely angular visual information, which is precise and requires no calibration. The approach we present is completely distributed. Each robot uses only information from its formation neighbors and the global centroid to obtain its motion commands. These individual behaviors naturally result in the complete team of robots reaching the desired global configuration.

This paper addresses the estimation of planar camera motion using 1D homographies. As contributions, we show analytically that, contrary to what occurs with the 2D homography, there is a family of infinite solutions to the 1D homography decomposition, and therefore infinite possible motion reconstructions. In addition, we propose a new method to compute the planar motion between two images from the information provided by two different 1D homographies, employing their associated homology transformations. Therefore, our approach computes a general planar camera motion from only two 1D views, when previous works needed three 1D views for this task. The use of 1D information makes the method particularly suitable for omnidirectional cameras, due to the wide field of view and precise angular information provided by this kind of sensors. The performance of our proposal is illustrated through simulations and experiments on real images.

This paper addresses the problem of visual control of a set of mobile robots. In our framework, the perception system consists of an uncalibrated flying camera performing an unknown general motion. The robots are assumed to undergo planar motion considering nonholonomic constraints. The goal of the control task is to drive the multirobot system to a desired rendezvous configuration relying solely on visual information given by the flying camera. The desired multirobot configuration is defined with an image of the set of robots in that configuration without any additional information. We propose a homography-based framework relying on the homography induced by the multirobot system that gives a desired homography to be used to define the reference target, and a new image-based control law that drives the robots to the desired configuration by imposing a rigidity constraint. This paper extends our previous work, and the main contributions are that the motion constraints on the flying camera are removed, the control law is improved by reducing the number of required steps, the stability of the new control law is proved, and real experiments are provided to validate the proposal.

This paper presents a new method for visual homing to be used on a robot moving on the ground plane. A relevant issue in vision-based navigation is the field-of-view constraints of conventional cameras. We overcome this problem by means of omnidirectional vision and we propose a vision-based homing control scheme that relies on the 1D trifocal tensor. The technique employs a reference set of images of the environment previously acquired at different locations and the images taken by the robot during its motion. In order to take advantage of the qualities of omnidirectional vision, we define a purely angle-based approach, without requiring any distance information. This approach, taking the planar motion constraint into account, motivates the use of the 1D trifocal tensor. In particular, the additional geometric constraints enforced by the tensor improve the robustness of the method in the presence of mismatches. The interest of our proposal is that the designed control scheme computes the robot velocities only from angular information, being this very precise information; in addition, we present a procedure that computes the angular relations between all the views even if they are not directly related by feature matches. The feasibility of the proposed approach is supported by the stability analysis and the results from simulations and experiments with real images.