In mini-invasive surgery, the laparoscopic ultra-sound probe is visible in the laparoscopic image. We address the problem of estimating the probe pose with respect to the laparoscope without using markers and additional sensors. We propose the first method using a single standard laparoscopic monocular RGB image. It is robust, initialization-free and runs at 10 fps, thus forming a promising tool to improve robotic and augmented reality-based surgery.

In this article, we propose a general unified tracking-servoing approach for controlling the shape of elastic deformable objects using robotic arms. Our approach works by forming a lattice around the object, binding the object to the lattice, and tracking and servoing the lattice instead of the object. This makes our approach have full control over the deformation of elastic deformable objects of any general form (linear, thin-shell, and volumetric) in 3-D space. Furthermore, it decouples the runtime complexity of the approach from the objects' geometric complexity. Our approach is based on the as-rigid-as-possible deformation model. It requires no mechanical parameter of the object to be known and can drive the object toward desired shapes through large deformations. The inputs to our approach are the point cloud of the object's surface in its rest shape and the point cloud captured by a 3-D camera in each frame. Overall, our approach is more broadly applicable than existing approaches. We validate the efficiency of our approach through numerous experiments with elastic deformable objects of various shapes and materials (paper, rubber, plastic, and foam).

Most deformable object manipulation tasks still rely on skillful human operators. To automate such tasks, a robotic system should not only be able to deform an object to a desired shape but also servo its deformation along a specific path towards the desired shape. We propose a shape servoing control scheme to automate such tasks. Our scheme controls the deformation trajectory towards the desired shape by imposing task-focused convergence constraints. The constraints impose how fast the different regions of the object converge to the desired shape. Integrating such a behavior in shape servoing forms our main contribution. Experiments, carried out on rubber layer assembly tasks, show that our control scheme outperforms a state-of-the-art shape servoing scheme.

We address the monocular visual shape servoing problem. This pushes the challenging visual servoing problem one step further from rigid object manipulation towards deformable object manipulation. Explicitly, it implies deforming the object towards a desired shape in 3D space by robots using monocular 2D vision. We specifically concentrate on a scheme capable of controlling large isometric deformations. Two important open subproblems arise for implementing such a scheme. (P1) Since it is concerned with large deformations, perception requires tracking the deformable object's 3D shape from monocular 2D images which is a severely underconstrained problem. (P2) Since rigid robots have fewer degrees of freedom than a deformable object, the shape control becomes underactuated. We propose a template-based shape servoing scheme in which we solve these two problems. The template allows us to both infer the object's shape using an improved Shape-from-Template algorithm and steer the object's deformation by means of the robots' movements. We validate the scheme via simulations and real experiments.

This paper shows how to push an unknown object in the plane from an initial pose to a target pose with two cooperating mobile robots. On the object motion, we deliberately impose non-holonomic velocity constraint with pushing mobile robots. This yields smooth and efficient trajectories. Pushing manipulation is performed, for the first time, with a new uncalibrated image-based control scheme. This is achieved by transforming the image information to a scaled Euclidean space without using any metric information or calibration. Stability of the control law is also demonstrated.

As the end-effector pose is an external property of a parallel robot, it is natural to use exteroceptive sensors to measure it in order to suppress inaccuracies coming from modelling errors. Cameras offer this possibility. So, it is possible to obtain higher accuracy than in the case of classic control schemes (based on geometrical model). In some cases, it is impossible to directly observe the end-effector, but the leg directions can instead be used. In this case, however, unusual results were recorded, namely: (i) the possibility of controlling the robot by observing a number of legs less than the total number of legs, and that (ii) in some cases, the robot does not converge to the desired end-effector pose, even if the observed leg directions did. These results can be explained through the use of the hidden robot concept, which is a tangible visualisation of the mapping between the observed leg direction space (internal property) and Cartesian space (external property). This hidden robot has different assembly modes and singular configurations from the real robot, and it is a powerful tool to simplify the analysis of the aforementioned mapping. In this paper, the concept of hidden robot model is generalised for any type of parallel robot controlled through visual servoing based on observation of the leg directions. Validation has been accomplished through experiments on a Quattro robot with 4 dof.

This paper proposes a complete procedure for the structural synthesis of dexterous hands. This procedure fuses the theories already developed for the structural synthesis of dexterous hands and parallel robots. Unlike others, this procedure allows one to synthesize any kind of dexterous hand with the desired structural design parameters: mobility, connectivity, overconstraint, and redundancy. Two examples of dexterous hands, which are synthesized to have 3 dof planar motion and 6 dof spatial motion, are also given.

To control dynamics of a parallel robot, we should measure the state feedback accurately and fast. In this paper, we show how to estimate positions and velocities simultaneously (i.e., the state feedback) at a reasonable accuracy and speed. We did this using only the sequential visual contours of the legs. A single-iteration virtual visual servoing scheme regulates rapidly an error of these contours. We validated this theory, a step to control parallel robots at high speed by their leg kinematics, with simulations and experiments.

This paper discusses variable selection for the efficient dynamic control of the Quattro parallel robot through an inverse dynamic model expressed by means of leg orientations. A selection is made within a group of variables where each can imply the state of the robot. Besides, in this work, steering a parallel robot dynamically using its self-projection onto the image plane (where the edges of the lower-legs are exploited in control) is proposed and validated for the first time. In the light of the realistic control simulation, the formative points of better control of the Quattro robot are figured out.

One of the key steps in high-speed control of a parallel robot is to define an efficient dynamic model. It is usually not easy to have such a model for parallel robots, since many of them have complex structures. Here, we propose a vector-based approach, which employs the robot leg orientations, to obtain a simplified inverse dynamic model. At the least, this vector-based methodology is pioneering, when combined with the observation of orientations by a calibrated camera, in the sense of solving the entire control-oriented (hard) modeling problem, both kinematics and dynamics, in an almost algebraic manner through the knowledge of only a nominal set of image features: the edges of the robot legs and their time derivatives. Proposed method is verified on a simulator of the Quattro robot with a computed torque control where the leg orientations are steered.

Although algebraic or so-called “implicit polynomial ” curves have been studied rather extensively for several decades, to the best of our knowledge, a dynamic formulation of them, similar to active contours, has not been done yet. This paper develops a dynamic formulation for implicit polynomial curves based on level set formalism. In particular, it is shown that utilization of an implicit polynomial distance function in the level set equation yields an ordinary differential equation (ODE) for the temporal behavior of the polynomial coefficients. Using a control theoretic approach, several problems such as curve morphing, dynamic conic fitting without and with constraint, i.e. dynamic ellipse fit, and dynamic curve fitting can be tackled within this new framework. Results are verified by several examples on real images. 1

This paper presents an experimental comparison of conventional (calibrated and uncalibrated) image based visual servoing methods in various microsystem applications. Both visual servoing techniques were tested on a microassembly workstation, and their regulation and tracking performances are evaluated. Calibrated visual servoing demands the optical system calibration for the image Jacobian estimation and if a precise optical system calibration is done, it ensures a better accuracy, precision and settling time compared with the uncalibrated approach. On the other hand, in the uncalibrated approach, optical system calibration is not required and since the Jacobian is estimated dynamically, it is more flexible.