Abstract Due to the sequential, one-dimensional mechanical scanning of optical coherence tomography (OCT), a rolling shutter acquisition mode is induced that produces geometric artefacts due to the relative motion between the imaged object and the OCT probe. In this paper, we propose a technique that exploits motion artefacts in OCT volumes, to estimate both the 3D shape and motion of an arbitrary sample. Our proposed method begins by mathematically modelling motion artefacts in OCT in the 3D Cartesian space SE (3). Subsequently, we develop a methodology to estimate the motion using a volume similarity method that compares a reference OCT volume with a motion-distorted OCT volume, as a whole, rather than pointwise through 3D-to-3D point matching. The estimated motion is then utilised to correct any artefacts present in the motion-distorted volume. This methodology is then enhanced to estimate both the shape and motion from only one OCT volume acquired using the rosette non-Cartesian scan trajectory (without prior knowledge of the shape). Our results reveal that accurate shape and motion estimation are achievable using both numerically simulated and experimental OCT data, demonstrating the robustness of our method.

This paper presents a dense depth estimation approach from light-field (LF) images that is able to compensate for strong rolling shutter (RS) effects. Our method estimates RS compensated views and dense RS compensated disparity maps. We present a two-stage method based on a 2D Gaussians Splatting that allows for a “render and compare” strategy with a point cloud formulation. In the first stage, a subset of sub-aperture images is used to estimate an RS agnostic 3D shape that is related to the scene target shape “up to a motion”. In the second stage, the deformation of the 3D shape is computed by estimating an admissible camera motion. We demonstrate the effectiveness and advantages of this approach through several experiments conducted for different scenes and types of motions. Due to lack of suitable datasets for evaluation, we also present a new carefully designed synthetic dataset of RS LF images. The source code, trained models and dataset will be made publicly available at: https://github.com/ICB-Vision-AI/DenseRSLF.

In this paper, we propose an approach to address the problem of 3D reconstruction of scenes from a single image captured by a light-field camera equipped with a rolling shutter sensor. Our method leverages the 3D information cues present in the light-field and the motion information provided by the rolling shutter effect. We present a generic model for the imaging process of this sensor and a two-stage algorithm that minimizes the re-projection error while considering the position and motion of the camera in a motion-shape bundle adjustment estimation strategy. Thereby, we provide an instantaneous 3D shape-and-pose-and-velocity sensing paradigm. To the best of our knowledge, this is the first study to leverage this type of sensor for this purpose. We also present a new benchmark dataset composed of different light-fields showing rolling shutter effects, which can be used as a common base to improve the evaluation and tracking the progress in the field. We demonstrate the effectiveness and advantages of our approach through several experiments conducted for different scenes and types of motions. The source code and dataset are publicly available at: https://github.com/ICB-Vision-AI/RSLF.

This paper proposes a new control-oriented model to compute the magnetic field created by a coil. A major challenge for untethered microscale mobile robotics is the control of objects for precise and fast displacements. In this work, we propose to use an alternative implementation of a model based on elliptic integral functions to control magnetically actuated micro-robots. It allows to compute the magnetic field even in the area close to the coil quickly and accurately. This model is evaluated numerically and compared to classical approaches - dipole approximation, map-based interpolation and classical elliptic integral models - in terms of accuracy, computation time and memory requirement. Simulation results show that this works allows to have an accurate model in the whole workspace by avoiding numerical issues encountered in previous works. It can be computed in a few milliseconds, making it the right candidate for closed-loop control of magnetically actuated micro-robots.

Flexible and helical magnetic microswimmers have been well reviewed in the literature because they could be exploited for envisaged applications such as targeted drug delivery, material removal, and micromanipulation. In this article, scaled-up versions of those robots are introduced to study in detail their maneuverability and dexterity while swimming. The robots were immersed in pure glycerol, thus, reproducing a low Reynolds scenario. The proposed robots were previously optimized, achieving their best performances. The experiments assess the performances of these two kinds of robots in terms of rapidity, and steering error following 3D trajectories in environments with high viscous variations.

This paper deals with an OCT (optical coherence tomography) geometric calibration method. OCT medical imaging system has received a growing interest during the last two decades. In medical purposes, OCT images are generally called optical biopsies which allows in-vivo investigation almost similar to a histopathological study. The physician can rely on the OCT images to establish a rapid and direct diagnosis. But the OCT images formation suffered numerous distortions due in particular to the optical path, from the source to the viewed sample passing through the two reflecting mirrors and a scan objective. The obtained optical biopsies include several spectral and geometric distortions. The proposed calibration model aims to compensate the geometrical ones. More precisely, two models were developed allowing the correction of both 2D images (B-Scan slices) and 3D images (volume). These models were experimentally validated (in both artificial and biological samples) using a spectral domain OCT system. It has demonstrated a significant enhancement of the OCT images accuracy.

This paper presents the design, modeling and fabrication of a millimeter-size Continuum Soft Robot (CSR). The robot consists of active flexible polymer actuator-based multisegment robot. A multiphysics model based on multilayer cantilever for large displacement is established between the input voltages to the distal tip position of a single segment robot. The extension of the model to multisegment CSR is derived. The proposed model is validated experimentally then a two-segment CSR and three-segment CSR in 3D arrangement are investigated, demonstrating the model efficiency for obtaining complex configuration. Moreover, various configurations can be explored to derive complex kinematics then increasing the robot capability.

The Remote Centre of Motion (RCM) is an essential movement for most of the medical robotic systems during the minimal invasive surgeries. In literature, there are many references which had analysed the techniques for modelling the problem of RCM constraint. However, only very few have discussed the RCM with Path Following Controller which ensures the complexity of surgical tasks. Therefore, this article is focusing on presenting the Task Priority Controller. Such task controller deploys exteroceptive sensor (visual servoing scheme) in order that the surgical tool follows the reference path while maintaining the RCM constraints. The experimental results showed good performance of the proposed controller.

This paper presents preliminary results on OCT (Optical Coherence Tomography) based control of a concentric tube robot (CTR). The potential use of OCT system is presented then its ability to perform an optical biopsy and an eye-to-hand configuration for robot control is highlighted. A prototype of CTR with three curved tubes is shortly described then a model-less OCT-based controller is detailed. The extraction of visual features from OCT images is performed by normalized cross-correlation. The proposed control scheme was experimentally validated. The positioning accuracy of CTR control was evaluated in benchtop trials. The error in tip placement can reach 10 μm along X and 74 μm along Y according to controller parameters. The 2D positioning (XY plan) is also tested then a final error about 55 μm is achieved.

Previously, we developed a 3D path following algorithm to improve the microrobot performances to overcome the modeling errors and environmental disturbances in order to perform real tasks, tested experimentally in a scaled-up helical microswimmer. In this paper, we show that adapting the propulsion mode, the general path following algorithm can be used for any microswimmer behaving as a nonholonomic system. For that purpose, we study a magnetic robot with a flexible tail that mimics the spermatozoa locomotion mechanism. A frequency and amplitude characterization of the flexible swimmer using an oscillating magnetic field is shown. To adapt the propulsion mode, we develop a 3D magnetic field control based on the steering angular velocities which are computed from the path following algorithm in order to propel and steer the flexible robot to reach a desired location in space and achieve for the first time a 3D closed-loop motion control using a swimmer with flexible flagella.

International audience

This paper deals with the development of a new 6 degrees-of-freedom (DOF) vision-based controller for robot-assisted medical applications. The main objective is to use visual information extracted from Ultrasounds (US) images to control an US probe held by a robot. Instead of the conventional use of visual features e.g., geometric features (points, lines, moments, etc.) in the visual control law design, the described method uses the shearlet coefficients. More precisely, the time-variation of the coarsest level of shearlet decomposition coefficients are linked to the US probe (respectively to the robot) spatial velocity and then related to the task-function control law. The proposed control law was experimentally tested and validated using a realistic abdominal phantom. The obtained results demonstrated promising performances in terms of accuracy, repeatability, robustness and convergence behavior.

This paper addresses the problem of 3D path following of magnetic helical microswimmers in closed-loop. An error kinematic model in a local frame with sideslip and attack angles is used to express the motion of the helical microswimmer. A new derivation of the chained form with three inputs and five states is used to linearize the kinematic model in order to design a decoupled stable control. In experimentation, the 3D path following is validated using a scaled-up magnetic helical microswimmer with visual servo control by following first a spatial straight line, then a helix trajectory and finally an inclined sinusoidal trajectory. The closed-loop control is also compared with the open-loop control to illustrate the robustness and the accuracy of both controllers to the disturbances.

This paper proposes the design of a vision-guided control law for microrobotic-assisted biomedical applications. More precisely, the developed control law is to servo an optical coherence tomography (OCT) system in real-time to carry out repetitive optical biopsy tasks. The OCT images are simultaneously used to perform the optical biopsy and to control the sample holder (microrobotic work-flow). Instead of extracting visual features from the OCT images, the vision-based controller uses the concept of frequency domain information to compute relative motion between two successive OCT B-scan (cross-section) images. Therefore, the visual controller is designed to minimize the error between current and desired images by controlling the microrobotic platform. The proposed approach was experimentally validated, demonstrating more than satisfactory results especially in terms of accuracy, convergence, and robustness.

This paper presents the development of a new visual servoing control law using global information image. It consists in the use of the wavelet spatial coefficients as the visual signal to design visual controller. More precisely, we introduce the concept of multiple resolution interaction matrix that links the time-variation of wavelet coefficients to the robot spatial velocity and the associated task function controller. The proposed controller was successfully validated using a 6 degrees-of-freedom 3PPSR parallel kinematic manipulator in an eye-to-hand configuration. Experimental results show the accuracy, efficiency and show good behavior of such features.

A novel structure of a 2 DoF telescopic soft robot using a tri-layer Polypyrrole (PPy) soft micro-actuator with deployment is presented in this paper. The kinematic model is introduced and the Position Based Visual Servo (PBVS) control with path-planning and obstacle avoidance algorithms is developed. A prototype is presented and the control schemes are validated experimentally. A satisfactory accuracy with a submillimetric positioning error is obtained namely 287.6 microns for a circular path and 210 microns for obstacle avoidance.

This paper deals with the development of a vision-based controller for robot-assisted medical applications. It concerns the use of shearlet coefficients in case of ultrasounds (US) images as visual signal inputs and the design of the associated interaction matrix. The proposed controller was validated in both simulation and on an experimental test bench which consists of a robotic arm holding an US probe in contact with a realistic abdominal phantom. Also, the proposed control scheme was compared to the photometry-based visual servoing approach in order to evaluate its efficiency in different conditions of use (nominal and unfavorable conditions).

This paper addresses the design of a partitioned vision-guided scheme for repetitive optical biopsies. More precisely, our approach uses two image modalities to perform 6 degrees of freedom (DOF) positioning task. The development aims to partition the control into 3 DOF controlled by the B-scan images acquired with an optical coherence tomography (OCT) system and the remaining 3 DOF controlled by the white light images provided by a CCD camera. Moreover, for the control and instead of conventional visual features (e.g., points, lines, moments, etc.) extracted using algorithms combined with visual tracking approaches, our visual servoing method uses the multiresolution wavelet coefficients. The developed method was experimentally validated using a parallel kinematic structure equipped with a Telesto-II OCT benchtop. The validation task consisted of an automatic spatial repositioning of the robotic structure to precisely retrieve the position of an initial optical biopsy. Several tests are achieved, which clearly demonstrate the reliability of the proposed controller.

This paper deals with mutual information-based numerical and physical registration of white light images vs. fluorescence images for microrobotic laser microphonosurgery of the vocal folds. More precisely, it presents two techniques: a numerical registration of multimodal images and a vision feedback control for positioning an endoscope with regards to a preoperative image (fluorescence image). Nelder-Mead Simplex for nonlinear optimization is used to minimize the cost-function. The proposed methods are successfully validated in an experimental set-up using preoperative fluorescence images and real-time white light images of the vocal folds.

In this paper, a method to predict the precision of parallel microstructures made of carbon fibers, polyimide layers and piezoelectric materials is proposed. To this aim, we propose to combine the exponential representation of transformation matrix with intervals to bound the uncertainties in each flexure joint of the microstructures. Based on interval techniques and algorithms, the derived model permits to guarantee a bound on the precision value of the end effector. The method has been demonstrated with a numerical example of parallel microrobot devoted to orient the laser spot towards vocal fold during phonomicrosurgery. This example considers the uncertainties that occur in the compliant joints caused by the fabrication technique.

Laser surgery requires accurate following of a path defined by the surgeon, while the velocity on this path is dependent on the laser-tissue interaction. Therefore, path following and velocity profile control must be decoupled. In this paper, nonholonomic control of the unicycle model is used to implement velocity-independent visual path following for laser surgery. The proposed controller was tested in simulation, as well as experimentally in several conditions of use: different initial velocities (step input, successive step inputs, sinusoidal inputs), optimized/nonoptimized gains, time-varying path (simulating a patient breathing), and complex curves with curvatures. Thereby, experiments at 587 Hz (frames/s) show an average accuracy lower than 0.22 pixels (~10 μm) with a standard deviation of 0.55 pixels (~25 μm) path following, and a relative velocity distortion of less than 10-6%.

Helical microswimmers that are capable of propulsion at low Reynolds numbers have great potential for numerous applications. Several kinds of artificial magnetic-actuated helical microswimmers have been designed by researchers. However, they are primarily open-loop controlled. This paper aims to investigate methods of closed-loop control of a magnetic-actuated helical swimmer at low Reynolds number by using visual feedback. For many in-vitro applications, helical swimmers should pass through a defined path, for example along channels with no prerequisite on the velocity profile along the path. Therefore, the main objective of this paper is to achieve a velocity-independent planar path following task. Since the planar path following is based on 3-D steering control of the helical swimmer, a 3-D pose estimation of a helical swimmer is introduced based on the real-time visual tracking with a stereo vision system. The contribution of this paper is in two parts: The 3-D steering of a helical swimmer is demonstrated by visual servo control; and the path following of a straight line with visual servo control is achieved, then compared with open-loop control. We further expect that with this visual servo control method, the helical swimmers will be able to follow reference paths at the microscale.

Helical microswimmers capable of propulsion at low Reynolds numbers have been proposed for many applications. However, closed-loop controlled helical swimmers are still challenging because of the limits of optical tracking, and a lack of control parameters lying on the swimming characteristics of both linear propulsion and steering. Although the linear propulsion characteristics of helical swimmers were extensively studied, the steering characteristics have not yet been clearly shown. Helical microswimmers are efficient in propulsion, whereas their high surface-to-volume ratio limits the steering performances. In this paper, we characterized both the direction and inclination steering using a real-time visual tracking of orientation. The direction steering efficiency could be increased in terms of response time with a higher inclination angle both in floating conditions and on a sticky substrate. We thus developed a 3D steering strategy by combining direction and inclination steering to improve the steering performance. We further expect that the characterization of steering performance can contribute to defining the control parameters for future closed-loop control.

This paper deals with the study of a weakly calibrated multiview visual servoing control law for microrobotic laser phonomicrosurgery of the vocal folds. It consists of the development of an endoluminal surgery system for laser ablation and resection of cancerous tissues. More specifically, this paper focuses on the part concerning the control of the laser spot displacement during surgical interventions. To perform this, a visual control law based on trifocal geometry is designed using two cameras and a laser source (virtual camera). The method is validated on a realistic testbench and the straight point-to-point trajectories are demonstrated.

To control dynamics of a parallel robot, we should measure the state feedback accurately and fast. In this paper, we show how to estimate positions and velocities simultaneously (i.e., the state feedback) at a reasonable accuracy and speed. We did this using only the sequential visual contours of the legs. A single-iteration virtual visual servoing scheme regulates rapidly an error of these contours. We validated this theory, a step to control parallel robots at high speed by their leg kinematics, with simulations and experiments.

It has been demonstrated that magnetic fields are relevant for manipulating an untethered magnet, either using fixed coils or mobile permanent magnets. This paper shows however, that any magnetic manipulation method is prone to singular configurations and that the simple numerical analysis of the rank of the “manipulation matrix” is not enough to detect them. Alternatively, we propose a geometrical analysis to interpret and detect the singularities as well as to decide on the acceptability of a reference trajectory. Then, we present results obtained by simulating a planar manipulation system including a multiple mobile coils and a Helmholtz like set-up.

Micro and nanorobots can change many aspects of medicine by enabling targeted diagnosis and therapy, and minimal invasive surgery. A helical nanobelt with a magnetic head was proposed as a microrobot driven by rotating magnetic field in prior works. Magnetically coated tails were already shown in some works. However the control of such surface magnetic tails is not clearly realized yet. This paper aims to obtain control parameters for the modeling and simulation of the influence of surface magnets onto the swimming performances. For this, we created scaled-up helical nanobelts and the experimental testbed to get the control parameters and to prepare future closed-loop control.

This paper is dedicated to vision-based modeling and control of large-dimension parallel robots driven by inextensible cables of non-negligible mass. An instantaneous inverse kinematic model devoted to vision is introduced. This model relies on the specificities of a parabolic profile hefty cable modeling and on the resulting simplified static analysis. By means of a kinematic visual servoing method, computer vision is used in the feedback loop for easier control. According to the modeling derived in this paper, measurements that allow the implementation of this visual servoing method consist of the mobile platform pose, the directions of the tangents to the cable curves at their drawing points and the cable tensions. The proposed visual servoing scheme will be applied to the control of a large parallel robot driven by eight cables. To this end, in order to obtain the aforementioned desired measurements, we plan to use a multi-camera setup together with force sensors.

This paper discusses variable selection for the efficient dynamic control of the Quattro parallel robot through an inverse dynamic model expressed by means of leg orientations. A selection is made within a group of variables where each can imply the state of the robot. Besides, in this work, steering a parallel robot dynamically using its self-projection onto the image plane (where the edges of the lower-legs are exploited in control) is proposed and validated for the first time. In the light of the realistic control simulation, the formative points of better control of the Quattro robot are figured out.

This paper deals with the vision-based control of cable-driven parallel robots. First, a 3D pose visual servoing is proposed, where the end-effector pose is indirectly measured and used for regulation. This method is illustrated and validated on a cable-driven parallel robot prototype. Second, to take into account the dynamics of the platform and using a Cartesian pose and velocity estimator, a vision-based computed torque control is developed and validated in simulation.

Vision has often been considered as not suitable for dynamic control of robots. The experimental results presented in this paper show that it is possible to perform better with a vision based dynamic control than with a model-based control. These results were obtained using a Cartesian computed torque control fed back, without any joint sensing, by a novel Cartesian pose and velocity estimator. The latter is designed as a virtual visual servoing scheme based on sequential acquisition of sub-images and a constant acceleration motion assumption.

One of the key steps in high-speed control of a parallel robot is to define an efficient dynamic model. It is usually not easy to have such a model for parallel robots, since many of them have complex structures. Here, we propose a vector-based approach, which employs the robot leg orientations, to obtain a simplified inverse dynamic model. At the least, this vector-based methodology is pioneering, when combined with the observation of orientations by a calibrated camera, in the sense of solving the entire control-oriented (hard) modeling problem, both kinematics and dynamics, in an almost algebraic manner through the knowledge of only a nominal set of image features: the edges of the robot legs and their time derivatives. Proposed method is verified on a simulator of the Quattro robot with a computed torque control where the leg orientations are steered.

This paper presents a high speed visual tracking method based on non simultaneous subimages acquisition. This method is formulated as a virtual visual servoing scheme. The sequential acquisition of regions of interest has a double benefit on visual servoing. The first one is that this acquisition method allows to increase the visual control sampling frequency by reducing the data amount to acquire and to transmit by the camera. The second one is that the associated image projection model depends on the observed object pose and velocity. Thanks to this property, a new vision-based control law can be defined. The particularity of this control law is that the control output consists of the kinematic and the dynamic twists. This allows to enhance the control performance in trajectory tracking applications. The experimental results in high speed visual tracking application show the effectiveness of this approach.

This paper deals with the visual control of the Gough-Stewart platform using a central catadioptric camera observing the platform's legs. This allows a large field of view to be obtained and avoids the occlusion problems observed when a classical perspective camera is used. An automatic and simple method to detect the projections of the leg in the image is also proposed. The control scheme presented here is shown to encompass the classical perspective camera case, as well as catadioptric ones. Finally, experimental results comparing two kinds of visual features (leg directions and leg edges) are described.

This paper presents a novel method for high speed pose and velocity computation from visual sensor. The main problem in high speed vision is the bottleneck phenomenon which limits the video rate transmission. The proposed approach circles the problem out by increasing the information density instead of the data rate transmission. This strategy is based on a rotary sequential acquisition of selected regions of interest (ROI) which provides space-time data. This acquisition mode induces an image projection deformation of dynamic objects. This paper shows how to use this artifact for the simultaneous measure of both pose and velocity, at the same frequency as the ROI's acquisition one.

In this paper, a novel approach for parallel kinematic machine control relying on a fast exteroceptive measure is implemented and validated on the Orthoglide robot. This approach begins with rewriting the robot models as a function of the only end-effector pose. It is shown that such an operation reduces the model complexity. Then, this approach uses a classical Cartesian space computed torque control with a fast exteroceptive measure, reducing the control schemes complexity. Simulation results are given to show the expected performance improvements and experiments prove the practical feasibility of the approach.

This paper addresses the kinematic calibration of the Gantry-Tau parallel kinematic manipulator. It offers two closed-form solutions to this problem, which do not need any initial guess, contrarily to most calibration methods that are based on non-linear least square minimization. This makes the proposed methods applicable by non expert users in a context of mechanical modularity of the robot. Experimental results validate the approach.

Recent work shows that recovering pose and velocity from a single view of a moving rigid object is possible with a rolling shutter camera, based on feature point correspondences. We extend this method to line correspondences. Owing to the combined effect of rolling shutter and object motion, straight lines are distorted to curves as they get imaged with a rolling shutter camera. Lines thus capture more information than points, which is not the case with standard projection models for which both points and lines give two constraints. We extend the standard line reprojection error, and propose a nonlinear method for retrieving a solution to the pose and velocity computation problem. A careful inspection of the design matrix in the normal equations reveals that it is highly sparse and patterned. We propose a blockwise solution procedure based on bundle-adjustment-like sparse inversion. This makes nonlinear optimization fast and numerically stable. The method is validated using real data.

This paper proposes a method to control an I4R parallel robot by the observation of its legs with a calibrated camera. We show that the control law depends only on the edges of its forearms extracted from the image and that no proprioceptive sensors are used. Indeed, the variables needed for control (namely, the directions of the arms and forearms as well as the positions of the wrists) can be reconstructed from the forearms edges. Experimental validation of the reconstruction is given and simulation of the control with realistic noises is performed showing the validity of the approach.

This work deals with the control by vision of the Gough-Stewart platform. For that, a central catadioptric camera is used to observe the platform legs. This allows to obtain a large field of view, and then avoids the occlusion problems observed when a classical perspective camera is used. The leg projections onto the catadioptric plane are used to determine their orientation in the camera frame. Finally, the computed orientations will be used in a visual servoing scheme of the platform effector.

This paper proposes an original and novel vision sensing method to be used in vision-based dynamic identification of parallel robots. Indeed, it is shown that in the latter problem one requires to estimate (to the least) or measure (to the best) the end-effector pose and its time derivatives. The sensor we propose, based on a clever modelling of CMOS rolling shutter camera, measures simultaneously the end-effector pose of a calibrated visual pattern and its Cartesian velocity using a single view. Although motivated by parallel robot identification, this low cost sensor does not make any assumption on the kinematics of the robot and can thus be used for other applications. Experimental results with real data confirm the relevance of the approach and show the sensor good practical measurement accuracy

Motivated by the growing interest for omnidirectional sensors on robotic applications and particularly on vision-based control, we present a new framework to handle in a visual servoing scheme the projection of line features into the image plane of a central catadioptric camera. As it is well known, the projection of a 3D line in the image plane of a central catadioptric camera is a conic curve. We propose to use the polar line of the image center with respect to this conic curve to define the input of the vision-based control scheme. The visual observations obtained from the polar lines lead to a minimal representation of projected lines. An efficient control scheme based only on two image features can then be designed. Simulation and experimental results confirm the validity of our approach

An original method for computing instantaneous 3D pose and velocity of fast moving objects using a single view is presented. It exploits image deformations induced by rolling shutter in CMOS image sensors. First of all, a general perspective projection model of a moving 3D point is presented. A solution for the pose and velocity recovery problem is then described. The method is based on bundle adjustment and uses point correspondences. The resulting algorithm enables to transform a CMOS low cost and low power camera into an original velocity sensor. Finally, experimental results with real data confirm the relevance of the approach.

In this paper, we show that visual feedback reduces the complexity of parallel robot Cartesian control. Namely, 3D pose visual servoing, where the end-effector pose is indirectly measured and used for regulation, is shown to be well suited to this task since it relieves the control from the difficult forward kinematic problem. Moreover, this complexity reduction is not coming with an increase of the implementation complexity since off-the-shelf hardware and software are now available for visual servoing. It is also shown that such a control gets rid of joint sensors. All this makes 3D pose visual servoing the most straightforward Cartesian control for parallel robots. Experimental results are provided using an open source visual servoing C++ library

This paper is related to the vision-based control of parallel robots. Indeed, a method is proposed to estimate the reduced set of kinematic parameters appearing in such a control. To do so, it extends a linear method, obtained for a perspective camera, to the case of an omnidirectional camera, using an existing unifying projection model. The proposed method remains linear, once adequate information is extracted from the omnidirectional images, and does not require any calibration pattern. It works both with perspective and omnidirectional cameras, which is underlined by the reported experiments

This paper presents a new hybrid decoupled vision-based control scheme valid for the entire class of central catadioptric sensors (including conventional perspective cameras). First, we consider the structure from motion problem using imaged 3D lines (conics). Polar lines of the principal point with respect to the conic curves are exploited to estimate a generic homography matrix from which a partial Euclidean reconstruction is obtained. The polar lines and the information extracted from the homography are then used to design a control law which allow us to fully decouple rotational motions from translational motions

In this paper, it is shown that computer vision, used as an exteroceptive redundant metrology mean, simplifies the control of a Gough-Stewart parallel robot. Indeed, contrary to the usual methodology, where the robot is modeled independently from the control law which will be implemented, we take into account that vision will be used for control, from the early modeling stage. Hence, kinematic modeling and projective geometry are fused into a control-devoted projective kinematic model. Thus, a novel vision-based kinematic modeling of such a robot is proposed through the observation of its legs. Inspired by the geometry of lines, this model unifies and simplifies both identification and control. Indeed, it has a reduced parameter set, and allows us to propose a linear solution to its calibration. Using the same model, a visual servoing scheme is presented, where the attitudes of the nonrigidly linked legs are servoed, rather than the end-effector pose. Finally, theoretical results concerning the stability of this control law are provided

In this paper, a vision-based measuring device is proposed and experimentally demonstrated to be an accurate, flexible, and low-cost tool for the kinematic calibration of parallel mechanisms. The accuracy and ease of use of the proposed vision sensor are outlined, with the suppression of the need for an accurate calibration target, and adequacy to the kinematic calibration process is investigated. In particular, identifiability conditions with the use of such an exteroceptive sensor are derived, considering the calibration with inverse or implicit models. Extensive results are given, with the evaluation of the measuring device and the calibration of an H4 robot. Using the full-pose measurement, an experimental analysis of the optimal calibration model is achieved, with study of the kinematic behavior of the mechanism. The efficiency of the provided method is thus evaluated, and the applicability of vision-based measuring devices to the context of kinematic calibration of parallel mechanisms is discussed.

This paper presents a novel approach for vision-based control of the end-effector of parallel mechanisms. It is based on the metrological redundancy paradigm, which simplifies their kinematic models by introducing additional proprioceptive sensors. By observing the mechanism legs, vision replaces advantageously these sensors by delivering, in a Cartesian frame, an exteroceptive measurement of the internal state of the mechanism. Formally, the latter is expressed by an original concept of vision-based kinematics for parallel mechanisms. Based on it, control is derived that visually servoes the direction of the legs, rather than the end-effector pose. The method is illustrated and validated on a Gough-Stewart platform simulation.

To close the loop between motion and vision, tracked visual features must remain in the camera field of view (visibility constraint). To overcome the visibility constraint, visual servoing methods can benefit from panoramic sensors such as catadioptric cameras (combining both mirrors and lenses). In this paper, we present a vision-based framework to control a nonholonomic mobile robot using a catadioptric imaging system. We particularly focus on a suitable catadioptric image-based control strategy of a nonholonomic robot in order to follow a 3D straight line. Such strategy can be applied to navigate in indoor or urban environment since the extraction and the tracking of straight lines are natural. First the control objectives are formulated in the catadioptric image space. The control law is then designed according to a well suited chained system for a mobile robot state vector directly expressed in the image space using a generic camera model. Simulation results illustrate the control strategy in the case of hypercatadioptric and paracatadioptric cameras.

In this paper, we present how the 2 1/2 D visual servoing scheme can be used with omnidirectional cameras. Motivated by the growing interest for omnidirectional sensors on robotic applications and particularly on vision-based control, we extend this framework to the entire class of central catadioptric systems. Indeed, conventional cameras suffer from restricted field of view. Central catadioptric systems have larger fields of view thus overcoming the visibility problem encountered when using conventional cameras. The 2 1/2 D visual servoing is based on the estimation of the partial camera displacement between two views, given by the current and desired images. Geometrical relationships are exploited to enable a partial Euclidean reconstruction by decoupling the interaction between translation and rotation components of a homography matrix. First we describe how to obtain a generic homography matrix for central catadioptric cameras from the projection model of an entire class of camera. Then the information obtained from the homography is used to develop a 2 1/2 D visual servoing scheme.

In this paper, a novel approach is proposed for the kinematic calibration of parallel mechanisms with linear actuators at the base. The originality of the approach lies in the observation of the mechanism legs with a camera, without any mechanism modification. The calibration can hence be achieved online, as no calibration device is linked to the end-effector, on any mechanism since no additional proprioceptive sensor installation is necessary. Because of the conditions of leg observability, several camera locations may be needed during the experimentation. The associated calibration method does not however require any accurate knowledge of the successive camera positions. The experimental procedure is therefore easy to perform. The method is developed theoretically in the context of mechanisms with legs linearly actuated at the base, giving the necessary conditions of identifiability. Application to an I4 mechanism is achieved with experimental results.

In this paper, an original approach is proposed for the kinematic calibration of parallel mechanisms. The originality lies in the use of vision to get information on all parts of the mechanism, i.e. its end-effector as well as its legs. Metrological redundancy is therefore maximized to improve the calibration efficiency. The approach is implemented for the calibration of the I4 parallel mechanism, with the use of the Jacobian matrix. No accurate camera location is needed so that the experimental procedure is easy to achieve. The calibration algorithm is detailed and experimentally demonstrated more efficient than other calibration methods based on legs observation or end-effector observation.

In this article, we present the kinematic calibration of H4 parallel robot using vision-based measuring device. Calibration is performed according to the inverse kinematic model method, using first the design model then a model developed for calibration purpose. With a precision of the order of magnitude of 0.2 mm and 0.03/spl deg/, our vision system allowed us to obtain a final positioning accuracy of the end-effector lower than 0.5 mm. Conclusions are given for the calibration of this class of mechanisms.

In this paper, a new pose selection criterion for the kinematic calibration of parallel mechanisms is proposed. It enables one to take into account the measurement noise amplification that may occur for parallel mechanisms, as well as the variation of amplitude and anisotropy of the measuring device accuracy. This new criterion is applied to vision-based calibration of an Orthoglide mechanism, both in simulation, with comparison to existing criteria, and experimentally.

In this article, an evaluation of a vision-based measuring system for parallel machine-tool calibration is performed. Simultaneous measurement of the 6 pose components enables one to perform calibration using the efficient inverse kinematic method. The system is composed of a single camera and a calibration board generated on a LCD monitor. Calibration board size can be adapted to the camera field of view, and specific points of interest can be generated, in order to improve pose measurement. Based on a single industrial camera, the measuring system is low-cost and easy-to-use. An experimental evaluation of the system is performed on a machine-tool axis. Measurement bias and precision are estimated by comparison with laser interferometry, and the influence of focal length and sensor resolution is examined.

We use an alternative formulation of the Euclidean Plucker coordinates to define the new normalized Plucker coordinates for alignment of lines. This concept is more relevant than usual image alignment to position a single calibrated camera with respect to a set of known 3D lines. An explicit control law is derived which realizes such a task both for one or several lines. It is tested by positioning the camera with respect to an orthogonal trihedron. This application is all the more challenging in that it requires depth to be observed. We do this by using the image projection of a laser pointer and adequately completing the control law.

We address visual servoing through a fixed stereo rig using an image Jacobian. Existing methods are based on the stacking of monocular servo image Jacobians, resulting in largely over-constrained control commands. In this paper we formally show that the epipolar constraint between two images can be taken into account explicitly. We then show that use of stereo significantly increases the quality of the servo task execution, especially where precision, robustness and smoothness of movement is concerned.