Visual servoing and vibration suppression of spatial flexible-link manipulators with a fixed camera setup are addressed in this article. The singular perturbation method is adopted to decouple the dynamic equations of the flexible manipulator; hence, two subsystems that represent the rigid robot motion and flexible-link vibration are obtained, respectively. Then, for the slow subsystem related to the rigid motion, an image-based controller is designed to converge the image errors with the consideration of compensating for the errors of approximating the Jacobian matrix. For the fast subsystem corresponding to the elastic vibration, to eliminate the requirements of measuring the vibration states, an observer is designed to estimate the fast states and then a feedback controller of the fast subsystem is presented to suppress the vibration of the flexible manipulator by using the estimation values. The closed-loop stabilities of the slow and fast subsystem are both proved by employing the Lyapunov theory. Numerical simulations demonstrate the effectiveness of the proposed controller, which shows that the image errors approach zero with the vibration of the flexible manipulator damped out simultaneously.

We consider the uncalibrated vision-based control problem of robotic manipulators in this work. Though lots of approaches have been proposed to solve this problem, they usually require calibration (offline or online) of the camera parameters in the implementation, and the control performance may be largely affected by parameter estimation errors. In this work, we present new fully uncalibrated visual servoing approaches for position control of the 2DOFs planar manipulator with a fixed camera. In the proposed approaches, no camera calibration is required, and numerical optimization algorithms or adaptive laws for parameter estimation are not needed. One benefit of such features is that exponential convergence of the image position errors can be ensured regardless of the camera parameter uncertainties. Generally, existing uncalibrated approaches only can guarantee asymptotical convergence of the position errors. Moreover, different from most existing approaches which assume that the robot motion plane and the image plane are parallel, one of the proposed approaches allows the camera to be installed at a general pose. This also simplifies the controller implementation and improves the system design flexibility. Finally, simulation and experimental results are provided to illustrate the effectiveness of the presented fully uncalibrated visual servoing approaches.

This article proposes a novel image-based visual servoing for a flexible aerial refueling boom with an eye-in-hand camera. The dynamic model of the flexible refueling boom is established by absolute nodal coordinate formulation (ANCF), after which the dynamic model is decomposed into a slow subsystem and a fast subsystem based on the singular perturbation approach. With respect to slow subsystem, the image feedback is used to control the flexible refueling boom so that the projection of the point marker on the back of the receiver converges to the desired position. With respect to fast subsystem, linear quadratic regulator (LQR) is applied to stabilize the vibration of the flexible refueling boom. The asymptotic convergence of the image error to zero is verified based on the Lyapunov theory. Simulation is used to demonstrate the effectiveness of the proposed method.

To make the controller implementation easier and to enhance the system robustness and control performance in the presence of the camera parameter uncertainties, it is very desired to develop vision-based control approaches without any offline or online camera calibration. In this article, we propose a new calibration-free image-based trajectory tracking control scheme for nonholonomic mobile robots with a truly uncalibrated fixed camera. By developing a novel camera-parameter-independent kinematic model, both offline and online camera calibration can be avoided in the proposed scheme, and any knowledge of the camera is not needed in the controller design. The proposed trajectory tracking control scheme can guarantee exponential convergence of the image position and velocity tracking errors. To illustrate the performance of the proposed scheme, experimental results are provided in this article. Note to Practitioners-This article was motivated by the vision-based motion control problem of mobile robots in uncalibrated environments. Existing vision-based motion control approaches for nonholonomic mobile robots generally depend on offline precise/coarse or online numerical/adaptive calibration of the camera intrinsic and extrinsic parameters and require precise or coarse knowledge of the camera in their implementation. This article presents a novel calibration-free image-based trajectory tracking control scheme, which can be implemented easily in real environments without any offline or online calibration of the camera parameters and can be used to efficiently control the motion of nonholonomic mobile robots with an arbitrarily placed and truly unknown overhead camera. Experimental results show that the proposed scheme can achieve satisfactory trajectory tracking control performance despite the lack of any knowledge about the camera intrinsic and extrinsic parameters and the presence of unknown camera lens distortions, and hence, can provide a simple but efficient solution to the vision-based motion control problem of nonholonomic mobile robots.

Although many vision-based control methods have been proposed for nonholonomic mobile robots, in their implementation, it is usually necessary to calibrate the camera intrinsic and/or extrinsic parameters using offline/online parameter estimation algorithms or online adaptation laws. To avoid the tediousness of camera calibration and to make the system performance highly robust to camera parameter uncertainties, in this article, we propose novel image-based pose stabilization control approaches for nonholonomic mobile robots with a truly uncalibrated overhead fixed camera. In the proposed approaches, only image position information of three feature points from an overhead camera is used for controller design, while information from other sensors (such as wheel encoders) is not required. Furthermore, either offline or online camera calibration is not necessary, and no knowledge about the camera intrinsic and extrinsic parameters is needed, which also can greatly simplify the controller implementation. Simulation and experimental results are given to demonstrate the feasibility and effectiveness of the proposed purely image-based pose stabilization approaches.

Place recognition and loop-closure detection are main challenges in the localization, mapping and navigation tasks of self-driving vehicles. In this paper, we solve the loop-closure detection problem by incorporating the deep-learning based point cloud description method and the coarse-to-fine sequence matching strategy. More specifically, we propose a deep neural network to extract a global descriptor from the original large-scale 3D point cloud, then based on which, a typical place analysis approach is presented to investigate the feature space distribution of the global descriptors and select several super keyframes. Finally, a coarse-to-fine strategy, which includes a super keyframe based coarse matching stage and a local sequence matching stage, is presented to ensure the loop-closure detection accuracy and real-time performance simultaneously. Thanks to the sequence matching operation, the proposed approach obtains an improvement against the existing deep-learning based methods. Experiment results on a self-driving vehicle validate the effectiveness of the proposed loop-closure detection algorithm.

Most existing formation control approaches are based on the assumption that the global/relative position and/or velocity measurements of mobile robots are directly available. To extend the application domain and to improve the formation control performance, it is extremely necessary to avoid the use of position and velocity measurements in the design of formation controllers. In this paper, we propose new leader-following formation tracking control schemes for nonholonomic mobile robots with onboard perspective cameras, without using both position and velocity measurements. To address the unavailability issue of position measurements, the leader-follower kinematics model in the image space is developed, which can facilitate the complete elimination of measurement/estimation of the position information. Furthermore, feedback information from the perspective camera of the follower robot is used to design adaptive observers to estimate the leader linear velocity for feedforward compensation, which can handle the absence of velocity measurements such that the proposed schemes can be applied to control formations of mobile robots without mutual communication abilities. By using the Lyapunov stability theory, a rigorous stability analysis based on the nonlinear formation dynamics is provided to show that the global stability of the combined observer-controller closed-loop system can be guaranteed. Both simulation and experimental results are also given to demonstrate the performance of the proposed formation tracking control schemes.

In this paper, the task-space cooperative tracking control problem of networked robotic manipulators without task-space velocity measurements is addressed. To overcome the problem without task-space velocity measurements, a novel task-space position observer is designed to update the estimated task-space position and to simultaneously provide the estimated task-space velocity, based on which an adaptive cooperative tracking controller without task-space velocity measurements is presented by introducing new estimated task-space reference velocity and acceleration. Furthermore, adaptive laws are provided to cope with uncertain kinematics and dynamics and rigorous stability analysis is given to show asymptotical convergence of the task-space tracking and synchronization errors in the presence of communication delays under strongly connected directed graphs. Simulation results are given to demonstrate the performance of the proposed approach.

In this paper, we will study the uncalibrated vision-based positioning problem of wheeled mobile robots by using a ceiling-mounted camera. A new image-based visual servoing scheme will be proposed, which can cope with the unknown intrinsic and extrinsic parameters of the camera and the uncertain distance parameter of the feature point from geometric center of the mobile robot. The presented approach is developed via extending the depth-independent interaction matrix framework for robot manipulators to mobile robots such that the nonlinear dependence on unknown parameters can be removed from the image-Jacobian matrix and then, we can linearly parameterize uncertain parameters in the closed-loop system. In this way, an estimation scheme for the online updating of uncertain parameters can be developed very efficiently. To show that image errors can be guaranteed to be asymptotically convergent, stability analysis will be carried out by using Lyapunov theory. To validate the performance of the presented approach, simulation and experimental results will also be provided.

As we know, the dynamic visual-tracking control, which is based on the depth-independent interaction matrix, has been proposed to cope with the general 3-D motion of robot manipulators. To deal with the unknown camera parameters, an adaptive law has been designed. It is noted, however, that the designed adaptive law uses a vector signal defined by the true depths of feature points, which are unavailable when the camera parameters are unknown. Nevertheless, such an adaptive law can still be implemented by replacing the exact depths with the estimated depths, since the estimated depths can be readily calculated by using the estimated values of the camera parameters. In this case, however, we cannot definitely say whether or not the robot system can be guaranteed to achieve asymptotical stability or even stable behavior. To overcome this problem, in this paper, we redefine the mentioned vector signal using the estimated depths and show that the design based on the new vector signal can make the robot system asymptotically stable. Additionally, the existing tracking control design based on the concept of depth-independent interaction matrix is initial-state dependent. Thus, in this paper, we also modify the existing design to obtain an initial-state-independent result. To show the performance of the proposed designs, simulation results based on a two-link planar manipulator are presented. In addition, preliminary experimental results using an industrial manipulator are also given.