A flatness-based control approach which is implemented in successive loops is used to solve the control problem for the multivariable and nonlinear dynamics of industrial robotic manipulators and autonomous vehicles. The state-space model of these robotic systems is separated into two subsystems, which are connected between them in cascading loops. Each one of these subsystems can be viewed independently as a differentially flat system and control about it can be performed with inversion of its dynamics as in the case of input-output linearized flat systems. The state variables of the second subsystem become virtual control inputs for the first subsystem. In turn exogenous control inputs are applied to the first subsystem. The whole control method is implemented in two successive loops and its global stability properties are also proven through Lyapunov stability analysis. The validity of the control method is confirmed in two case studies: (a) control of a 3-DOF industrial rigidlink robotic manipulator, (ii) control of a 3-DOF autonomous underwater vessel.

1A nonlinear H-infinity (optimal) control approach is developed for the problem of the control of the spherical rolling robot. The solution of such a control problem is a nontrivial case due to underactuation and strong nonlinearities in the system's state-space description. The dynamic model of the robot undergoes approximate linearization around a temporary operating point which is recomputed at each timestep of the control method. The linearization relies on Taylor series expansion and on the computation of the system's Jacobian matrices. For the linearized dynamics of the spherical robot an H-infinity controller is designed. To compute the controller's feedback gains an algebraic Riccati equation in solved at each iteration of the control algorithm. The global asymptotic stability properties of the control method are proven through Lyapunov analysis. Finally, for the implementation of sensorless control for the spherical rolling robot, the H-infinity Kalman Filter is used as a robust state estimator.

This paper describes a whole new framework to quickly develop dynamic visual servoing systems embedded in an FPGA. Parallel design of the algorithms increases the precision of this kind of controllers, whereas minimizes the response time. Embedding dynamic visual servoing algorithms converts these systems into real-time systems. Additionally, an optimal control framework to dynamically visual control robot arms is proposed. The architecture re-configurability is tested with the implementation of two dynamic visual servoing controllers. One of the implemented controllers is derived from the proposed control framework and the other is the well-known image transpose Jacobian controller. From these experiments, FPGA resources utilization and timeline of the pipeline show the optimization of the proposed framework.

This paper presents a novel control approach to dynamic visual servo control multi-fingered robotic hands for the manipulation performance of an object. The control approach allows the path tracking of the object motion by means of visual information and takes into account both the dynamics model of the robot hand and the grasping force of the fingertips. In addition, an optimal approach is employed to obtain the desired behaviour in the joint space based on an indicated cost function which determines how the control effort is distributed over the robot fingers' joints. Finally, authors show experimental verifications on a robotic manipulation system for some of the visual servo controllers derived from the control approach.

Traditional visual servoing systems have been widely studied in the last years. These systems control the position of the camera attached to the robot end-effector guiding it from any position to the desired one. These controllers can be improved by using the event-based control paradigm. The system proposed in this paper is based on the idea of activating the visual controller only when something significant has occurred in the system (e.g. when any visual feature can be loosen because it is going outside the frame). Different event triggers have been defined in the image space in order to activate or deactivate the visual controller. The tests implemented to validate the proposal have proved that this new scheme avoids visual features to go out of the image whereas the system complexity is reduced considerably. Events can be used in the future to change different parameters of the visual servoing systems.

This paper describes the modelling, simulation and visual servo control of a robot-arm designed for supervision of complex task environments. This robot (mini-robot) has 2 degrees of freedom (hereafter d.o.f.) and it is attached at the end of another anthropomorphic robot with 7 d.o.f. (main robot). In order to achieve a suitable supervision, the mini-robot has to perform a given trajectory around the workspace where the task is developed. With the purpose of obtaining an accurate and fast supervision, this paper presents a novel tracking method carried out in the image space using direct visual servo control.

The depth perception in the objects of a scene can be useful for tracking or applying visual servoing in mobile systems. 3D time-of-flight (ToF) cameras provide range images which give measurements in real time to improve these types of tasks. However, the distance computed from these range images is very changing with the integration time parameter. This paper presents an analysis for the online adaptation of integration time of ToF cameras. This online adaptation is necessary in order to capture the images in the best condition irrespective of the changes of distance (between camera and objects) caused by its movement when the camera is mounted on a robotic arm.

In this paper, a new method to track image trajectories by visual servoing is proposed. This method solves the problem of the previous proposed time-independent tracking systems based on visual servoing. With the proposed method, the robot can track a previously generated trajectory affording a correct tracking not only in the image but also in the 3D space. This new method presents several improvements over the previous ones such as the possibility of specifying the desired tracking velocity, a less oscillating behavior or a correct tracking in the 3D space when high velocities are used. In order to demonstrate the correct behavior of the visual servoing system, an eye-in-hand camera system is used.

Calibration techniques allow the estimation of the intrinsic parameters of a camera. This paper describes an adaptive visual servoing scheme which employs the visual data measured during the task to determine the camera intrinsic parameters. This approach is based on the virtual visual servoing approach. However, in order to increase the robustness of the calibration several aspects have been introduced in this approach with respect to the previous developed virtual visual servoing systems. Furthermore, the system is able to determine the value of the intrinsic parameters when they vary during the task. This approach has been tested using an eye-in-hand robotic system.

This paper describes a method to detect changes in given surfaces tracked by the robot end-effector. To do so, first an approach to combine visual and force information is described. By filtering the interaction forces, a parameter which provides the probability of a change in the tracking surface is obtained. In order to obtain a more accurate determination of the change in the surface, information from a laser located at the robot end-effector is also employed. Therefore, a robust method for detecting surface discontinuities which employs force and visual information jointly with structured light is presented

In this paper, the so-called 2-D movement flow-based visual servoing system is proposed, which allows the tracking of trajectories planned in the image space. This method is time-independent, which is useful when problems such as obstructions occur during the tracking. This method allows the tracking of image trajectories assuring the correct behaviour in the 3-D space and avoiding the limitations of the time-dependent systems used up to now for tracking trajectories. This aspect has allowed its application to manipulation tasks in which the trajectory can be obstructed during the tracking.

A new approach for fusing visual and force information is shown. First, a new method for tracking trajectories, called movement flow-based visual servoing system, which presents the correct behavior in the image and in the three-dimensional space, is described. The information obtained from this system is fused with that obtained from a force control system in unstructured environments. To do so, a new method of recognizing the contact surface and a system for fusing visual and force information are described. The latter method employs variable weights for each sensor system, depending on a criteria based on the detection of changes in the interaction forces processed by a Kalman filter.