Due to the presence of strong inner dynamic coupling and changes in center of mass (CoM) during tasks execution, the precise tracking control problem of aerial manipulator systems becomes challenging. When the mounted manipulator is performing a task, the movements of the manipulator will cause the inherent dynamic coupling force/torque disturbances. Such disturbances are quickly acted on the position loop and the orientation loop of the UAV body, to the detriment of the control accuracy. On the other hand, the fluctuation of the UAV body leads to the base floating, resulting in a significantly adverse influence on the precision of the manipulator end-effector. Since different disturbances have distinct mathematical properties, i.e., norm bounded and rate bounded, currently, there is no control framework that is able to tackle the dynamic coupling, model uncertainty and base floating simultaneously. In this paper, a refined anti-disturbance control architecture is proposed for the decentralized aerial manipulator model, where various disturbances with different mathematical properties are well explored and tackled according to their positions and effects acting on the system. The stability of the proposed control framework is ensured by using Lyapunov-like analysis. Experimental results are presented to illustrate the performance of the proposed control framework.Note to Practitioners—One of the key challenges that hinder the aerial manipulator potential applications is its stability and accuracy due to the existence of various disturbances. Most of disturbance rejection control methods in the literature for aerial manipulator always deal with the various disturbances as the lumped one. Their distinct mathematical properties and different impacts on the system are not well explored, which may result in the performance degradation and limit its practical implementation. In this article, a refine anti-disturbance architecture is proposed, which is able to handle various disturbances in a more systematical way according to their mathematical properties and effects acting on the system. Physical experiments suggest that finely tackling the different disturbances enjoys a better performance in aerial manipulator trajectory tracking control problem and thus can promote the aerial manipulator to be deployed in the tasks demanding on the accuracy. In addition, such control strategy can also be extended to other robotic systems suffering from various disturbances.

Theoretical analysis of transfer learning of the deep neural networks (DNN) is crucial in ensuring stability or convergence and gaining a better understanding of the networks for further development. However, most current transfer learning methods are black-box approaches that are more focused on empirical studies. This paper develops a transfer learning algorithm for deep convolutional neural networks (CNN) with batch normalization layers. A convergence-guaranteed transfer learning algorithm is proposed to train the classifier of a deep CNN with pretrained convolutional layers. Two classification case studies based on VGG11 with the MNIST dataset and CIFAR10 dataset are presented to demonstrate the performance of the proposed approach and explore the effect of batch normalization layers on transfer learning,

In past decades, much progress has been obtained in vision-based robot control theories with traditional image processing methods. With the advances in deep-learning-based methods, convolutional neural network (CNN) has now replaced the traditional image processing methods for object detection and recognition. However, it is not clear how the CNN-based methods can be integrated into robot control theories in a stable and predictable manner for object detection and tracking, especially when the aspect ratio of the object is unknown and also varies during manipulation. In this article, we develop a vision-based control method for robots with an eye-in-hand configuration, which can be directly integrated with existing CNN-based object detectors. The task variables are generated based on parameters of the bounding box from the output of any real-time CNN object detector such as you only look once (Yolo). To address the chattering problem of bounding box, long short-term memory (LSTM) is used to provide smoothed bounding box information. A vision-based controller is then proposed following task-space motion control design formulation in order to keep the object of unknown aspect ratio in the center of field of view of the camera. The stability of the overall closed-loop control system is analyzed rigorously using the Lyapunov-like approach. Experimental results are presented to illustrate the performance of the proposed CNN-based robot controller.

Traditional applications of robot manipulators are mainly limited to tracking control tasks, in which the desired objectives are specified as desired positions or trajectories. In such applications, a convenient and easy way of programming robots is the traditional teach-and-playback method. However, advances in sensing and robotic technologies have led to the requirements of more demanding tasks, in which robots may need to interact with a human or follow the human’s instructions in performing a sequence of more complex tasks. In such applications, it is not sufficient to just learn the positions or motion and play it back using a robot controller. In this article, a task learning approach is proposed for human–robot interaction systems, where a set of interaction behaviors is formulated and solved by specifying the task requirements in terms of potential energy. The motion behaviors demonstrated by humans can thus be acquired by the robot by seeking the appropriate task parameters of the dynamic potential energy function. To play back and combine the tasks in a sequential way by using a single controller, a new robot controller is also proposed. Lyapunov-like analysis is adopted to guarantee the stability of the control system, and experimental results are given to validate the performance of the proposed learning and control strategies.

The inner/low-level control loop of most industrial robotic manipulators is protected from any modification by a closed control architecture. The only way to specify joint inputs to them is through position or velocity commands. Furthermore, the inner controller is unknown/uncertain, as it is not revealed to the user. This makes it very difficult to determine the configuration of the inner controller, including the structure and values of the control gains. As a result, integrating external sensory feedback systems into a closed architecture system becomes a difficult task because the interaction of the external feedback loop with the inner control loop can affect the stability of the overall system. In this paper, a stable control strategy is proposed to generate the joint velocity commands for industrial robotic manipulators with uncertain closed control architecture. Based on the proposed method, external feedback controllers can be added regardless of the inner control loop configuration. The proposed external (or outer) control loop approach provides a high degree of design flexibility by enabling smooth implementation of various modern control applications on industrial robots. Unlike previous studies, the proposed control method is not limited to a particular configuration of the controller in the inner loop, nor to the structure of its control gains. The proposed controller design is validated on the UR 5e industrial manipulator.

Optical tweezers have received significant attention in the past few decades and many techniques have been proposed to achieve assorted manipulation tasks on cells or micro-objects. While traditional techniques for optical tweezers utilize laser beams for direct trapping and manipulation of cells, several approaches have also been proposed to achieve grasping and manipulation of cells. Most grasping approaches, however, have failed to solve the existing difficulties in the grasping process of cells due to limited sensing capability and presence of Brownian perturbation in the micro world. In this paper, a novel human-assisted grasping technique for manipulation of biological cells is proposed to facilitate the grasping process of biological cells. In the proposed technique, several microbeads are first trapped and used as fingertips to perform a grasping task on a biological cell. Based on human observation, the positions of the laser beams that trap the microbeads are remotely and simultaneously controlled through a touchscreen, and thus generating a grasping formation of the trapped microbeads to grasp the cell. After the cell has been grasped, a simple region control technique is utilized for automated manipulation of the grasped cell. This paper offers a flexible, reliable and efficient approach for grasping and manipulation of biological cells, and thus extending the feasible applications of optical tweezers. The stability of the control system for manipulation of the grasped cell is investigated by using a Lyapunov approach, and the feasibility of the proposed human-assisted grasping and manipulation technique is demonstrated by experiment result.

Most industrial optical tweezers have a closed control architecture which restricts any modification to its inner/low-level control loop. As a result, the advanced robotic controller that requires an open low-level control loop cannot be implemented directly on the optical tweezers system with closed architecture. To resolve this issue, we propose an adaptive outer loop control scheme to generate the velocity (or position) command for the low-level control loop. We introduce an outer feedback loop to the closed architecture system so that the velocity (or position) command can be effectively realized on industrial optical tweezers in a stable manner. Thus with this design, the outer user-specified control loop is totally independent of the fixed inner control loop. We demonstrate the performance of the proposed approach through numerical simulations.

Modeling is an important task in classic control system design. However, as the robotics systems are getting more complex, the modeling tasks using fundamental physical principles are becoming more difficult. One of the emerging approaches to avoid direct modeling is the data-driven techniques in which measurement data are collected, extracted and analyzed by some algorithms for the purpose of modeling and controlling the robots. In this paper, we present a data-driven technique that employs neural network (NN) to approximate robot kinematics without knowing the robot structure. The convergence of the algorithm is rigorously analyzed. Simulation results are presented to illustrate the performance of the proposed algorithm.

A high confidence in object detection is very crucial for object detector modules to be used in real world applications. Though the confidence scores in object detection can be improved by using better and larger training data set and using more robust architectures, which is an approach from the computer vision side, this paper aims to improve the detection confidence by finding the effective viewpoints which makes a dependable use of the available object detectors. In particular, we investigate the effect of viewing distance on detection confidence of an object detector, which can further be used to control a camera mounted on a mobile robot to approach a better viewing position. We consider the cases with both fixed focal length and variable focal length camera. Experimental results are presented to demonstrate the efficacy and validity of the techniques presented in this work.

In this paper, we propose a shared control approach for optical manipulation of biological cells. A robotic control methodology is developed that allows association between human and an autonomous optical cell manipulation system. Human, when applicable or necessary, is able to associate with an autonomous cell manipulation system in a stable manner, and thus guiding the system to perform the operation task effectively. While current optical cell manipulation systems are only capable of performing either autonomous task or manual operation, the proposed approach offers an opportunity for human to associate with an autonomous optical cell manipulation system, when applicable or necessary, so as to deal with complex operation tasks. The stability of the shared control system is analyzed with a Lyapunov-like method, and the effectiveness of the proposed approach is validated with experimental results.

This paper proposes an autonomous grasping technique for micro-objects using flexible micro-hands. In this control technique, a micro-hand is constructed by several fingertips which are optically trapped and actuated by using laser beams. As the fingertips are driven towards a transformable area specified around a target micro-object, a grasping configuration of fingertips is thus formed to grasp the object. In this control technique, the effect of the Brownian perturbations on maneuvering of the micro-hand is considered so as to gain insight into the real nature of micro-manipulation using optical tweezers. The stability of the control system for autonomous grasping using flexible micro-hands is investigated with a stochastic analysis, and experimental illustration is presented to demonstrate the effectiveness of the proposed control technique.

The effect of Brownian motion on maneuvering of micro-objects in a fluid medium is one of the fundamental differences between micro-manipulation and robotic manipulators in the physical world. Besides, due to the limitation of feasible sensors and actuators in micro-manipulation, current control techniques for manipulation of micro-objects or cells are mostly dependent on the physical properties of target micro-objects or cells. In this paper, we propose the first stochastic control technique to achieve simultaneous orientation and transportation of micro-objects with Brownian perturbations. Several micro-particles which are optically trapped and driven by laser beams are utilized as fingertips to first grasp a target micro-object. Cooperative control of robot-assisted stage and the fingertips is then performed to achieve the control objective, in which the target micro-object is transported toward a desired position by using the robot-assisted stage, and at the same time, it is oriented toward a desired angular position by using the fingertips. This paper provides a stochastic control framework for simultaneous orientation and transportation of micro-objects with arbitrary types in the micro-world, and thus bringing micro-manipulation using optical tweezers closer to robotic manipulation in the physical world. Rigorous mathematical formulation and stability analysis for simultaneous orientation and positioning feedback control of micro-objects in the presence of the stochastic perturbations are derived, and experimental results are also presented.

Existing control systems for optical manipulation of multiple micro-objects do not allow users to interact with the systems during manipulation to deal with unexpected events, while ensuring stability of the overall control systems. In this paper, we propose a robotic control technique for human-guided optical manipulation of multiple micro-objects using robotic tweezers. Humans, when necessary, are able to interact or intervene with an automated optical manipulation system in a stable manner, and thus guiding a group of micro-objects to be manipulated towards a desired region while ensuring collision avoidance during manipulation. Both the ability of humans, in term of decision making, when needed, and the advantages of an automated optical manipulation system which provides precise and productive manipulation of micro-objects, are consolidated into one single technique. A theoretical foundation is developed and investigated to achieve the control objective. Experimental results are presented to illustrate the effectiveness of the proposed control technique.

Existing control techniques for optical trapping of cells or micro-objects can only perform either positioning control or separate control of orientation and position in a sequential manner. In this paper, we propose a robotic control technique to achieve simultaneous orientation and positioning control of a microscopic object using multiple laser-driven fingertips and robotic motorized stage control. Several optically trapped micro-particles are first utilized as the laser-driven fingertips to grasp a target micro-object. Simultaneous control of the laser-driven fingertips and the robotic motorized stage is then performed to achieve the control objective, in which the target object is oriented to a desired angular position by controlling the laser-driven fingertips, and at the same time, it is manipulated to follow a desired trajectory by maneuvering the robotic stage. Rigorous mathematical formulation and solution are developed for simultaneous orientation and positioning control of a micro-object using optical tweezers.

Current control techniques for optical manipulation of multiple micro-objects employ laser tweezers to directly trap and manipulate the target microscopic objects, and thus it is not capable of coordinating and manipulating micro-objects with arbitrary types in the micro-world, including large micro-objects, laser sensitive biological cells, or optically untrappable ones. In this paper, we propose a robotic control technique for optical manipulation of multiple microscopic objects by using micro-hands with multiple fingertips. In this control technique, several micro-hands are first formed by coordinating multiple optically trapped micro-particles that serve as the laser-driven fingertips, and then utilized for grasping and coordinative manipulation of multiple micro-objects. The proposed technique offers a robotic control framework for coordination and optical manipulation of multiple microscopic objects with arbitrary types in the micro-world, including large micro-objects, lasersensitive cells, or even untrappable microscopic objects.

In network systems with a huge number of nodes, it is not possible to apply input signals to all network nodes to control them. In this paper, we show that this issue can be addressed by designing a network topology so that the nodes in the network system are controllable by a few nodes in the system. A theoretical framework that provides the basic link between structural controllability of network systems and the topology design problem is developed. The results also shed light on how new nodes can be added to the network system without having to introduce new control nodes. Hence, the results are useful in dealing with topology design to obtain a controllable network. Moreover, the results also show under what circumstances a network system with multiple identical nodes is uncontrollable. In many applications, groups of identical nodes are connected to each other which is called network of groups. Here, we address the structural controllability problem for multiple groups of network systems which provides information on proper topology design at both network level (i.e., interconnection of groups) and node level (i.e., interconnection of nodes within a group).

In this paper, we propose an integrated optimal planning and control technique for optical manipulation of a group of cells beyond the field of view. The proposed technique consists of a planning layer where a model predictive control is utilized to obtain a virtual input for manipulation of cells, and a control layer where the desired inputs for laser beams and motorized stage are obtained by means of the control allocation technique. Unlike existing methods on simultaneous manipulation of laser and motorized stage which the desired inputs for lasers and stage are given separately, here, the planning and control layers are integrated together to obtain optimal inputs for both laser and motorized stage. Finally, experimental results are presented to illustrate the performance of the proposed method.

Several control schemes have been proposed for optical tweezers, but most existing methods assume that the optical trapping is maintained throughout the manipulation, and Brownian motion is ignored for the simplification of stability analysis. However, the optical trapping is not effective when a biological cell is initially outside the optical trap, and even if the cell is initially trapped, it may escape from the optical trap due to random Brownian perturbations and kinetic energy gained during manipulation. This paper presents a new robotic control technique to address the trapping and manipulation problem of biological cell in presence of random Brownian perturbations. By using the proposed method, the cell can be automatically trapped when it is not inside the optical trap and the optical manipulation of cell is enhanced by saturating the position feedback. The stability of the closed-loop system is analysed from stochastic perspectives. Experimental results are presented to illustrate the performance of the proposed control method.

In existing control techniques for optical manipulation, the effect of the Brownian motion is usually disregarded for ease of analyzing the control systems. However, due to the prevalence of the Brownian motion and its effect on micro-manipulation of tiny objects, it is important for the Brownian effect to be properly taken into account so as to ensure the performance and stability of the control systems. This paper proposes a robotic control technique to achieve automated manipulation of multiple microscopic objects with the consideration of the Brownian perturbations. In this control technique, a desired dynamic region is first initiated to enclose all the trapped micro-objects. The trapped objects are then manipulated as a whole while being kept inside the desired region, and at the same time, avoiding collisions between each other during the course of manipulation. Rigorous mathematical formulation is developed to analyze the control system for automated manipulation of multiple micro-objects with stochastic perturbations. Experimental results are presented to illustrate the performance of the proposed control technique.

Automating item picking in the e-commerce warehouse is pressing but challenging, due to a massive variety of items, tight environmental constraints and item location uncertainty. In this paper, we present an effective and efficient strategy-based planning approach to implement the robotic picking from shelves for e-commerce. Making full advantage of a gripper with multiple securing methods, differentiated strategies are modeled as picking primitives with different securing methods. A strategy generator is proposed to produce feasible potential pickings as quickly and as successfully as possible. A strategy evaluator considering reachability, collision, object-bias preference and the securing performance is also presented for ranking the picking strategies. Experiments were conducted to validate that the robotic picker is able to plan a picking strategy within 2 ms and pick daily items from the shelves with an average success rate of 68%.

Micromanipulation has received increasing attention from robotics researchers due to its wide applications in the manipulation of microobjects like biological cells and Bio-MEMS components. The demand for accurate and precise manipulation of microobjects opens up new challenges in automation of micromanipulation tasks. In this paper, we present a concurrent framework for optical manipulation of multiple groups of microobjects using robotic tweezers. The proposed framework is based on laser-stage coordination control and consists of two concurrent subschemes: 1) local coordination achieved by asynchronous manipulation of multiple groups of microobjects using laser beams and 2) global coordination achieved by manipulation of whole groups using a motorized stage. Unlike existing methods that are limited to the manipulation of a single microobject or a single group of microobjects, the proposed method considers concurrent laser-stage coordination of multiple groups of microobjects, which enhances the capability and flexibility in micromanipulation tasks. In addition, we introduce a unified social interaction function to achieve various cellular behaviors. A mathematical formulation is provided and stability analysis is presented. Using the proposed method, we are able to manipulate multiple groups of microobjects to construct time-varying microformations. Experimental results are presented to illustrate the performance of the proposed method.

Various automatic manipulation techniques have been developed for manipulating micro-objects using optical tweezers. Because of the small trapping force of optical traps and increase in kinetic energy during manipulation, a trapped object may not remain trappable, especially in the presence of random Brownian perturbation. However, there is no theoretical analysis so far to help understand the effects of dynamic motion and Brownian forces on the trappability problem of optical tweezers. This paper investigates the optical manipulation of micro-objects under random perturbations. Here, we provide for the first time a theoretical and experimental analysis of the dynamic trapping problem from stochastic perspectives. We derive the relationship between trapping probability and maximum manipulation velocity. A controller with appropriate velocity bound is then proposed to ensure that the system is bound and stable. The experimental results confirm the accuracy of our theoretical analysis and illustrate the necessity and usefulness of the proposed controller.

Existing control techniques for optical tweezers utilize a single focused laser beam to directly trap and manipulate a target cell. However, a typical force generated by an optical trap is extremely small (few pico-newtons) and therefore it is not sufficient to manipulate a larger cell or object. The optical trap is also sensitive to the shape of the biological cell and the refractive index. Therefore, current automatic control techniques for optical tweezers cannot be used to manipulate a large cell or cell with irregular shape. In addition, excessive irradiation of the laser beam to the cell may also cause photodamage of even lead to death of the cell. In this paper, we propose a robotic control technique for optical tweezers to achieve automated manipulation of cell, which is beyond the capability of a single optical trap. First, multiple laser beams are generated, and each laser beam is used to trap one micro-particle to create a formation around the target cell to hold it. Then the target cell is manipulated to a desired position by controlling the motorized stage. The proposed control technique is particularly suitable for automated manipulation of sensitive biological cells, cells with large size or cells of irregular shape. Rigorous mathematical formulations have been developed to analyze the control system for automated cell manipulation. Experimental results are presented to illustrate the performance of the proposed controller.

Microbiology is mainly concerned with the study of coexistence, cooperation and interaction among groups of microorganisms. In order to study the biological systems consist of interacting groups of microorganisms, optical manipulation can be utilized as a useful tool. In fact, the ability to manipulate several groups of microorganisms is an essential step towards studying how biological microorganisms communicate and cooperate to perform a wide range of multicellular behaviours. In addition, decomposition of a group of cells into smaller groups gives capability and flexibility in manipulation tasks and remarkably increases environmental adjustability. In this paper, we aim to provide a primary result in micromanipulation of multiple groups of cells. We develop a control methodology for dynamic coordination of multiple groups of cells. A rigorous mathematical formulation is provided and the stability analysis is presented. Using the proposed method, we are able to manipulate multiple groups of microparticles to construct time-varying micro-formations. The proposed method is also useful in examination of the interactions between several groups of living organisms in the desired inter-cellular distance. Experimental results are presented to illustrate the performance of the proposed method.

Various control approaches have been developed for micro/nanomanipulations using optical tweezers. Most existing methods assume that the micro/nanoparticles stay trapped during manipulations, and stochastic perturbations (Brownian motion) are usually ignored for the simplification of model dynamics. However, the trapped particles could escape from the optical traps especially in motion due to several possible reasons: small trapping stiffness, stochastic perturbations, and kinetic energy gained during manipulation. This paper investigates the conditions under which micro/nanoparticles will stay trapped while in motion. The dynamics of the trapped particles subject to stochastic perturbations is analyzed. Dynamic trapping is considered and the maximum manipulation velocity is determined from a probabilistic perspective. A controller with certain velocity bound is proposed, the stability of the system is analysed in presence of stochastic perturbation. Experimental results are presented to show the effectiveness of the proposed control approach.

Few comparisons have been performed across torque controllers for exoskeletons, and differences among devices have made interpretation difficult. In this study, we designed, developed and compared the torque-tracking performance of nine control methods, including variations on classical feedback control, model-based control, adaptive control and iterative learning. Each was tested with four high-level controllers that determined desired torque based on time, joint angle, a neuromuscular model, or electromyography. Controllers were implemented on a tethered ankle exoskeleton with series elastic actuation. Measurements were taken while one human subject walked on a treadmill at 1.25 m·s−1for one hundred steady-state steps. The combination of proportional control with damping injection and iterative learning resulted in the lowest errors for all high-level controllers. With time-based desired torque, root-mean-squared errors were 0.6 N·m (1.3% of peak desired torque) step by step, and 0.1 N·m (0.2%) on average. These results indicate that model-free, integration-free feedback control is suited to the uncertain dynamics of the human-robot system, while iterative learning is effective in the cyclic task of walking.

Each year, stroke and traumatic brain injury leave millions of survivors with motion control loss, which results in great demand for recovery training. The great labor intensity in traditional human-based therapies has recently boosted the research on rehabilitation robotics. Existing controllers for rehabilitative robotics cannot solve the closed-loop system stability with uncertain nonlinear dynamics and conflicting human–robot interactions. This paper presents a theoretical framework that establishes the passivity of the closed-loop upper-limb rehabilitative robotic systems and allows rigorous stability analysis of human–robot interaction. Position-dependent stiffness and position-dependent desired trajectory are employed to resolve the possible conflicts in motions between patient and robot. The proposed method also realizes the “assist-as-needed” strategy. In addition, it handles human–robot interactions in such a way that correct movements are encouraged and incorrect ones are suppressed to make the training process more effective. While guaranteeing these properties, the proposed controller allows parameter adjustment to provide flexibility for therapists to adjust and fine tune depending on the conditions of the patients and the progress of their recovery. Simulation and experiment results are presented to illustrate the performance of the method.

Multi-cellular aggregation is a fundamental phenomenon observed in many biological processes. Investigating cells aggregation helps us to have better understanding of many biological processes. Moreover, it is useful in finding cure for the diseases caused by cells aggregation. In this paper, we present a control methodology to obtain multi-cellular aggregation by using multiple optical trapping. The proposed method is also useful for study of multiple cell fusion which is an important cellular process. Experimental results are presented to show the effectiveness of the proposed method in achieving multi-cellular aggregation.

In this paper, a tracking control scheme is proposed for optical manipulation of biological cell with unknown trapping stiffness. The requirement on the model of the trapping stiffness is eliminated in the proposed formulation and thus system identification and calibration are not needed. The unknown trapping stiffness and the uncertain dynamic parameters are estimated separately, with on-line update laws. By using the proposed control scheme, the laser beam is able to manipulate the trapped cell to track various time-varying trajectories, to suit different applications in cell manipulation. The proposed control scheme is based on the dynamic formulation where the position of laser beam is controlled by closed-loop robotic manipulation techniques. The stability of the overall system is analyzed by using Lyapunov-like method, with consideration of the dynamics of both the cell and the manipulator of laser source. Experimental results are presented to illustrate the performance of the proposed tracking controller with unknown trapping stiffness.

Emerging applications of robot systems that involve close physical interaction with human have opened up new challenges in robot control. For these applications, it is important to consider the stability and coordination of human-robot interaction. While various control techniques have been developed for human-robot interaction, existing methods do not take the advantages of human ability in responding and adapting to unknown environment. In this paper, a human-guided manipulation problem which is able to take advantages of both the human knowledge and the robot's ability, is formulated and solved. The workspace is divided into a human region, where human play a more active role in the manipulation task, and a robot region, where the robot is more dominant in the manipulation. The proposed formulation allows the involvement of human control action to deal with unforeseen changes or uncertainty in the real world. We present a theoretical foundation that allows the stability and coordination of the human-guided manipulation problem to be analyzed. Based on the human region and the robot region, an adaptive tracking controller is developed. Experimental results are presented to illustrate the performance of the proposed control method.

Microscopic optics and cameras are commonly used in micromanipulation or biomanipulation workstations since they provide a large spectrum of visual details and information. The visual feedback information also improves robustness to uncertainty and accuracy of micromanipulation. Among various micromanipulation systems, optical tweezers are one of the most useful instruments that utilize a focused beam of light to manipulate biological cell or nanoparticles without physical contact. However, current optical manipulation techniques fail if the laser beam is not within the field of view (FOV) of the microscope. To solve this problem, we present a robotic control technique for optical manipulation with limited FOV of microscope. The proposed control strategy consists of a vision based control that manipulates the trapped cell to move to a desired position inside the FOV and a Cartesian-space feedback control that drives the laser beam back when it is outside the FOV. Thus, the proposed method allows the laser beam to leave the FOV during the course of manipulation and the transition from one feedback to another is smooth. The stability of the closed-loop system is analysed by using Lyapunov-like methods, with consideration of the dynamic interaction between the cell and the manipulator of the laser source. Experimental results are presented to illustrate the performance of the proposed method.

While several automatic manipulation techniques have recently been developed for optical tweezer systems, the measurement of the velocity of cell is required and the interaction between the cell and the manipulator of laser source is usually ignored in these formulations. Although the position of cell can be measured by using a camera, the velocity of cell is not measurable and usually estimated by differentiating the position of cell, which amplifies noises and may induce chattering of the system. In addition, it is also assumed in existing methods that the image Jacobian matrix from the Cartesian space to image space of the camera is exactly known. In the presence of estimation errors or variations of depth information between the camera and the cell, it is not certain whether the stability of the system could still be ensured. In this paper, vision-based observer techniques are proposed for optical manipulation to estimate the velocity of cell. Using the proposed observer techniques, tracking control strategies are developed to manipulate biological cells with different Reynolds numbers, which do not require camera calibration and measurement of the velocity of cell. The control methods are based on the dynamic formulation where the laser source is controlled by the closed-loop robotic manipulation technique. The stability is analyzed using Lyapunov-like analysis. Simulation and experimental results are presented to illustrate the performance of the proposed cell manipulation methods.

Research on rehabilitation robotics has been rising as a substitute to human practice to help neuro-damaged patients to restore impaired or lost functionalities. Most control methods for rehabilitative robotics do not consider the closed-loop system stability in presence of uncertainty of nonlinear dynamics, and conflicting movements between patient and robots. In this paper, we present a theoretical framework which allows rigorous stability analysis of human-robot interaction in rehabilitative robotic system. Position-dependant stiffness and desired trajectory are proposed to resolve the possible conflicts in motions between patient and robot. The proposed method also realizes the assist-as-needed policy and possesses the ability to be customized for operations during different stages of patient recovery. In addition, the proposed controller handles human-robot interactions in such a way that correct movements are encouraged and incorrect ones are suppressed to make the training process more effective. Experimental results are presented to illustrate the performance of the controller.

In region based shape control of multi-agent systems, the agents move together as a group inside a desired region while maintaining minimum distance among themselves. The desired formation is specified as the desired region for the whole group instead of desired trajectories for individual agents. One limitation of region based shape formation method is the necessity of access to the desired reference of the region i.e. desired movement of the entire group. This paper presents a distributed shape formation control method for multi-agent systems. We develop a state estimator for each agent to construct the desired reference based on local information. A Lyapunov-like function is proposed to examine the stability of the overall system. Simulation results are presented to illustrate the performance of the proposed method in distributed shape formation.

In this paper, an observer based adaptive control method is proposed for optical manipulation of cell. The dynamics of the robotic manipulator of the laser source is introduced in the optical tweezers system, so that a closed-loop control method is formulated and solved, and a backstepping approach is used to derive a control input for the manipulator. The interaction between the cell dynamics and the manipulator dynamics leads to a fourth-order overall dynamics, and hence a nonlinear observer is constructed to avoid the use of high-order derivatives of the positions in the control input. Stability of the closed-loop system is analyzed by using Lyapunov-like analysis. Simulation results are presented to illustrate the performance of the proposed control methods.

Task-space sensory feedback information such as visual feedback is used in many modern robot control systems as it improves robustness to model uncertainty. However, existing sensory feedback control schemes are only valid locally in a finite task space within a limited sensing zone where singularity of the Jacobian matrix is avoided. In this paper, the global stability problem of task-space sensory feedback control system is formulated and solved. The proposed method is based on multiple regional feedback information where each feedback information is employed in a local region. The combination of the local feedback covers the entire workspace and thus guarantees the global movement of the robot. In addition, the switching from one feedback information to another is embedded in the controller without using any hard or discontinuous switching. Experimental results are presented to illustrate the performance of the proposed controller.

Current manipulation techniques of optical tweezers treat the position of the laser beam as the control input and an open-loop kinematic controller is designed to move the laser source. In this paper, a closed-loop robotic control method for optical tweezers is formulated and solved. While robotic manipulation has been a key technology driver in factory automation, robotic manipulation of cells or nanoparticles is less well understood. The proposed formulation shall bridge the gap between traditional robot manipulation techniques and optical manipulation techniques of cells. A dynamic region controller is proposed for cell manipulation using optical tweezers. The desired objective can be specified as a dynamic region rather than a position or trajectory, and the desired region can thus be scaled up and down to allow flexibility in the task specifications. Experimental results are presented to illustrate the performance of the proposed controller.

Singularity issue has been a long standing problem in task-space control of robot. It is commonly assumed in the theoretical analysis of task-space control system that the robot is operating in a finite task space such that singularity problem can be avoided. This limits the potential workspace of the robot when task-space control is employed. In this paper, a singularity-robust task-space controller is proposed for tracking control of robot manipulator. The proposed controller consists of a joint-space position controller that is activated when the robot is near singular configurations, and a task-space tracking controller that is used when the robot leaves the singular region. Therefore, the robot can start from the singular regions and transit smoothly from joint space to task space. It can also enter the singular region during the course of movement. The stability of the closed-loop system is analyzed with consideration of the singularity issue. Experimental results are presented to illustrate the performance of the proposed controller.

In this paper, we consider leader-based shape coordination for multi-robot systems. It is shown that having knowledge about desired velocity and effective interaction among individuals can help group members to continue the path even after leader failure. Since any collective behavior emerges from interaction among individuals, we propose interactive force to maintain minimum distance among agents as well as group unity during movement even after leader failure. Simulation results are presented to illustrate the performance of proposed method.

This paper presents an adaptive region tracking control for Autonomous Underwater Vehicle (AUV). The AUV is required to track a moving region to accomplish a given task. The desired target is specified as a region rather than a point so that the control effort used to track the region is minimal. In the applications where the accuracy is of utmost importance, the desired region can be chosen to be small so that the precision is not lost. The desired region can be scaled up or scaled down so that the AUV can adjust its position to suit the applications. A Lyapunov-like function is presented for the stability analysis. Simulation results on AUV with 6 degrees of freedom are presented to demonstrate the effectiveness of the proposed controller.

It is interesting to observe from human visually guided tasks that visual feedback is not used for the entire movement, but only at end phases when our hand is near the target. We are able to move our hand from an initial position that is not within our field of view and transit smoothly and easily into visual feedback when the target is near. Inspired by this natural action, this paper presents a new task-space adaptive controller with dual feedback information. The proposed controller consists of a Cartesian-space region reaching controller at the initial stage and a vision based tracking controller that is only activated when the end effector enters an image region. A new potential energy function is proposed such that the image region can be fixed as the field of view of the camera and does not have to vary with the desired trajectory. The proposed task-space controller can transit smoothly from Cartesian-space reaching to vision-space tracking control. The stability of the closed-loop system is analyzed with consideration of the nonlinear dynamics. The proposed adaptive controller is implemented on an industrial robot and experimental results are presented to illustrate the performance of the proposed controller.

In conventional task-space control problem of robots, a single task-space information is used for the entire task. When the task-space control problem is formulated in image space, this implies that visual feedback is used throughout the movement. While visual feedback is important to improve the endpoint accuracy in presence of uncertainty, the initial movement is primarily ballistic and hence visual feedback is not necessary. The relatively large delay in visual information would also make the visual feedback ineffective for fast initial movements. Due to limited field of view of the camera, it is also difficult to easure that visual feedback can be used for the entire task. Therefore, the task may fail if any of the features is out of view. In this paper, we present a new task-space control strategy that allows the use of dual task-space information in a single controller. We shall show that the proposed task-space controller can transit smoothly from Cartesian-space feedback at the initial stage to vision-space feedback at the end stage when the target is near.

This paper presents a dynamic region following formation control method for a swarm of robots. In this control strategy, a swarm of robots shall move together as a group inside a dynamic region that can rotate or scale to enable the robots to adjust the formation. Various desired shapes can be formed by choosing appropriate functions. Unlike existing formation control methods, the proposed method do not need to have specific identities or orders in the group but yet dynamic formation can be formed for a large group of robots. This enables a swarm of robots to adjust the formation during the course of maneuver. The system is also scalable in the sense that any robot can move into the formation or leave the formation without affecting the other robots. Lyapunov-like function is presented for convergence analysis of the multi-robot systems. Simulation results are presented to illustrate the performance of the proposed controller.

This paper presents a novel method for shape control of a swarm of robots based on region control concept. Multiplicative potential energy function is used to form the overall desired shape of the entire swarm. The shape formed using this method is a union of all the regions defined by corresponding inequality functions. This proposed method is a complement to our previous method where the additive potential energy is used to form the desired shape. By combining the multiplicative and additive potential energies, a variety of complicated shapes can be formed. Lyapunov-like function is presented for convergence analysis of the multi-robot systems. Simulation results are presented to illustrate the performance of the proposed method.

It is interesting to observe that humans are able to manipulate an object easily and skillfully without the exact knowledge of the object, contact points, or kinematics of our fingers. However, research so far on multifingered robot control has assumed that the kinematics and contact points of the fingers are known exactly. In many applications of multifingered robot hands, the kinematics and contact points of the fingers are uncertain and structures of the Jacobian matrices are unknown. In this paper, we propose an adaptive neural network (NN) Jacobian controller for multifingered robot hand with uncertainties in kinematics, Jacobian matrices, and dynamics. It is shown that using NNs, the uniform ultimate boundedness of the position error can be achieved in the presence of the uncertainties. Simulation results are presented to illustrate the performance of the proposed controller.

In this paper, a region reaching controller with motion constraints is proposed for robot manipulator. In this control concept, the desired objective can be specified as a region instead of a point. In addition, physical constraints can be imposed on the robot motion by defining an additional region for the robot. For example, a region for the boundary of the end effector to avoid singular points or a region to ensure visibility of the features during visual servoing. We show that the stability of the system can be ensured in the presence of the motion constraints. The proposed region reaching controller is implemented on an industrial robot and experimental results are presented to illustrate the performance of the proposed controllers.

In this paper, a region following formation control method for multi-robot systems is proposed. In this control method, the robots move as a group inside a desired region while maintaining a minimum distance among themselves. Various shapes of desired region can be formed by choosing the appropriate objective functions. The robots do not need to have specific identities since the proposed controller does not need specific orders of robots within the group. Therefore, the system is scalable since any robot can come in or go out of the group without affecting the system. Lyapunov-like function is presented for convergence analysis of the multi-robot systems. Simulation results are presented to illustrate the performance of the proposed controller.

In this paper, a vision based tracking controller with adaptation to uncertainty in depth information is presented. Depth uncertainty plays a special role in visual tracking as it appears nonlinearly in the overall Jacobian matrix and hence cannot be adapted together with other uncertain kinematic parameters. We propose a novel parameter update law to update the uncertain parameters of the depth. It is proved that system stability can be guaranteed for the visual tracking task in presence of uncertainties in depth information, robot kinematics and dynamics. Simulation results are presented to illustrate the performance of the proposed controller.

In this paper, a new region control scheme is proposed for underwater vehicle-manipulator systems (UVMS). In the proposed control concept, the desired objective can be specified as a region instead of a point. The proposed region control concept is a generalization of setpoint control problem because when the desired region is specified arbitrarily small, the control objective reduces to a point. Lyapunov-like function is proposed for the stability analysis. Simulation studies are presented to demonstrate the effectiveness of the proposed controller.

In many applications of robot manipulators, the end-effector is required to make contact with environment. In these applications, it is necessary to control not only the position but also the interaction force between the robot end-effector and environment. Most research so far on motion and force tracking control has assumed that the kinematics and constraint surface are exactly known. In this paper, we propose a visually-servoed adaptive Jacobian controller for motion and force tracking control with structural uncertainties in kinematics, dynamics and constraint surface. It is shown that uniform ultimate boundedness of the tracking errors can be guaranteed. Simulation results are presented to illustrate the performance of the proposed control law.

In conventional setpoint control problem of robots, the desired position is specified as a point. However, it is interesting to observe from most human reaching movements that the desired targets are regions rather than points. In fact, when the desired region is specified arbitrarily small, it reduces to a point. In this paper, we propose a new control concept called region reaching control for robots. In this new control concept, the desired objective can be specified as a region instead of a point. Since the desired region can be specified arbitrarily small, the region control concept is also a generalization of setpoint control problem. Experimental results are presented to illustrate the performance of the proposed controllers.

Most research on vision and force control of robot manipulators has assumed that the kinematics and constraint surface are known exactly. In this paper, the vision and force control problem of robots with uncertain kinematics, dynamics and constraint is addressed. An adaptive setpoint control law based on neural networks is proposed. Sufficient conditions for choosing the feedback gains are presented to guarantee the stability. Simulation results are presented to demonstrate the effectiveness of the proposed controller

In this paper, we show that a duality property exists in task-space regulation of robots in the sense that the transformation from task space to joint space can be either defined as transpose Jacobian or inverse Jacobian. The two basic transformations, namely transpose Jacobian and inverse Jacobian, are said to be dual and any task-space setpoint controller can be obtained from the other by replacing the transpose Jacobian by the inverse Jacobian, and vice versa. Our result also provides a unified analysis for the transpose Jacobian setpoint control and inverse Jacobian setpoint control problems. Experiment results are also presented to verify the theory

In this paper, we propose a new region reaching control scheme for robots with uncertain kinematics and dynamics. In this control method, the desired objective can be specified as a region instead of a point. Since the desired region can be specified arbitrarily small, the region control concept is also a generalization of setpoint control problem. Based on this observation, we revisit the setpoint control problem from region control point of view, to gain new insights into the robot control problem. We shall show that the approximate Jacobian setpoint controller is a special case of the proposed region reaching controller

Most research so far on motion and force tracking control of robots has assumed that the kinematics and dynamics are exactly known. In this paper, we propose an adaptive Jacobian controller for motion and force tracking with uncertainties in kinematics and dynamics. It is shown that the robot end-effector can track the desired position and force trajectories with the uncertain parameters updated online. Simulation results are presented to illustrate the performance of the proposed control law

Joint flexibility is an important factor to consider in the robot control design if high performance is expected for the robot manipulators. The research work on control of rigid-link flexible-joint (RLFJ) robot in the literature has assumed that the kinematics of the robot is known exactly. There have been no results so far that can deal with the kinematics uncertainty in RLFJ robot. In this paper, we present the first study on this problem and propose an adaptive regulation method which can deal with the kinematics uncertainty and uncertainties in both link and motor dynamics of the RLFJ robot system. An observer is designed to avoid the use of acceleration due to the fourth-order overall dynamics. Sufficient conditions are derived to guarantee the asymptotic stability of the closed-loop system. Simulation result illustrates the effectiveness of proposed control method

This paper presents a task-space saturated-proportional, integral and differential (SP-ID) regulation approach for robot manipulators with uncertain kinematics and actuator model. The proposed approach is computationally efficient and easy to implement due to its simple structure. It's interesting to observe that in this paper the simple PID type controller is shown not only capable of compensating unknown gravity force, as has been known for long in robot control literature, but also capable of dealing with uncertainties in robot kinematics and actuator model. Sufficient conditions to guarantee system stability are provided and simulation results are presented to show the performance of proposed control method

Most research so far on of multi-fingered robot control has assumed that the kinematics is known exactly. However, in many applications of multi-fingered robot hands, the kinematics is uncertain. In this paper, a vision based control problem for multi-fingered robot hands with uncertain kinematics, dynamics and camera model is addressed. Adaptive neural network control law is proposed and it is shown that the stability can be achieved in the presence of the uncertainties. Sufficient conditions for choosing the feedback gains are presented to guarantee the stability

Most research so far on motion and force tracking has assumed that the kinematics and dynamics are exactly known. In this paper, we propose an visually-servoed adaptive Jacobian controller for motion and force tracking with uncertainties in kinematics, dynamics and camera model. It is shown that the robot end-effector can track the desired position and force trajectories with the uncertain parameters updated online. Simulation results are presented to illustrate the performance of the proposed control law

Most research so far on trajectory tracking control of robot has assumed that the kinematics of the robot is known exactly. This paper extends our recent work on adaptive Jacobian tracking control by deriving a new algorithm for trajectory tracking of robots with uncertain kinematics and dynamics. The algorithm requires only to measure the end-effector position in visual space, besides the robot’s joint angles and joint velocities. Experimental results are presented to illustrate the performance of the proposed controllers. In the experiments, we demonstrate that the robot’s shadow can be used to control the robot.

In conventional setpoint control problems of robot, the desired position is specified as a point. However, in some applications, the desired target is a region instead of a point. In this paper, we propose a new control concept called region reaching control for robot manipulators. In this new control concept, the desired objective can be specified as a region instead of a point. Several region reaching controllers are proposed in task space. Since the desired region can be specified arbitrarily small, the region control concept is also a generalization of setpoint control problem. Lyapunov-like functions are presented for the convergence analysis. Experiment results based on a SONY SCARA robot are presented to illustrate the performance of the proposed region reaching controllers.

In this paper, the adaptive regulation problem of rigid-link electrically driven (RLED) robotic manipulators with uncertain kinematics is addressed. A new task-space control scheme is proposed to overcome the uncertainties in actuator dynamics, robot dynamics and kinematics. By using a novel adaptive regressor, we avoid the overparameterization problem which is often met in the adaptive control problem with uncertain actuator model. Sufficient conditions for choosing the feedback gains, approximate Jacobian matrix are provided to guarantee system stability. Simulation results are presented to verify the effectiveness of the proposed control scheme.

Task-space regulation of robot manipulators can be classified into two fundamental approaches, namely, transpose Jacobian regulation and inverse Jacobian regulation. In this paper, two inverse Jacobian regulators with gravity compensations are presented, and the stability problems are formulated and solved. It is shown that the inverse Jacobian systems can be stabilized, and there exists a region of attraction such that the system remains stable. Our results show that the two fundamental approaches are two dual controllers, in the sense that the transpose Jacobian matrix can be replaced by the inverse Jacobian matrix and vice versa. The theoretical results are verified experimentally by implementing the inverse Jacobian regulators on an industrial robot, PUMA560.

Research so far on trajectory tracking control of robot has assumed that the kinematics of the robot is known exactly. In this paper, a new approximate Jacobian adaptive controller is proposed for trajectory tracking of robot with uncertain kinematics and dynamics. It is shown that the robot end effector is able to converge to a desired trajectory with the uncertain kinematics and dynamics parameters being updated online by parameter update laws. Experimental results are presented to illustrate the performance of the proposed controllers.

Most research so far on vision control of robot manipulator has been focused on free motion control. In order to expand the applications of vision-based controllers, it is necessary to control the force in addition to the motion. The vision and force control problem of robot manipulators with uncertain kinematics, dynamics, camera model and constraint surface is addressed. An adaptive set point control law is proposed for vision and force control. Sufficient conditions for choosing the feedback gains are presented to guarantee the stability. It is shown that the stability can be achieved in the presence of the uncertainties.

Most research so far in robot control has assumed either kinematics or Jacobian matrix of the robots from joint space to Cartesian space is known exactly. Unfortunately, no physical parameters can be derived exactly. In addition, when the robot picks up objects of uncertain lengths, orientations, or gripping points, the overall kinematics from the robot's base to the tip of the object becomes uncertain and changes according to different tasks. Consequently, it is unknown whether stability of the robot could be guaranteed in the presence of uncertain kinematics. In order to overcome these drawbacks, in this paper, we propose simple feedback control laws for setpoint control without exact knowledge of kinematics, Jacobian matrix, and dynamics. Lyapunov functions are presented for stability analysis of feedback control problem with uncertain kinematics. We shall show that the end-effector's position converges to a desired position in a finite task space even when the kinematics and Jacobian matrix are uncertain. Experimental results are presented to illustrate the performance of the proposed controllers.

Many controllers have been developed for setpoint control of robotic manipulators. Adaptive PD controller is one of the simplest and most effective setpoint controller in the presence of uncertainty in gravitational force. However, an exact model of gravity regressor is required in the adaptive PD control. In this paper we propose an adaptive setpoint controller with modeling error in the gravity regressor, and show that convergence can be guaranteed even when the gravity regressor is uncertain. A new Lyapunov function is presented for convergence analysis of such problem. As a by-product of the result we also show that existing setpoint controllers, such as the adaptive SP-D and SP-ID in the literature, can be analyzed and designed in an unifying way as special cases of the proposed controller.

Many controllers have been developed for setpoint control of robotic manipulators. An adaptive proportional and derivative (PD) controller is one of the simplest and most effective setpoint controller in the presence of uncertainty in gravitational force. However, an exact model of gravity regressor is required in the adaptive PD control. In this paper, we propose an adaptive setpoint controller with modeling error in the gravity regressor and show that convergence can be guaranteed even when the gravity regressor is uncertain. A new Lyapunov function is presented for the stability analysis of such problem. As a byproduct of the result, we also show that existing setpoint controllers such as an adaptive saturated proportional-derivative (SP-D) and saturated proportional-integral and derivative (SP-ID) in the literature can be analyzed and designed in a unifying way as special cases of the proposed controller.

Most researches so far on robot control have assumed that the exact kinematics and Jacobian matrix of the manipulator from joint space to Cartesian space are known. Unfortunately, no physical parameters could be derived exactly. In addition, the robot is required to interact with its environment and hence the overall parameters would change according to different tasks. In the paper, simple feedback control laws are proposed for setpoint control of robots with uncertain kinematics and dynamics. We show that the end-effector's position converges to a desired position in a finite task space even when the kinematics is uncertain.

Most research so far on robot control assumes that the kinematics and Jacobian matrix of the manipulator from joint space to task space are known exactly. This assumption leads to several open problems in the literature of robot control and limits the potential research and applications of robots. In this paper, we present an approximate Jacobian PID control law for set-point control of robot with uncertain kinematics from joint space to task space. Simulation results are presented to illustrate the results.

Most research on multifingered robot control has assumed that the Jacobian matrices from joint space to task space is exactly known. This implies that the locations of contact points, geometry of the object, kinematics of the multifingered robot hands must be exactly known. In this paper, a task-space feedback control problem of multifingered robot hands with uncertain Jacobian matrices is formulated and solved. The stability and robustness of the proposed controllers to the uncertainties in Jacobian matrices are analyzed.

Research of robotics aims to realise some aspects of human functions into mechanical system. Human can manipulate things skilfully without the exact knowledge of both dynamics and kinematics of arms. Our arms are also able to overcome singular position by moving along it or passing through it. The exploration of a robot controller to cope with the uncertainties in both dynamics and kinematics is an important step towards understanding the dextrous movement of mechanical systems. In this paper, simple feedback control laws are proposed for setpoint control of robots with uncertain kinematics and dynamics. We shall show that the end-effector's position converges to the desired position in a finite task space even when the actual Jacobian matrix is singular.

In this paper, an iterative learning impedance control problem for robotic manipulators is formulated and solved. A target impedance is specified and a learning controller is designed such that the system follows the desired response specified by the target model as the actions are repeated. A design method for analyzing the convergence of the learning impedance system is developed. A sufficient condition for guaranteeing the convergence of the system is also derived. The proposed learning impedance control scheme is implemented on an industrial selective compliance assembly robot arm (SCARA) robot, SEIKO TT3000. Experimental results verify the theory and confirm the effectiveness of the learning impedance controller.

Most researches on learning control of constrained robots have been focused on the problem of hybrid position/force control where the learning controllers are designed to track the desired motion and force trajectories. The learning impedance control of robotic manipulators, however, has not been developed so far. In this paper, a learning impedance control problem for robotic manipulators is formulated and solved. A target impedance is specified and a learning controller is designed such that the system follows the desired response specified by the target model as the actions are repeated. Sufficient conditions for guaranteeing the convergence of the system are derived. Simulation results of a cylindrical robot are presented to illustrate the performances of the proposed learning impedance controller.