Recent years have witnessed the popularity of integrating Siamese network into RGBT tracking for fast-tracking. However, these trackers mostly utilize the feature information of the last output layer and ignore the benefits of multi-layer information. In addition, they often adopt feature-level fusion for different modalities but fail to explore the strength of decision-level fusion, which may easily decrease their flexibility and independence. In this article, a novel multi-layer attention aggregation Siamese network on the decision level is proposed for robust RGBT tracking. To be specific, a hierarchical channel attention Siamese network is built to recalibrate the extracted multi-layer features from RGB and thermal infrared images. This can focus on more discriminative features to learn robust feature representation. Then, a depth-wise correlation operation is performed to produce RGB and thermal response maps, respectively. To better exploit and utilize the complementary RGB and thermal information, a contribution-aware aggregation network is designed to adaptively aggregate them. Lastly, a classification and regression network is adopted to complete the bounding box prediction. Extensive experiments on four large-scale RGBT benchmarks demonstrate outstanding tracking ability over other state-of-the-art trackers.

Vision-based localization and mapping solution is promising to be adopted in the automated valet parking task. In this paper, a semantic SLAM framework that leverages the hybrid edge information on bird’s-eye view images is presented. To extract useful edges from the synthesized bird’s-eye view image and the free-space contours for the SLAM task, different segmentation methods are designed to remove the noisy glare edges and distorted object edges caused by the inverse perspective mapping in view synthesis. Since only the free-space segmentation model needs training, our methods can dramatically reduce the labeling burden compared with previous road marking based methods. Those incorrect and incomplete edges are further cleaned and recovered by a temporal fusion of consecutive edges in a local map, respectively. Both a semantic edge point cloud map and an occupancy grid map can be built simultaneously in real time. Experiments in a parking garage demonstrate that the proposed framework can achieve higher accuracy and perform more robustly than previous point feature based methods.

Reverse dictionary(RD) is to map a description or definition to a name of a concept. Most of the existing reverse dictionary methods are word-based, failing to handle the polysemy of words. In this paper, we propose Non-Ambiguous Reverse Dictionary(NARD), a synset-based model using synset embeddings to replace word embeddings. NARD provides different and effective embeddings for different senses in multi-sense words, allowing for a more convergent mapping of words and descriptive sentences at the semantic level. With a cross-dictionary sense tagging, NARD can utilize all word-based dictionaries for training. The pre-trained synset embeddings and the multi-task learning methods are adopted to improve the generalization performance of the model. NARD are trained on one-fifth of the synset-labeled English datasets, and achieves the new state-of-the-art results.

Reinforcement learning (RL) has been extensively studied for robotic skill acquisition. Nevertheless, existing methods require extensive environmental interactions or high-quality demonstrations, which limits their application in practice. To alleviate this problem, a practical algorithm, named self-imitation learning with guide reward (SILGR), is proposed. The algorithm selects relatively good trajectories as expert data instead of external demonstrations and then assigns a guide reward to each transition. The criterion of the guide reward generator improves consistently with the evolution of the agent. In this way, the agent explores the environment in a task-relevant direction and exploits the experience more effectively, improving sample efficiency and performance. The results on four continuous locomotion tasks indicate that the proposed scheme achieves better performance than other state-of-the-art deep RL methods.

Reinforcement learning (RL) and imitation learning (IL), especially equipped with deep neural networks, have been widely studied for autonomous robotic skill acquisition and control tasks. However, these methods and their extensions require extensive environmental interactions during training, which greatly prevents them from being applied to real-world robots. To alleviate this problem, we present an efficient model-free off-policy actor-critic algorithm for robotic skill acquisition and continuous control, by fusing the task reward with a task-oriented guiding reward, which is formulated by leveraging few and imperfect expert demonstrations. In this framework, the agent can explore the environment more intentionally, thus sampling efficiency can be achieved; moreover, the agent can also exploit the experience more effectively, thereby substantially improved performance can be realized simultaneously. The empirical results on robotic locomotion tasks show that the proposed scheme can lower sample complexity by 2-10 times in contrast with the state-of-the-art baseline deep RL (DRL) algorithms, while achieving performance better than that of the expert. Furthermore, the proposed algorithm achieves significant improvement in both sampling efficiency and asymptotic performance on tasks with sparse and delayed reward, wherein those baseline DRL algorithms struggle to make progress. This takes a substantial step forward to implement these methods to acquire skills autonomously for real robots.

Feature Pyramid Network(FPN) employs a top-down path to enhance low level feature by utilizing high level feature. However, further improvement of detector is greatly hindered by the inner defect of FPN. The dilution issue in FPN is analyzed in this paper, and a new architecture named Semantic Feature Pyramid Network(SFPN) is introduced to address the information imbalance problem caused by information dilution. The proposed method consists of two simple and effective components: Semantic Pyramid Module(SPM) and Semantic Feature Fusion Module(SFFM). To compensate for the weaknesses of FPN, the semantic segmentation result is utilized as an extra information source in our architecture. By constructing a semantic pyramid based on the segmentation result and fusing it with FPN, feature maps at each level can obtain the necessary information without suffering from the dilution issue. The proposed architecture could be applied on many detectors, and non-negligible improvement could be achieved. Although this method is designed for object detection, other tasks such as instance segmentation can also largely benefit from it. The proposed method brings Faster R-CNN and Mask R-CNN with ResNet-50 as backbone both 1.8 AP improvements respectively. Furthermore, SFPN improves Cascade R-CNN with backbone ResNet-101 from 42.4 AP to 43.5 AP.

Feature pyramids are widely used to improve scale invariance for object detection. Most methods just map the objects to feature maps with relevant square receptive fields, but rarely pay attention to the aspect ratio variation, which is also an important property of object instances. It will lead to a poor match between rectangular objects and assigned features with square receptive fields, thus preventing from accurate recognition and location. Besides, the information propagation among feature layers is sparse, namely, each feature in the pyramid may mainly or only contain single-level information, which is not representative enough for classification and localization sub-tasks. In this paper, Bidirectional Matrix Feature Pyramid Network (BMFPN) is proposed to address these issues. It consists of three modules: Diagonal Layer Generation Module (DLGM), Top-down Module (TDM) and Bottom-up Module (BUM). First, multi-level features extracted by backbone are fed into DLGM to produce the base features. Then these base features are utilized to construct the final feature pyramid through TDM and BUM in series. The receptive fields of the designed feature layers in BMFPN have various scales and aspect ratios. Objects can be correctly assigned to appropriate and representative feature maps with relevant receptive fields depending on its scale and aspect ratio properties. Moreover, TDM and BUM form bidirectional and reticular information flow, which effectively fuses multi-level information in top-down and bottom-up manner respectively. To evaluate the effectiveness of our proposed architecture, an end-to-end anchor-free detector is designed and trained by integrating BMFPN into FCOS. And the center-ness branch in FCOS is modified with our Gaussian center-ness branch (GCB), which brings another slight improvement. Without bells and whistles, our method gains +3.3%, +2.4% and +2.6% AP on MS COCO dataset from baselines with ResNet-50, ResNet-101 and ResNeXt-101 backbones, respectively.

In this paper, we present a multi-camera visual odometry (VO) system for an autonomous vehicle. Our system mainly consists of a virtual LiDAR and a pose tracker. We use a perspective transformation method to synthesize a surroundview image from undistorted fisheye camera images. With a semantic segmentation model, the free space can be extracted. The scans of the virtual LiDAR are generated by discretizing the contours of the free space. As for the pose tracker, we propose a visual odometry system fusing both the feature matching and the virtual LiDAR scan matching results. Only those feature points located in the free space area are utilized to ensure the 2D-2D matching for pose estimation. Furthermore, bundle adjustment (BA) is performed to minimize the feature points reprojection error and scan matching error. We apply our system to an autonomous vehicle equipped with four fisheye cameras. The testing scenarios include an outdoor parking lot as well as an indoor garage. Experimental results demonstrate that our system achieves a more robust and accurate performance comparing with a fisheye camera based monocular visual odometry system.

Convolutional neural networks(CNNs) have achieved overwhelming success in image recognition. For all the architectures of CNNs, the low-level features extracted from the input is essential for the whole model because it determines the upper bound of accuracy. However, due to the random initialization and end-to-end training mechanism, there is no guarantee that the learned convolutional layer is an excellent low-level feature extractor. Thus, we propose Gabor binary layer(GBL) as a more efficient alternative to the first convolutional layer in CNNs. The design principle of GBL is motivated by local Gabor binary patterns. The GBL is comprised of a module of pre-defined Gabor convolutional filters in different orientations and shapes, and a module of fixed randomly generated binary convolutional filters to encode the Gabor features. By replacing the first layer of Resnet-56, Resnet-110, and LBCNN with the GBL, we achieve better performances on SVHN, CIFAR-10 and CIFAR-100 than that of the original models. Results show that the proposed GBL outperforms the standard convolutional layer for extracting low-level features, and thus the GBL can easily improve the performances of CNNs on image recognition.

Convolutional Neural Networks (CNNs) are getting larger and deeper, and thus becoming harder to be deployed on systems with limited resources. Though convolutional filters benefit from the concept of receptive field, they still take up lots of resources to store these parameters in the large amounts of filters. Therefore, a compression method of pre-trained CNN models using "Linear Representation" of convolutional kernels is introduced in this paper. First, a codebook of template kernels "Kt". are generated by conducting unsupervised clustering on all convolutional kernels, with Pearson Correlation Coefficient set as distance. Then all the convolutional kernels are represented by the closest templates using linear fitting function a • Kt + b , which means that only two parameters and a codebook index are enough to represent a kernel. After that, the model is retrained with fixed template kernels and only two related parameters need to be finetuned for each kernel. Experiments show that convolutional kernels of a large CNN model can be represented using only a small amount of templates. Thus, this method can reach a compression rate of convolutional layers near 4×, with tiny impact on precision after retraining. Nevertheless, the proposed method can be performed with other compression approaches to get higher compression rate.

Sparse dictionary learning on face recognition focuses on representing a face linearly by a set of atoms from the dictionary. How to learn a dictionary is a key issue to sparse representation. Structured dictionary has been used during the process of dictionary learning in order to improve the performance of classification. However, we consider that dictionary should not only be composed of a discriminant dictionary for identity or class information, but also a common dictionary which may contains disturbances and some common features for all class. Meanwhile, most of the proposed methods learns features and dictionary separatively, which may decrease the classification ability. Because projecting the source domain into a low dimensional space before dictionary learning will fail to catch some vital class-specific information which may be learned from dictionary learning. In this paper, a discriminant dictionary learning method with sparse embedding is proposed. Both discriminant and common dictionary are learned under the constraints on pairwise distance of sparsity coefficients, and the projection matrix is learned jointly. Experiments show that our method achieves better performance than other state-of-art methods on face recognition.

The representation space (R-Space) is proposed and constructed for a robot system to evaluate its feasibility to a prescribed task. The process of a task realization is transformed to be a process that the system representations transit in its Rspace. All factors affecting the task realization, including system configurations and physical and environmental constraints, are denoted as unreachable areas in the R-space, indicating the limitation for the system to accomplish the task. Reachable area in the R-space formalizes the criteria for task realizability. And the optimal strategy to fulfil a realizable task can further be identified. Otherwise, causes for unrealizability are clarified to transform the task to be realizable. The proposed framework is applied to different robotic tasks to show the validity as well as its performance.

This paper proposes a Hierarchical Task-guided Motion Planning (HTMP) scheme integrating Representation Space (R-space) model to address the motion planning problem in complex visually-guided tasks. The main characteristic of HTMP scheme is the interaction between task level and motion level. Such an interaction is based on the modification of R-space. When current R-space model and planner is unable to generate a feasible path, the dimension of R-space can be expanded to increase the flexibility of planning, which meets the extra requirements of natural motion. Simulation results verified the performance of the proposed method in various tasks fulfilled by a humanoid robot.

Some regions (or blocks) and their affiliated features of face images are normally of more importance for face recognition. However, the variety of feature contributions, which exerts different saliency on recognition, is usually ignored. This paper proposes a new sparse facial feature description model based on salience evaluation of regions and features, which not only considers the contributions of different face regions, but also distinguishes that of different features in the same region. Specifically, the structured sparse learning scheme is employed as the salience evaluation method to encourage sparsity at both the group and individual levels for balancing regions and features. Therefore, the new facial feature description model is obtained by combining the salience evaluation method with region-based features. Experimental results show that the proposed model achieves better performance with much lower feature dimensionality.

This paper studies an attitude tracking control system of a quadrotor unmanned aerial vehicle (UAV) under the condition of large-angle rotational maneuvers. We first established the attitude error model, taking both external disturbances and internal uncertainties into account. Thereafter, a switching control strategy is proposed for both high tracking accuracy and velocity constraints. Experiments on attitude tracking validate higher control accuracy with proposed method. Tasks of flight at unknown initial attitude and flip are also presented to verify the effectiveness of this method under large-angle rotational maneuverability.

High dimension of the features employed for face recognition is the main reason to slow down the recognition speed. Additionally, selecting salient facial features has significant impact on the efficiency of face recognition. In order to get the sparse and salient facial features, this paper propose a new sparse learning approach for salient facial feature description. This approach is to learn the feature evaluation vector with the training samples composed of within- and between-class distance vector sets. Then, the feature evaluation vector is employed to construct a new model for salient facial feature description. Experimental results show that the proposed method achieves much better face recognition performance with lower feature dimensionality.

Teleoperated robots often have unpredictable behaviors due to uncertain time delay in data transmissions over Internet. The robot cannot accomplish commands or actions issued by the remote operator in time, which severely impairs its reliability and efficiency. This paper investigates the strategy of modeling the operator's intention online by the teleoperated robot via the commands it received. Having deduced the intention, the robot could identify and thus autonomously accomplish the corresponding task without frequent interactions with the operator. Therefore data transmissions between the operator and the robot for task realization and performance evaluation can be minimized, and the impact of uncertain transmission delays on the efficiency of the entire system can be reduced. The robot, as an efficient manipulation tool controlled over the Internet, could be endowed with specified capability to improve its performance in the distributed environment. Intention of controlling and operating the robot is first expressed in planar or spatial grids and then incrementally inferred based on Bayesian techniques. Experiments with a teleoperated office robot show the validity and feasibility of the proposed method.

We present full state feedback controllers for standing and walking balance of humanoid robot. The robot is simulated as a two-joint inverted pendulum for standing and a five-link model for walking, and is disturbed by a horizontal push with given size and location in the sagittal plane. We optimize the parametric controllers for different push sizes, locations, and directions. For standing balance, both impulsive and constant pushes are applied to simulate the hip strategy; for bipedal walking, instantaneous pushes are used as perturbations. The performance of optimized controllers are shown in handling different pushes for standing and walking balance.

This paper presents a nonlinear controller based on a trajectory library. To generate the library, we combine two trajectory optimization methods: a parametric trajectory optimization method that finds coarse initial trajectories and Differential Dynamic Programming (DDP) that further refines these trajectories and generates linear local models of the optimal control laws. To construct a controller from these local models, we maintain the consistency of adjacent trajectories. To keep the resultant library a reasonable size and also satisfy performance requirements, the library is generated based on the controller's predicted performance. It is applied to standing balance control of humanoid robots that explicitly handle pushes. Most previous work assumes that pushes are impulsive. The proposed controller also handles continuous pushes that change with time. We compared our approach with a Linear Quadratic Regulator (LQR) gain scheduling controller using the same optimization criterion. The effectiveness of the proposed method is explored with simulation and experiments.

This paper develops full state feedback parametric controllers for perturbed walking of humanoid robot in response to external perturbations. We simulate models in the sagittal and lateral plane and in 3-dimensions, use a horizontal push of a given size, direction, location, and time as a perturbation, and optimize parametric controllers for different push sizes, directions, locations, and times. During a simulated perturbation experiment, the appropriate controller is selected based on the detected push information. The performance of optimized controllers are shown in handling different instantaneous pushes.

This paper proposes a general motion planning and coordination strategy for robot systems. The representation space (RS) of a robot system is constructed to describe the distributions of system attributes. The reachable area in the RS, denoting the attribute set that the system can be of, indicates the system's ability to accomplish tasks. Moreover, it also describes the influences of the internal and external constraints on the system's capability. Task realization is transformed to finding a trajectory in the RS for the system attributes to transit along under constraints. Meanwhile, the realizable conditions of a prescribed task by the robot system of specific configurations are discussed. If the task is realizable, the optimal strategy for task execution could further be figured out. Otherwise, it could be transformed to be realizable via task reassignment or system reconfigurations so that a connected path could be found for the transition of the system attributes from the starting point to the goal in the RS. The proposed scheme contributes to designing, planning, and coordination of the robotic tasks. Experiments on path planning of a robot manipulator and formation movement of a multirobot system, as well as coordination of a mobile manipulator system, are conducted to show the validity and generalization of the proposed method.

This paper develops full-state parametric controllers for standing balance of humanoid robots in response to impulsive and constant pushes. We also explore a hypothesis that postural feedback gains in standing balance should change with perturbation size. From an engineering point of view this is known as gain scheduling. We use an optimization approach to see if feedback gains should scale with the perturbation for a simulated robot. We simulate models in the sagittal and lateral plane and in 3-dimensions, use a horizontal push of a given size, direction and location as a perturbation, and optimize parametric controllers for different push sizes, directions and locations. During a simulated perturbation experiment, the appropriate controller is continuously selected based on the current push. For an impulse, the simulated robot recovers back to the initial state; for a constant push, the robot moves to an equilibrium position which leans into the push and has zero joint torques. We show the performance of optimized parametric controllers in response to different external pushes.

View-independence and user-independence are two fundamental requirements for hand posture recognition during natural human-robot interaction. However only a few research concerns on the two issues simultaneously. The difficulty for natural gesture-based human-robot interaction lies in that appearances of the same hand posture vary with different users from different viewing directions. In this paper, we propose a systematic feature selection approach based on Zernike moments and Isomap dimensionality reduction. A hierarchical classifier based on multivariate decision tree and piecewise linearization is developed to deal with the irregular distribution of the same hand postures. The proposed method is compared with other commonly used ones in hand posture recognition. Experimental results indicate that the proposed method can effectively identify different hand postures, irrespective of viewing directions and users.

This paper addresses the basic behavior acquisition of the robot head based on the multisensor integration. A robot head generally has several degrees of freedom (D.O.F.) of motions as well as different kinds of sensors. The head motion is planned based on all sensors’ feedback. We take advantage of the Jacobian matrix to describe the differential relations between the sensor feedback and the motor motions. Hence, the relation between two sensors could be formulated by the two respective Jacobian matrices of both sensors to motors. Consequently, multisensor integration can be employed for better performance of the robot head. Experiments of basic behavior acquisition like gazing and head posture control are conducted. Performances of both basic behaviors of the robot head before and after multisensor integration are compared, which demonstrate that the proposed multisensor integration way improves the performance of control, robustness against some sensor’s failure, and reduces the overall computation.

This paper studies stable switch control between multiple cameras for uncalibrated visual servoing. To get the continuous dynamic image Jacobian matrix among robots and distributed visual sensors, we suggest switch images based on fusion. We analyze fusion structure and design fusion algorithm that is suitable with having dynamically adjustable fusion weights. Simulations without any knowledge of mobile robots and uncalibrated visual sensors show that the method has more adaptability capability than the traditional instant switch control method. In this way the method can enhance the system stability at the switching process

In telepresence, it is critical for the local user to control the remote agent’s movement through his own locomotion in order to ensure a high degree of realism. Since the local user’s environment is normally different from that of the remote agent, there exists a motion mapping from the remote agent to the local user. After the path of the remote agent is predicted or recognized, it should be transformed to fit into the local environment, considering the constraints from the local environment, and ensuring utmost similarities in the shape and length of the paths. Moreover, terminal position of the local user in the local environment after a piece of known movement should also be carefully arranged after path transformation for his consecutive motions. These issues are incrementally addressed from the optimization point of view. Two schemes are proposed for path transformation problem. Extensive simulations and comparisons show the feasibility and effectiveness of the proposed approaches.

Development of calligraphic robot is motivated by allowing humanoid robot and home service robot to mimic the writing skills of human beings. Similar to human beings, versatile sensors are involved in writing procedure of robot to percept stages of task fulfillment and status of writing-brush. It is well known that the structures of Chinese characters are very complex and delicate. The writing-brush is soft and can draw strokes of different and variable widths, which makes Chinese calligraphy a sort of art like painting. Since the relations of strokes of Chinese characters are complicated and the writing-brush is extremely intricate to handle that it might result in inconsistent strokes with even slight changes in environment, dynamic management of sensors is hence a critical issue for a successful writing of an elegant Chinese character. This paper presents a sensor management method based on fuzzy decision tree (FDT), with which necessary prerequisite knowledge of Chinese character writing can be integrated into control of calligraphic robot system. Properties of sensor management problem in calligraphic robot are analyzed. Implementation of the proposed method is discussed. Experiments evaluate the feasibility and advantages of the proposed method.

Many vision tasks rely upon the identification of sets of corresponding features among different images. In this paper, we proposed a new feature-matching algorithm only based on geometric constraints rather than scene-dependent constraints. Through four novel schemes, homography is successfully used to iteratively remove the ambiguity of correspondences that are produced by epipolar geometry, even for curved scenes that are of high depth variations and content complexities. Our experiment results show that the proposed method is effective and robust.

This paper describes a novel distributed networked architecture for an intelligent robot. It is proposed to address two challenges for an Internet/LAN based robot system: (1) how to utilize the networked resources of perception and computation, and (2) how to realize a dynamic structured and updateable robotic system. In this architecture, a generic distributed robotic framework evolved from hybrid robotic architecture is proposed, to integrate an autonomous robot into a distributed network structure. A distributed computing infrastructure and the spontaneous and PnP connection mechanisms are realized, which implement the scheme of distributed computation and information for the networked robot system.

To find new approaches to the stability and synchronization problems of Internet based telerobotic systems, a mission/behavior based robotic teleoperation scheme is proposed. A novel distributed perception strategy is presented to enable the robot to autonomously search and utilize sensors via the Internet. These two schemes are implemented in the Distributed Architecture for Internet Robot (DAIR) design. In this paper, the mechanism, utilization and explanation of these two schemes are given. Experiments have proved their feasibility and effectiveness for both performance improvement in teleoperation and intelligence advancement in Internet based robotic systems.

Homography can be computed from four or more corresponding feature pairs in the polyhedral environment. For a stereo image pair that is weakly calibrated, the homography can be estimated via three corresponding feature pairs in the image pair together with the pair of the epipoles. That makes possible to estimate homography in curved surface environment. Since the estimation procedure does not introduce redundant information, it is error-prone. This paper analyzes the origins that produce the errors, which are classified as scene-related origin and calculation-related origin, and differentiates the errors into two types, which are respectively defined as geometric error and algebraic error. The criterion to separate these two kinds of errors is also given. Algorithm to estimate the homography is then improved in its accuracy based on the error analysis. A matching-point accumulation procedure is proposed based on the prediction of the possible matching position from the estimated homography. Feasibility of the proposed algorithm is also discussed. Extensive simulation and experiments are given to illustrate the performances of the new algorithms.

Telepresent system enables a user in local environment to operate in a remote or virtual environment through a robotic operator (agent). One of the key techniques for the system is that the user could move in local environment as what the agent would like to do in remote environment. However, since the size of local environment is normally limited while the size of remote environment is arbitrary, motion of the remote robot should be compressed and mapped to the local user, taking account of limitation of local environment. Meanwhile, the shape and length of the path, along which the robot moves, should be retained when being mapped to the local user, to ensure the realism of the telepresence. In this paper, this problem is addressed from the optimization point of view. A universal framework to mathematically approach this problem is proposed. The presented solution is easy to be extended to more complicated cases with more constraints involved, such as multiple users share one local environment. Extensive simulations show the feasibility and the effectiveness of the proposed approach.

This paper addresses the calibration-free robotic eye-hand coordination in a way other than the conventional image Jacobian matrix approach that has been studied extensively in literature. A nonlinear mapping rather than the linear mapping used in the image Jacobian matrix between the image space and the robotic control space is proposed. This mapping is regarded as the system's unmodeled dynamics expressed in system state equations. An extended state observer is designed first to estimate the unmodeled dynamics as well as the external disturbance of the system. With the estimation results as the compensation, a system controller is designed based on the nonlinear state-error feedback control strategy. Convergence of the extended state observer as well as the overall controller for a typical eye-hand coordination system is proved. Compared with the conventional calibration-free robotic eye-hand coordination with a Jacobian matrix, the proposed controller is independent of specific tasks and system configurations. Thus, a general design procedure is proposed for the calibration-free robotic eye-hand coordination. Simulation and experiment results demonstrate the satisfactory performance and effectiveness of the proposed approach.

This paper proposes a standard method to approach the uncalibrated robotic hand-eye coordination problem that is system configuration- and task-independent. The unknown hand-eye relationship is first modeled as the modeling errors of a dynamic system. An extended state observer is then implemented to estimate summation of the system's modeling error and the system's external disturbances. With the estimation results as the compensation, the system control is accomplished from a nonlinear combination of the system state errors. A universal framework of controller design is provided for decoupled and coupled hand-eye systems of different configurations to execute dynamic tracking task.

This paper addresses multi-sensor data fusion with incremental learning ability. A new cost function is proposed for the receptive field weighted regression (RFWR) algorithm based on the idea of back propagation (BP), so that the computation efficiency and the learning strategy of the modified RFWR are much more applicable for multi-sensor data fusion problem. Thus a new fusion structure and algorithm with incremental learning ability is constructed by adopting the modified RFWR algorithm together with the weighted average algorithm. Experiments of a two-camera unified positioning system are implemented successfully to test the proposed computation structure and algorithms.

A new control scheme for uncalibrated robotic visual tracking problem is proposed that compromises the computational expenses of overall system with offline modeling and online control. A nonlinear visual mapping model for the uncalibrated hand-eye coordination is first proposed with an artificial neural network implementation. An online visual tracking controller is then developed together with a real-time motion planner. To improve the system performance, the control scheme is also integrated with a feedforward controller to compensate unknown object motions. Extensive simulations and experiments demonstrate the effectiveness of the proposed control scheme.

Pedal curves are the loci of the feet of perpendiculars to the tangents of a fixed curve to a fixed point called the pedal point. By varying the location of the pedal point, deformable pedal curves have an important feature of incorporating a global parameterized shape into the curve evolution framework. In this paper, a hybrid geometric active model based on deformable pedal curves for face contour extraction is presented. Taking advantage of the deformable pedal curves, the proposed model can allow for representation of global and local shape characteristic of human face. Moreover, by implementing the model in a level set framework, automatic topological changes can be achieved naturally. Experimental results show the validity of our approach.

This paper studies a novel method to deal with the unknown image Jacobian matrix model for the uncalibrated robotic hand-eye coordination. An extended state observer is designed for online estimation of the image Jacobian matrix that is regarded as the system's unmodeled dynamics. A nonlinear feedback control law is then adopted for the coordination controller. With this scheme, a universal calibration-free controller is developed, which is independent from specific tasks and system configuration, for uncalibrated hand-eye coordination. Simulations and experiments of 3-D robotic visual positioning and tracking tasks show the validity and feasibility of the proposed control scheme.

This paper studies the visual servoing problem with sensory feedback from uncalibrated stereo cameras. The linear image Jacobian matrix is used to describe the spatial and temporary approximation of the differential movement relation between image space and robotic workspace. We suggest to construct an instrumental dynamic system with state variables formed from elements of the image Jacobian matrix. Thus a Kalman-Bucy filter is used to estimate the state variables of the constructed system online, which is robust to system noise and external disturbances. A 3D tracking task by a robot manipulator with visual feedback from uncalibrated stereo cameras is exemplified to show the formation of the instrumental system, estimation process and performance of the image Jacobian matrix and the design of the servo controller. Effectiveness of the proposed method and satisfactory tracking process can be found from extensive simulations and experiments provided in the paper.