Vision-language navigation (VLN) is a pivotal area within embodied intelligence, where agents must navigate based on natural language instructions. While traditional VLN research has focused on enhancing environmental comprehension and decision-making policy, these methods often reveal substantial performance gaps when agents are deployed in novel environments. This issue primarily arises from the lack of diverse training data. Expanding datasets to encompass a broader range of environments is impractical and costly. To address this challenge, we propose Vision-Language Navigation with Continuous Learning (VLNCL), a framework that allows agents to learn from new environments while preserving previous knowledge incrementally. We introduce a novel dual-loop scenario replay method (Dual-SR) inspired by brain memory mechanisms integrated with VLN agents. This approach helps consolidate past experiences and improves generalization across novel tasks. As a result, the agent exhibits enhanced adaptability to new environments and mitigates catastrophic forgetting. Our experiment demonstrates that VLN agents with Dual-SR effectively resist forgetting and adapt to unfamiliar environments. Combining VLN with continual learning significantly boosts the performance of otherwise average models, achieving SOTA results.

Maintaining robust standing stability is critical for humanoid robots performing precision manipulation tasks, where traditional balance strategies (e.g., stepping) can disrupt task execution. Designing a controller that ensures both stability and effective disturbance rejection is challenging, as these goals impose conflicting requirements on body sway and stepping behavior. To address this, we propose a hierarchical control framework that decouples these objectives into three parts: (1) a Task-Oriented Standing Policy trained via reinforcement learning with strict penalties on stepping and body sway, specialized for non-stepping disturbance rejection; (2) Ankle Strategy Modulation embedded within the standing policy, which dynamically adjusts the Zero-Moment Point (ZMP) to eliminate Divergent Component of Motion (DCM) errors, thereby accelerating recovery from disturbances; and (3) a fall prediction and recovery mechanism that triggers a transition to a robust stepping policy when disturbances exceed the stability threshold. Extensive experiments on a Q-series humanoid robot, in both simulation and hardware, validate the controller’s effectiveness. We achieved a 100% task success rate over 100 trials in practical scenarios including robot archery and part grasping/placement.

Spiking Neural Networks (SNNs) have attracted significant attention for their energy-efficient and brain-inspired event-driven properties. Recent advancements, notably Spiking-YOLO, have enabled SNNs to undertake advanced object detection tasks. Nevertheless, these methods often suffer from increased latency and diminished detection accuracy, rendering them less suitable for latency-sensitive mobile platforms. Additionally, the conversion of artificial neural networks (ANNs) to SNNs frequently compromises the integrity of the ANNs' structure, resulting in poor feature representation and heightened conversion errors. To address the issues of high latency and low detection accuracy, we introduce two solutions: timestep compression and spike-time-dependent integrated (STDI) coding. Timestep compression effectively reduces the number of timesteps required in the ANN-to-SNN conversion by condensing information. The STDI coding employs a time-varying threshold to augment information capacity. Furthermore, we have developed an SNN-based spatial pyramid pooling (SPP) structure, optimized to preserve the network's structural efficacy during conversion. Utilizing these approaches, we present the ultralow latency and highly accurate object detection model, SUHD. SUHD exhibits exceptional performance on challenging datasets like PASCAL VOC and MS COCO, achieving a remarkable reduction of approximately 750 times in timesteps and a 30% enhancement in mean average precision (mAP) compared to Spiking-YOLO on MS COCO. To the best of our knowledge, SUHD is currently the deepest spike-based object detection model, achieving ultralow timesteps for lossless conversion.

To achieve accurate and robust object detection in the real-world scenario, various forms of images are incorporated, such as color, thermal, and depth. However, multimodal data often suffer from the position shift problem, i.e., the image pair is not strictly aligned, making one object has different positions in different modalities. For the deep learning method, this problem makes it difficult to fuse multimodal features and puzzles the convolutional neural network (CNN) training. In this article, we propose a general multimodal detector named aligned region CNN (AR-CNN) to tackle the position shift problem. First, a region feature (RF) alignment module with adjacent similarity constraint is designed to consistently predict the position shift between two modalities and adaptively align the cross-modal RFs. Second, we propose a novel region of interest (RoI) jitter strategy to improve the robustness to unexpected shift patterns. Third, we present a new multimodal feature fusion method that selects the more reliable feature and suppresses the less useful one via feature reweighting. In addition, by locating bounding boxes in both modalities and building their relationships, we provide novel multimodal labeling named KAIST-Paired. Extensive experiments on 2-D and 3-D object detection, RGB-T, and RGB-D datasets demonstrate the effectiveness and robustness of our method.

Spiking Neural Networks (SNNs) have gained attention for their apparent energy efficiency and significant biological interpretability, although they also face significant challenges such as prolonged latency and suboptimal tracking accuracy. Recent studies have explored the application of SNNs in object tracking tasks. Dynamic visual sensors (DVS) have become a popular way to implement SNN-based object tracking due to their asynchronous and spiking characteristics similar to SNNs. However, challenges such as the high cost of DVS cameras and the lack of object surface texture information hinder the utility and performance of DVS trackers. In contrast, RGB information has inherent advantages, including low acquisition cost and comprehensive object surface texture representation. However, RGB information is prone to excessive image blurring in low-light conditions or in fast-motion scenes. To address these challenges, we propose the “Motion Feature Extractor” and the "RGB-DVS Fusion Module". The “Motion Feature Extractor” can replace the DVS camera at a very low cost, and the "RGB-DVS Fusion Module" can deeply fuse the feature information of the two to make up for their respective deficiencies. In addition, we adopt a conversion method to obtain a lossless SNN version of the model. Through experiments, our model achieves a 13.6% improvement in the expected average overlap (EAO) index using only 1.47% of the energy consumption of SiamRPN (VOT2016 dataset). In addition, we deployed the model to a robot and then conducted tracking experiments, which confirmed that the model can operate on the robot losslessly with satisfactory results.

This article targets the task of novel category discovery (NCD), which aims to discover unknown categories when a certain number of classes are already known. The NCD task is challenging due to its closeness to real-world scenarios, where we have only encountered some partial classes and corresponding images. Unlike previous approaches to NCD, we propose a novel adaptive prototype learning method that leverages prototypes to emphasize category discrimination and alleviate the issue of missing annotations for novel classes. Concretely, the proposed method consists of two main stages: prototypical representation learning and prototypical self-training. In the first stage, we develop a robust feature extractor that could effectively handle images from both base and novel categories. This ability of instance and category discrimination of the feature extractor is boosted by self-supervised learning and adaptive prototypes. In the second stage, we utilize the prototypes again to rectify offline pseudo labels and train a final parametric classifier for category clustering. We conduct extensive experiments on four benchmark datasets, demonstrating our method's effectiveness and robustness with state-of-the-art performance.

Bionic muscle-driven musculoskeletal systems can dynamically adjust the stiffness between active and antagonistic muscles to improve stability. However, they retain many problems, such as difficulty with sensory feedback control, which arises from the strong coupling of models and large computational loads encountered when optimizing the muscle force online. In this study, based on a bionic muscle model, an event-triggered-sliding-mode controller for a discrete-time-muscle-driven musculoskeletal system (MDMS) was designed to drive the system into a bounded region. Specifically, based on the Hamilton principle and discretization of muscle contraction dynamics, a new discrete-time-muscle-driven musculoskeletal model was constructed to facilitate the decoupling of the muscle model, feedback of the muscle state, and improvement of model control accuracy. Second, to guarantee the boundedness of the closed-loop system, an even-triggered-sliding-mode control law was established by introducing the input-to-state stable (ISS) method to a new type of discrete-time MDMS. The design ensured the convergence of different triggering cases, and an event-triggered-sliding-mode control law was established to obtain faster and smoother response characteristics. Finally, stability was ensured using the Lyapunov synthesis principle. The experimental results demonstrated that the proposed controller could effectively reduce the computational load while maintaining the same performance using a time-based approach.Note to Practitioners—The motivation of this study is to delve into the utilization of the Lyapunov control theory to enhance the control performance of the bionic muscle model. Additionally, it aimed to explore the initial application of the event-trigger mechanism within the musculoskeletal system. The primary focus is on discrete musculoskeletal system modeling, which facilitates the achievement of sensory feedback control for nonlinear muscle models characterized by strong coupling and a multi-segment structure. Using the proposed method, it is possible to regulate the muscle force based on sensory feedback using the principles of Lyapunov control theory. Furthermore, the application of event triggering mechanisms is explored with the goal of reducing the communicational load within musculoskeletal systems.

For RGB-based temporal action segmentation (TAS), excellent methods that capture frame-level features have achieved remarkable performance. However, for motion-centered TAS, it is still challenging for existing methods that ignore the extraction of spatial features of joints. In addition, inaccurate action boundaries caused by the frames of similar motion destroy the integrity of the action segments. To alleviate the issues, an end-to-end Involving Distinguished Temporal Graph Convolutional Networks called IDT-GCN is proposed. First, we construct an enhanced spatial graph structure that adaptively captures the similar and differential dependencies between joints in a single topology through learning two independent correlation modeling functions. Then, the proposed Involving Distinguished Graph Convolutional (ID-GC) models the spatial correlations of different actions in a video by using multiple enhanced topologies on the corresponding channels. Furthermore, we design a generic modeling temporal action regression network, termed Temporal Segment Regression (TSR), to extract segmented encoding features and action boundary representations by modeling action sequences. Combining them with label smoothing modules, we develop powerful spatial-temporal graph convolutional networks (IDT-GCN) for fine-grained TAS, which notably outperforms state-of-the-art methods on the MCFS-22 and MCFS-130 datasets. Adding TSR to TCN-based baseline methods achieves competitive performance compared with the state-of-the-art transformer-based methods on RGB-based datasets, i.e., Breakfast and 50Salads. Further experimental results on the action recognition task verify the superiority of the enhanced spatial graph structure over the previous graph convolutional networks.

The industry is undergoing a transformation toward smart manufacturing, fostering intelligent operations, sustainability, and digitalization. However, the current state of the process industry falls short of this future vision. Key areas, such as hybrid modeling, autonomous control, dynamic scheduling, intelligent decision making, security and safety control, and predictive maintenance, still require significant development. Given that the industrial metaverse enables the virtualization and digitization of industrial processes using technologies, such as artificial intelligence, blockchain, cloud computing, and digital twins, it is promising to establish the industrial metaverse for manufacturing, encompassing the entire lifecycle based on the industrial Internet and other modern information technologies.

Tracking-and-inserting a moving peg using a robot manipulator is a challenging task in manufacturing. In the past decades, various visual-based methods have been proposed for robotic manipulating static targets, which usually ignore the time delay in robot command transmission and image processing. However, for tracking and inserting a moving peg, time delays cannot be overlooked because they can reduce the tracking performance and even cause manipulations to fail. In this article, a robot visual-based control with a time-delay compensator is presented to solve the problem of inserting a moving peg. The time-delay compensator was designed using radial basis function neural network, a feedback compensator aimed at eliminating the tracking errors caused by the time delays. Thus, we could manipulate a moving object using a commercial industry robot, even with the time-variant delays in the control loop. Furthermore, the visual-based controller with the pseudoinverse image Jacobian matrix was designed using a linearization model. Thus, the matrix could be efficiently updated using the model. In the experiment, we inserted a peg into a moving hole using an eye-in-hand robot with precision.

Manifold learning now plays an important role in machine learning and many relevant applications. In spite of the superior performance of manifold learning techniques in dealing with nonlinear data distribution, their performance would drop when facing the problem of data sparsity. It is hard to obtain satisfactory embeddings when sparsely sampled high-dimensional data are mapped into the observation space. To address this issue, in this article, we propose hierarchical neighbors embedding (HNE), which enhances the local connections through hierarchical combination of neighbors. And three different HNE-based implementations are derived by further analyzing the topological connection and reconstruction performance. The experimental results on both the synthetic and real-world datasets illustrate that our HNE-based methods could obtain more faithful embeddings with better topological and geometrical properties. From the view of embedding quality, HNE develops the outstanding advantages in dealing with data of general distributions. Furthermore, comparing with other state-of-the-art manifold learning methods, HNE shows its superiority in dealing with sparsely sampled data and weak-connected manifolds.

Parkinson’s disease (PD) is a chronic neurodegenerative disease ranked second in the world. PD is often diagnosed using brain magnetic resonance imaging (MRI), which is a promising technique for PD biomarker development. However, it is difficult to focus on the pathogenic areas in the brain MRI of PD. Therefore, accurately capturing the characteristics of pathogenic areas has become an important issue. We propose a novel computational model (FF-MSPAM) for PD diagnosis by learning T2 weighted 3D-MRI slice features. First, in order to reduce parameters and accelerate training speed, a mixed network with ordinary convolution and separable convolution (OS-CNN) is designed. Next, VGG19 is applied for feature fusion to extract richer features. Finally, a multi-scale parallel attention mechanism (MSP-AM) was established to focus and aggregate spatial and channel features at different scales. The applicability of the proposed model was demonstrated using T2 weighted 3D-MRI slices of 168 subjects obtained from publicly available database. We have achieved classification accuracy of up to 98% in the differential diagnosis of PD. The experiment shows that our method is successful. Good results were obtained in PD diagnosis task and compared with advanced research models. Our model can be used for the diagnosis and prognosis of PD.

Segmenting unseen objects is a crucial ability for the robot since it may encounter new environments during the operation. Recently, a popular solution is leveraging RGB-D features of large-scale synthetic data and directly applying the model to unseen real-world scenarios. However, the domain shift caused by the sim2real gap is inevitable, posing a crucial challenge to the segmentation model. In this paper, we em-phasize the adaptation process across sim2real domains and model it as a learning problem on the BatchNorm param-eters of a simulation-trained model. Specifically, we propose a novel non-parametric entropy objective, which formulates the learning objective for the test-time adaptation in an open-world manner. Then, a cross-modality knowledge distillation objective is further designed to encourage the test-time knowledge transfer for feature enhancement. Our approach can be efficiently implemented with only test images, without requiring annotations or revisiting the large-scale synthetic training data. Besides significant time savings, the proposed method consistently improves segmentation results on the overlap and boundary metrics, achieving state-of-the-art performance on unseen object instance segmentation.

Object goal visual navigation is a challenging task that aims to guide a robot to find the target object based on its visual observation, and the target is limited to the classes pre-defined in the training stage. However, in real households, there may exist numerous target classes that the robot needs to deal with, and it is hard for all of these classes to be contained in the training stage. To address this challenge, we study the zero-shot object goal visual navigation task, which aims at guiding robots to find targets belonging to novel classes without any training samples. To this end, we also propose a novel zero-shot object navigation framework called semantic similarity network (SSNet). Our framework use the detection results and the cosine similarity between semantic word embeddings as input. Such type of input data has a weak correlation with classes and thus our framework has the ability to generalize the policy to novel classes. Extensive experiments on the AI2-THOR platform show that our model outperforms the baseline models in the zero-shot object navigation task, which proves the generalization ability of our model. Our code is available at: https://github.com/pioneer-innovation/Zero-Shot-Object-Navigation.

Robust object detection methods in traffic surveillance scenarios often encounters challenges due to large-scale deformations and illumination variations in outdoor scenes. To enhance the tolerance of such methods against these variations, we design a cross-scale and illumination-invariant detection model (CSIM) based on the You Only Look Once (YOLO) architecture. A main cause of false detection in large-scale detection tasks is the inconsistency between various feature scales. To address this issue, we introduce an adaptive cross-scale feature fusion model to ensure the consistency of the constructed feature pyramid. To overcome the influence of uneven light, we build an illumination-invariant chromaticity space on the CSIM model, which is independent of the correlated color temperature. In addition, we adopt spatial attention modules, K-means clustering and the Mish activation function for further model optimization. The obtained experimental results show that the proposed CSIM produces excellent detection results for addressing the challenges derived from large-scale deformations and the illumination changes encountered during traffic surveillance. Compared with state-of-the-art object detection methods on public datasets, our proposed model has achieved competitive results in robust object detection tasks in traffic surveillance scenarios.

Imbalanced classification tasks are widespread in many real-world applications. For such classification tasks, in comparison with the accuracy rate (AR), it is usually much more appropriate to use nondecomposable performance measures such as the area under the receiver operating characteristic curve (AUC) and the$F_β$measure as the classification criterion since the label class is imbalanced. On the other hand, the minimax probability machine is a popular method for binary classification problems and aims at learning a linear classifier by maximizing the AR, which makes it unsuitable to deal with imbalanced classification tasks. The purpose of this article is to develop a new minimax probability machine for the$F_β$measure, called minimax probability machine for the$F_β$-measures (MPMF), which can be used to deal with imbalanced classification tasks. A brief discussion is also given on how to extend the MPMF model for several other nondecomposable performance measures listed in the article. To solve the MPMF model effectively, we derive its equivalent form which can then be solved by an alternating descent method to learn a linear classifier. Further, the kernel trick is employed to derive a nonlinear MPMF model to learn a nonlinear classifier. Several experiments on real-world benchmark datasets demonstrate the effectiveness of our new model.

Alzheimer's disease (AD) is the main cause of dementia in the elderly. To date, it remains largely unknown whether and how dynamic characteristics of the functional networks differ from cognitively normal (CN) to AD. Here, we propose an AD dynamic network complexity intelligent detecting algorithm based on visibility graph. The focal regions that caused the dynamic abnormality of the connection mode were intelligently detected by creating a dynamic complexity network on the basis of the dynamic functional network. The results showed that the brain areas with different dynamic complexity gradually shifted from the frontal lobe to the temporal lobe and the occipital lobe. This was significantly related to the disorder of clinical patients from mood to memory and language. The increased dynamic complexity illustrates the compensatory effect of the brain area of AD lesions. In addition, the small-world topological properties of the dynamic complexity network have significant differences from CN to AD. To the best of our knowledge, this is the first time that such a concept is proposed. Our method of intelligently detecting the complexity of AD dynamic network provides new insights for understanding the internal dynamic mechanism of AD brain.

This paper presents an event-triggered model predictive control (MPC) strategy with learning terminal cost for robotic manipulators containing model uncertainty and input constraints. In the proposed MPC structure, an adaptive predictive model for the robotic system is established by radial basis function neural networks (RBFNNs) firstly. Then, a terminal cost adjusted by the global learning mechanism is constructed. Both global steady-state optimization and transient fast convergence are achieved by adding the learning terminal cost to the MPC scheme. After that, a triggering condition of the MPC solving is developed based on the predictive model’s weights and the predictive tracking error. Besides, the condition to avoid Zeno behavior is obtained. The recursive feasibility of the proposed MPC strategy is verified, and the ultimately uniformly boundedness (UUB) of all variables is proved according to the Lyapunov theorem. Finally, experiments based on an xMate7 Pro robot are conducted to demonstrate the effectiveness of the presented method. Note to Practitioners—The tracking control of robotic manipulators is a common and important problem in industrial applications, such as grasping, loading, unloading, et al. There exist some limitations in existing control methods. For example, general control strategies such as adaptive control, sliding mode control neglect the balance between control costs and expected performance; existing optimal control approaches rarely consider global steady-state optimization and transient fast convergence under the influence of model uncertainty. This paper is motivated by these limitations of robotic control design and inspired by model predictive control and optimal control theory. It develops a novel strategy for tracking control of robotic manipulators, which includes three following items: (1) adopt an approximation model constructed by neural networks as the predictive model for estimating the robotic system; (2) introduce a critic network into the terminal cost of the objective function of MPC for learning global optimal solution; (3) establish an event-triggered mechanism for solving of the optimization problem. The proposed method is verified by experimental results. In our future work, the proposed control strategy will be applied to more industrial processes.

Current robotic studies are focused on the performance of specific tasks. However, such tasks cannot be generalized, and some special tasks, such as compliant and precise manipulation, fast and flexible response, and deep collaboration between humans and robots, cannot be realized. Brain-inspired intelligent robots imitate humans and animals, from inner mechanisms to external structures, through an integration of visual cognition, decision making, motion control, and musculoskeletal systems. This kind of robot is more likely to realize the functions that current robots cannot realize and become human friends. With the focus on the development of brain-inspired intelligent robots, this article reviews cutting-edge research in the areas of brain-inspired visual cognition, decision making, musculoskeletal robots, motion control, and their integration. It aims to provide greater insight into brain-inspired intelligent robots and attracts more attention to this field from the global research community.

Grasping is one of the most fundamental problems in robotic manipulation. In recent years, with the development of data-driven methods, reinforcement learning has been used in solving robotic grasping problems. However, grasping is a long-horizon and sparse reward task, whose natural reward only appears when the task is successfully achieved. Therefore, it brings great challenges to the deployment of reinforcement learning methods. To tackle this difficulty, we propose a new method called Trajectory-based Split Hindsight Reverse Curriculum Learning. This method of reverse learning from the goal can greatly improve the learning efficiency and the final performance of the tasks. Specifically, based on referred trajectories, the agent starts to learn in a small state space near the goal and then gradually in larger state spaces until covering the entire state space. Through split hindsight experience replay, the sampled trajectory is divided into segments that match the current subspace's size; then, they are modified to successful trajectories to enable more efficient learning. In both simulation and real-world experiments, our method surpasses the existing methods and achieves the goal-oriented grasping tasks with higher success rates and better data efficiencies. The detailed experimental results can be viewed at https://youtu.be/7uNRzmRZhDk.

This letter proposes a lightweight two-stream convolutional neural network (CNN) for synthetic aperture radar (SAR) target recognition. Specifically, the two-stream CNN first extracts low-level features by three alternating convolution layers and max-pooling layers. Then two streams are followed to extract local and global features. One stream uses global maximum pooling to extract local features with the greatest response; the other uses large-stride convolution kernels to extract global features. Finally, the two streams are combined for target recognition. Therefore, the two-stream CNN can learn rich multilevel features to achieve high recognition accuracy for SAR target recognition. Moreover, compared to other popular CNNs, the two-stream CNN is very lightweight. The experimental results on the moving and stationary target acquisition and recognition (MSTAR) data set demonstrate that the proposed method not only can improve the recognition accuracy but also reduce the number of parameters of the model dramatically.

A single feature is hard to describe the content of images from an overall perspective, which limits the retrieval performances of single-feature-based methods in image retrieval tasks. To fully describe the properties of images and improve the retrieval performances, multifeature fusion ranking-based methods are proposed. However, the effectiveness of multifeature fusion in image retrieval has not been theoretically explained. This article gives a theoretical proof to illustrate the role of independent features in improving the retrieval results. Based on the theoretical proof, the original ranking list generated with a single feature greatly influences the performances of multifeature fusion ranking. Inspired by the principle of three degrees of influence in social networks, this article proposes a reranking method named k -nearest neighbors' neighbors' neighbors' graph (N3G) to improve the original ranking list by a single feature. Furthermore, a multigraph fusion ranking (MFR) method motivated by the group relation theory in social networks for multifeature ranking is also proposed, which considers the correlations of all images in multiple neighborhood graphs. Evaluation experiments conducted on several representative data sets (e.g., UK-bench, Holiday, Corel-10K, and Cifar-10) validate that N3G and MFR outperform the other state-of-the-art methods.

Owing to its potential superiorities in terms of flexibility, compliance, and robustness, the musculoskeletal robotic system has become a promising direction for next-generation robots. However, motion learning and generalization of musculoskeletal systems are still challenging problems. In this article, a muscle-synergies-based neuromuscular control is proposed. First, a new computational model of time-varying muscle synergies is constructed, which utilizes both phasic and tonic muscle synergies to characterize the basic features of muscle excitations more sufficiently. Second, a novel neuromuscular control method is proposed for realizing the motion learning and generalization of musculoskeletal systems. Therein, a radial basis function (RBF) neural network is designed to modulate muscle synergies according to different movement targets. Muscle excitations are computed with the combination of modulated muscle synergies. Covariance matrix adaptation evolutionary strategy is applied to realize the synchronous optimization of muscle synergies and the RBF neural network. In the experiment, a sophisticated musculoskeletal system learns to perform center-out reaching tasks through trial-and-error learning on a few targets. With the muscle synergies and neural modulation learned from a few targets, the musculoskeletal system can also reach many unexperienced targets. The proposed method not only improves the speed and accuracy of motion learning but also enhances motion generalization. This article also promotes the development of the musculoskeletal robotic system and the fusion of neuroscience and robotics.

This article proposes a novel decision-making framework that bridges a gap between model-based (MB) and model-free (MF) control processes through only adjusting the planning horizon. Specifically, the output policy is obtained by solving a model predictive control problem with a locally optimal state value as terminal constraints. When the planning horizon decreases to zero, the MB control will transform into the MF control smoothly. Meanwhile, inspired by the neural mechanism of emotion modulation on decision-making, we build a biologically plausible computational model of emotion processing. This model can generate an uncertainty-related emotional response on the basis of the state prediction error and reward prediction error, and then dynamically modulates the planning horizon in the tasks. The simulation results demonstrate that the proposed decision-making framework can produce better policies than traditional methods. Emotion modulation can shift the MB and MF control well to improve the learning efficiency and the speed of decision-making.

Wishart distance of covariance matrices and Euclidean distance of polarimetric features are two important similarity measurements for polarimetric synthetic aperture radar (Pol-SAR) image classification. This paper proposes a feature extraction method by combing the two distances for Pol-SAR image classification. Firstly, two weight graphs are constructed based on the two distances to represent the local information of the data. Specifically, the neighbouring samples are sought in a local region to reduce the computation burden and utilize the spatial information. Then the dimensionality reduction model is constructed based on the two weight graphs and co-regularization. The co-regularization aims to minimize the dissimilarity of low-dimensional features corresponding to two graphs. Finally, the obtained low-dimensional features are used for PolSAR image classification. Experimental results on real PolSAR datasets demonstrate the effectiveness and superiority of the proposed method.

Feature correspondence lays the foundation for many computer vision and image processing tasks, which can be well formulated and solved by graph matching. Because of the high complexity, approximate methods are necessary for graph matching, and the continuous relaxation provides an efficient approximate scheme. But there are still many problems to be settled, such as the highly nonconvex objective function, the ignorance of the combinatorial nature of graph matching in the optimization process, and few attention to the outlier problem. Focusing on these problems, this paper introduces a continuation method directly targeting at the combinatorial optimization problem associated with graph matching. Specifically, first a regularization function incorporating the original objective function and the discrete constraints is proposed. Then a continuation method based on Gaussian smoothing is applied to it, in which the closed forms of relevant functions with respect to the outlier distribution are deduced. Experiments on both synthetic data and real world images validate the effectiveness of the proposed method.

For polarimetric synthetic aperture radar (PolSAR) image classification, each pixel can be represented by multiple features from different perspectives, such as polarimetric feature (PF), texture feature (TF) and color feature (CF). Both multi-view canonical correlation analysis (MCCA) and multi-view spectral embedding (MSE) are two unsupervised multi-view subspace learning methods which search for different projection matrices for different features to combine multiple features in a common low-dimensional feature space. However, MCCA emphasizes the correlation of multiple features and MSE learns the complementarity of multiple features. To deeply learn the relation of multiple features, we incorporate MCCA with MSE based on the label information and a symmetric version of revised Wishart (SRW) distance for supervised PolSAR image feature extraction. Experimental results confirm that the proposed method can improve the classification performance.

Spatial information helps improve the performance of polarimetric synthetic aperture radar (PolSAR) image classification. Some existing methods have combined the spatial information and polarimetric features by the third-order tensor representation for feature extraction. They describe a pixel with the patch centered on this pixel. But they neglect the spatial heterogeneity, which may influence the classification performance. Therefore, we firstly seek k nearest samples based on the polarimetric feature similarity for each pixel to construct the second-order tensor, whose first order denotes the nearest samples and the second order denotes the polarimetric features. Moreover, k nearest samples are searched in a spatial local region rather than the full image, which can exploit the spatial information and reduce the computational burden. Then we employ tensor principal component analysis (TPCA) to extract low-dimensional features. Experimental results demonstrate that the proposed method can improve the classification performance compared with other methods.

In this study, we proposed a novel human-machine interface (HMI) for robotic manipulator control. The specific target was to adjust the impedance coefficients of the robot controller in real time by measuring the human forearm muscle contractions. We firstly designed a HMI system. Different from the frequently used sEMG technologies, the interface in our study could detect muscle morphological changes within the skin by the electrical impedance tomography (EIT). The sensing front-end was a soft elastic fabric band which was compatible to different arm shapes. With the specific designed sensing hardware and the re-construction algorithms, EIT images indicating forearm muscle shapes were obtained. We then designed a hybrid positon/impedance controller on a UR5 with the impedance coefficients being tuned in real time by the grasp force estimation. A sigmoid regression algorithm was used to map the EIT images to the grasp forces. After implementation of the whole system, two experiments were carried out. The first experiment was the off-line grasp force estimation. With the 1:1 cross validation, an average R2of 0.83±0.04 and an average of the relative root mean square error (RRMSE) of 0.31±0.10 across 5 subjects were yielded. The second experiment was the real-time robot control. Trajectory tracking task with dynamic uncertainties were investigated and grasp forces were estimated in real-time. With higher muscle contraction levels, smaller position errors were observed and shorter time was needed to return to the expected trajectory when there were external disturbances. The results proved the feasibility of the new approach on human-robot interaction tasks. Future endeavours will be made to get more promising results.

Video-based person re-identification (re-id) matches two tracks of persons from different cameras. Features are extracted from the images of a sequence and then aggregated as a track feature. Compared to existing works that aggregate frame features by simply averaging them or using temporal models such as recurrent neural networks, we propose an intelligent feature aggregate method based on reinforcement learning. Specifically, we train an agent to determine which frames in the sequence should be abandoned in the aggregation, which can be treated as a decision making process. By this way, the proposed method avoids introducing noisy information of the sequence and retains these valuable frames when generating a track feature. On benchmark data sets, experimental results show that our method can boost the re-id accuracy obviously based on the state-of-the-art models.

Policy search in reinforcement learning (RL) is a practical approach to interact directly with environments in parameter spaces, that often deal with dilemmas of local optima and real-time sample collection. A promising algorithm, known as guided policy search (GPS), is capable of handling the challenge of training samples using trajectory-centric methods. It can also provide asymptotic local convergence guarantees. However, in its current form, the GPS algorithm cannot operate in sequential multitask learning scenarios. This is due to its batch-style training requirement, where all training samples are collectively provided at the start of the learning process. The algorithm’s adaptation is thus hindered for real-time applications, where training samples or tasks can arrive randomly. In this paper, the GPS approach is reformulated, by adapting a recently proposed, lifelong-learning method, and elastic weight consolidation. Specifically, Fisher information is incorporated to impart knowledge from previously learned tasks. The proposed algorithm, termed sequential multitask learning-GPS, is able to operate in sequential multitask learning settings and ensuring continuous policy learning, without catastrophic forgetting. Pendulum and robotic manipulation experiments demonstrate the new algorithms efficacy to learn control policies for handling sequentially arriving training samples, delivering comparable performance to the traditional, and batch-based GPS algorithm. In conclusion, the proposed algorithm is posited as a new benchmark for the real-time RL and robotics research community.

The fast and accurate classification of polarimetric synthetic aperture radar (PolSAR) data in dynamically changing environments is an important and challenging task. In this paper, we propose an Incremental Multi-view Passive-Aggressive Active learning algorithm, named IMPAA, for PolSAR data classification. This algorithm can deal with online two-view multi-class categorization problem by exploiting the relationship between the polarimetric-color and texture feature sets of PolSAR data. In addition, the IMPAA algorithm can handle the dynamic large-scale datasets where not only the amount of data but also the number of classes gradually increases. Moreover, this algorithm only queries the class labels of some informative incoming samples to update the classifier based on the disagreement of different views' predictors and a randomized rule. Experiments on real PolSAR data demonstrate that the proposed method can use a smaller fraction of queried labels to achieve low online classification errors compared with previously known methods.

Feature extraction is a very important step for polarimetric synthetic aperture radar (PolSAR) image classification. Many dimensionality reduction (DR) methods have been employed to extract features for supervised PolSAR image classification. However, these DR-based feature extraction methods only consider each single pixel independently and thus fail to take into account the spatial relationship of the neighboring pixels, so their performance may not be satisfactory. To address this issue, we introduce a novel tensor local discriminant embedding (TLDE) method for feature extraction for supervised PolSAR image classification. The proposed method combines the spatial and polarimetric information of each pixel by characterizing the pixel with the patch centered at this pixel. Then each pixel is represented as a third-order tensor, of which the first two modes indicate the spatial information of the patch (i.e. the row and the column of the patch) and the third mode denotes the polarimetric information of the patch. Based on the label information of samples and the redundance of the spatial and polarimetric information, a supervised tensor-based dimensionality reduction technique, called TLDE, is introduced to find three projections which project each pixel, that is, the third-order tensor into the low-dimensional feature. Finally, classification is completed based on the extracted features using the nearest neighbor (NN) classifier and the support vector machine (SVM) classifier. The proposed method is evaluated on two real PolSAR data sets and the simulated PolSAR data sets with various number of looks. The experimental results demonstrate that the proposed method not only improves the classification accuracy greatly, but also alleviates the influence of speckle noise on classification.

Reducing samples through convex hull vertices selection (CHVS) within each class is an important and effective method for online classification problems, since the classifier can be trained rapidly with the selected samples. However, the process of CHVS is NP-hard. In this paper, we propose a fast algorithm to select the convex hull vertices, based on the convex hull decomposition and the property of projection. In the proposed algorithm, the quadratic minimization problem of computing the distance between a point and a convex hull is converted into a linear equation problem with a low computational complexity. When the data dimension is high, an approximate, instead of exact, convex hull is allowed to be selected by setting an appropriate termination condition in order to delete more nonimportant samples. In addition, the impact of outliers is also considered, and the proposed algorithm is improved by deleting the outliers in the initial procedure. Furthermore, a dimension convention technique via the kernel trick is used to deal with nonlinearly separable problems. An upper bound is theoretically proved for the difference between the support vector machines based on the approximate convex hull vertices selected and all the training samples. Experimental results on both synthetic and real data sets show the effectiveness and validity of the proposed algorithm.

Human action segmentation is important for human action analysis, which is a highly active research area. Most segmentation methods are based on clustering or numerical descriptors, which are only related to data, and consider no relationship between the data and physical characteristics of human actions. Physical characteristics of human motions are those that can be directly perceived by human beings, such as speed, acceleration, continuity, and so on, which are quite helpful in detecting human motion segment points. We propose a new physical-based descriptor of human action by curvature sequence warp space alignment (CSWSA) approach for sequence segmentation in this paper. Furthermore, time series-warp metric curvature segmentation method is constructed by the proposed descriptor and CSWSA. In our segmentation method, descriptor can express the changes of human actions, and CSWSA is an auxiliary method to give suggestions for segmentation. The experimental results show that our segmentation method is effective in both CMU human motion and video-based data sets.

We propose a weighted common subgraph (WCS) matching algorithm to find the most similar subgraphs in two labeled weighted graphs. WCS matching, as a natural generalization of equal-sized graph matching and subgraph matching, has found wide applications in many computer vision and machine learning tasks. In this brief, WCS matching is first formulated as a combinatorial optimization problem over the set of partial permutation matrices. Then, it is approximately solved by a recently proposed combinatorial optimization framework-graduated nonconvexity and concavity procedure. Experimental comparisons on both synthetic graphs and real-world images validate its robustness against noise level, problem size, outlier number, and edge density.

In this paper, we address the semisupervised distance metric learning problem and its applications in classification and image retrieval. First, we formulate a semisupervised distance metric learning model by considering the metric information of inner classes and interclasses. In this model, an adaptive parameter is designed to balance the inner metrics and intermetrics by using data structure. Second, we convert the model to a minimization problem whose variable is symmetric positive-definite matrix. Third, in implementation, we deduce an intrinsic steepest descent method, which assures that the metric matrix is strictly symmetric positive-definite at each iteration, with the manifold structure of the symmetric positive-definite matrix manifold. Finally, we test the proposed algorithm on conventional data sets, and compare it with other four representative methods. The numerical results validate that the proposed method significantly improves the classification with the same computational efficiency.

Polarimetric synthetic aperture radar (PolSAR) plays an indispensable part in remote sensing. With its development and application, rapid and accurate online classification for PolSAR data becomes more and more important. PolSAR data can be depicted by different features such as polarimetric, texture and color features, which can be considered as multiple views. In this paper, we propose an online multiview learning method based on the passive aggressive algorithm, named OMVPA, for PolSAR data real-time classification. The OMVPA method makes full use of the consistency and complementary properties of different views. Experimental results on real PolSAR data demonstrate that the proposed method maintain a smaller mistake rate compared with other methods.

Brain-inspired models have become a focus in artificial intelligence field. As a biologically plausible network, the recurrent neural network in reservoir computing framework has been proposed as a popular model of cortical computation because of its complicated dynamics and highly recurrent connections. To train this network, unlike adjusting only readout weights in liquid computing theory or changing only internal recurrent weights, inspired by global modulation of human emotions on cognition and motion control, we introduce a novel reward-modulated Hebbian learning rule to train the network by adjusting not only the internal recurrent weights but also the input connected weights and readout weights together, with solely delayed, phasic rewards. Experiment results show that the proposed method can train a recurrent neural network in near-chaotic regime to complete the motion control and working-memory tasks with higher accuracy and learning efficiency.

Automated assembly, especially peg-in-hole insertion, is a common task in manufacturing. In particular, the high-precision assembly is achieved by high-precision manipulator and sensing system. However, uncertainty and various parts for assembly are still challenges for robotic assembly, especially for low-precision robot and sensors. It is noteworthy that human can implement assembly tasks although the precision of the arm and hand is not comparable with a common industrial robot, in which process compliance is the key characteristic of their motion. In this paper, we present a human-inspired compliant strategy for peg-in-hole assembly task using the environmental constraint and coarse force information. In the proposed strategy, a constraint region is designed for motion planning and utilized for eliminating the uncertainty of the initial positioning error of the peg. Force sensor is applied to sense the contact force of which the direction is used to adjust the movement of the peg. Therefore, high-precision sensor is not necessarily required. Inspired by human compliant assembly, a from coarse to fine adjustment strategy is executed. The contribution of our strategy is that high precision assembly task can be solved by low precision system. The constraint region and force guided directional adjustment have increased the robustness of the system. The strategy is carried out in simulation for round peg-in-hole assembly task. The experimental results show that the assembly task can be successfully completed and demonstrate the effectiveness of our strategy.

This letter proposes a novel multiview feature extraction method for supervised polarimetric synthetic aperture radar (PolSAR) image classification. PolSAR images can be characterized by multiview feature sets, such as polarimetric features and textural features. Canonical correlation analysis (CCA) is a well-known dimensionality reduction (DR) method to extract valuable information from multiview feature sets. However, it cannot exploit the discriminative information, which influences its performance of classification. Local discriminant embedding (LDE) is a supervised DR method, which can preserve the discriminative information and the local structure of the data well. However, it is a single-view learning method, which does not consider the relation between multiple view feature sets. Therefore, we propose local discriminant CCA by incorporating the idea of LDE into CCA. Specific to PolSAR images, a symmetric version of revised Wishart distance is used to construct the between-class and within-class neighboring graphs. Then, by maximizing the correlation of neighboring samples from the same class and minimizing the correlation of neighboring samples from different classes, we find two projection matrices to achieve feature extraction. Experimental results on the real PolSAR data sets demonstrate the effectiveness of the proposed method.

This issue marks the first anniversary issue since I was honored to serve as the Editor-in-Chief (EiC) of the IEEE Transactions on Neural Networks and Learning Systems (TNNLS). I am happy to report that we had a very successful year and here are a few highlights that I would like to share with the community.•The latest impact factor of TNNLS is 4.854 according to the Journal Citation Reports. This marks a record high impact factor for our journal and places TNNLS as the number one scholarly publication in Computer Science (Hardware & Architecture), number three in Computer Science (Theory & Methods), and number ten in Electrical and Electronic Engineering journals.

In this paper, the event-triggered state estimation problem is investigated for a class of discrete-time multidelayed neural networks with stochastic parameters and incomplete measurements. In order to cater for more realistic transmission process of the neural signals, we make the first attempt to introduce a set of stochastic variables to characterize the random fluctuations of system parameters. In the addressed neural network model, the delays among the interconnections are allowed to be different, which are more general than those in the existing literature. The incomplete information under consideration includes randomly occurring sensor saturations and quantizations. For the purpose of energy saving, an event-triggered state estimator is constructed and a sufficient condition is given under which the estimation error dynamics is exponentially ultimately bounded in the mean square. It is worth noting that the ultimate boundedness of the error dynamics is explicitly estimated. The characterization of the desired estimator gain is designed in terms of the solution to a certain matrix inequality. Finally, a numerical simulation example is presented to illustrate the effectiveness of the proposed event-triggered state estimation scheme.

Generalization ability is widely acknowledged as one of the most important criteria to evaluate the quality of unsupervised models. The objective of our research is to find a better dropout method to improve the generalization ability of convolutional deep belief network (CDBN), an unsupervised learning model for vision tasks. In this paper, the phenomenon of low feature diversity during the training process is investigated. The attention mechanism of human visual system is more focused on rare events and depresses well-known facts. Inspired by this mechanism, No Feature Left Behind Dropout (NFLB Dropout), an adaptive dropout method is firstly proposed to automatically adjust the dropout rate feature-wisely. In the proposed method, the algorithm drops well-trained features and keeps poorly-trained ones with a high probability during training iterations. In addition, we apply two approximations of the quality of features, which are inspired by theory of saliency and optimization. Compared with the model trained by standard dropout, experiment results show that our NFLB Dropout method improves not only the accuracy but the convergence speed as well.

Object tracking is one of the important tasks for mobile robot, and developing a robust and real-time visual tracking algorithm which can adaptively capture the varying appearance of target under challenging conditions for mobile robot is still an open problem. The main challenges of visual tracking for mobile robot come from variation of target's appearance and disturbance of environment. To cope with these problems, one of the most important topics is how to select the best tracking features. In this paper, we propose a novel adaptive probabilistic tracking method with discriminative feature selection for mobile robot Different from the existing adaptive tracking algorithms which select the discriminative features in a finite feature set, the proposed method treats feature selection as an estimation problem of the best feature tunable parameters in a continuous space. The estimation of the best tunable parameters and object tracking are implemented via different particle filters with novel observation models. A novel target model updating strategy is also proposed to adapt to the varying appearance of target and resist gradual drift. Experiments show the robustness of the proposed method under challenging conditions.

This letter proposes a method that can preserve the global and local discriminative information based on the tensor representation to achieve feature extraction for synthetic aperture radar (SAR) target configuration recognition. We model SAR images of targets with different configurations as different manifolds, and each manifold is represented as a collection of maximal linear patches (MLPs), each depicted by a subspace. The manifold-to-manifold distance and subspace-to-subspace distance are used to maintain the global discriminative structure of data. Meanwhile, point-to-point distance (PPD) in an MLP is exploited to keep the local discriminative information of data. These two terms are then integrated to maintain the structure of data. Experimental results on the moving and stationary target automatic recognition (MSTAR) database demonstrate the effectiveness of the proposed method.

Poisson denoising is an essential issue for various imaging applications, such as night vision, medical imaging, and microscopy. State-of-the-art approaches are clearly dominated by patch-based non-local methods in recent years. In this paper, we aim to propose a local Poisson denoising model with both structural simplicity and good performance. To this end, we consider a variational modeling to integrate the so-called fields of experts (FoE) image prior, that has proven an effective higher-order Markov random fields model for many classic image restoration problems. We exploit several feasible variational variants for this task. We start with a direct modeling in the original image domain by taking into account the Poisson noise statistics, which performs generally well for the cases of high signal-to-noise ratio (SNR). However, this strategy encounters problem in cases of low SNR. Then we turn to an alternative modeling strategy by using the Anscombe transform and Gaussian statistics derived data term. We retrain the FoE prior model directly in the transform domain. With the newly trained FoE model, we end up with a local variational model providing strongly competitive results against state-of-the-art nonlocal approaches, meanwhile bearing the property of simple structure. Furthermore, our proposed model comes along with an additional advantage, that the inference is very efficient as it is well suited for parallel computation on GPUs. For images of size $512 \times 512$, our GPU implementation takes less than 1 second to produce state-of-the-art Poisson denoising performance.

The Wishart fidelity and total variation (TV) based variational model (WisTV) with the positive definite (PD) constraint has shown to be effective for the whole PolSAR covariance data speckle reduction. However, the existing algorithms for solving the WisTV model only give approximation solutions by projecting the results onto the set of PD matrices, and their parameters depend strongly on the data. The purpose of this letter is to propose a new optimization algorithm to address the issues. To keep the uniformity of the parameters for different PolSAR data, a sigmoid function-based normalization method is designed, which ensures the applicability of the WisTV model for the normalized data. By using the orthogonal decomposition of the PD variables, the WisTV model is converted into an unconstrained optimization problem which is further transformed into a multivariable problem based on the equivalent representations of the trace and logdet functions. The alternative minimization technique is then utilized to solve the final optimization problem. The subproblems for each individual variable are convex and their solutions have explicit expressions. Moreover, the computational complexity of the algorithm is discussed. Experimental results on both synthetic and real PolSAR data demonstrate the validity of the proposed algorithm.

Recently, many biologically inspired visual computational models have been proposed. The design of these models follows the related biological mechanisms and structures, and these models provide new solutions for visual recognition tasks. In this paper, based on the recent biological evidence, we propose a framework to mimic the active and dynamic learning and recognition process of the primate visual cortex. From principle point of view, the main contributions are that the framework can achieve unsupervised learning of episodic features (including key components and their spatial relations) and semantic features (semantic descriptions of the key components), which support higher level cognition of an object. From performance point of view, the advantages of the framework are as follows: 1) learning episodic features without supervision-for a class of objects without a prior knowledge, the key components, their spatial relations and cover regions can be learned automatically through a deep neural network (DNN); 2) learning semantic features based on episodic features-within the cover regions of the key components, the semantic geometrical values of these components can be computed based on contour detection; 3) forming the general knowledge of a class of objects-the general knowledge of a class of objects can be formed, mainly including the key components, their spatial relations and average semantic values, which is a concise description of the class; and 4) achieving higher level cognition and dynamic updating-for a test image, the model can achieve classification and subclass semantic descriptions. And the test samples with high confidence are selected to dynamically update the whole model. Experiments are conducted on face images, and a good performance is achieved in each layer of the DNN and the semantic description learning process. Furthermore, the model can be generalized to recognition tasks of other objects with learning ability.

In this paper, we propose a nonlocal total variation (NLTV)-based variational model for polarimetric synthetic aperture radar (PolSAR) data speckle reduction. This model, named WisNLTV, is obtained based on the Wishart fidelity term and the NLTV regularization defined for the complex-valued fourth-order tensor data. Since the proposed model is non-convex, an equivalent bi-convex model is obtained using the property of conjugate functions. Then, an efficient iteration algorithm is developed to solve the equivalent bi-convex model, based on the alternating minimization and the forward-backward operator splitting technique. The proposed iteration algorithm is proved to be convergent under certain conditions theoretically and numerically. Experimental results on both synthetic and real PolSAR data demonstrate that the proposed method can effectively reduce speckle noise and, meanwhile, better preserve the details and the repetitive structures such as textures and edges, and the polarimetric scattering characteristics, compared with the other methods.

Speckle reduction is an important task in synthetic aperture radar. One extensively used approach is based on total variation (TV) regularisation, which can realise significantly sharp edges, but on the other hand brings in the undesirable staircasing artefacts. In essence, the TV‐based methods tend to create piecewise‐constant images even in regions with smooth transitions. In this study, a new method is proposed for speckle reduction via total generalised variation (TGV) penalty. This is reasonable from the fact that the TGV‐based model can reduce the staircasing artefacts of TV by being aware of higher‐order smoothness. An efficient numerical scheme based on the Nesterov's algorithm is also developed for solving the TGV‐based optimisation problem. Monte Carlo experiments show that the proposed scheme yields state‐of‐the‐art results in terms of both performance and speed. Especially when the image has some higher‐order smoothness, the authors’ scheme outperforms the TV‐based methods.

Graph regularized Sparse Coding (GSC) considers data relationships during Sparse Coding (SC) and thus has better performance in certain image analysis tasks. However, it is very time consuming. This letter aims at speeding up GSC. The alternating optimization framework for GSC involves repeatedly solving a variant of${\ell _1}$minimization referred to as GSRsub in this letter. Traditional ways to deal with GSRsub are to generalize optimization strategies for${\ell _1}$minimization to solve its primal problem that is strongly convex but non-differentiable, thus converging slowly. We propose that GSC can be accelerated by solving a new dual problem of GSRsub called D-GSRsub. Compared with the primal form and the existing dual form of GSRsub, D-GSRsub has a strongly convex and smooth objective function with less variables. Based on these properties, four dual gradient ascent strategies with lower computational complexities are developed. Experimental results on real-world datasets demonstrate that these strategies can dramatically and stably speed up GSC without affecting its performance in the corresponding image analysis tasks.

In this letter, we propose a novel saliency model for saliency detection, named sparse-distinctive (SD) saliency model. Different from the existing models that only consider sparsity or distinctness of image, the proposed model computes saliency based on sparsity and distinctness. The basic idea is that sparsity and distinctness contribute to saliency simultaneously and play different roles under different scenes. This sparse-distinctive saliency model is based on some key ideas introduced in this letter and supported by psychological evidence. Experimental results on public benchmark eye-tracking datasets show that considering the sparsity and distinctness for saliency can improve the accuracy of predicting human fixations, and the proposed model outperforms the mainstream models on predicting human fixations.

Development of a flexible low-cost robotic system to grasp various 3D workpieces is of practical important. Traditional approaches on 3D grasping usually need to compute the effective contact positions based on sufficient conditions, such as “force-closure” or “form-closure,” to prevent all the motions of the grasped objects. Compared with the previous work motivated by high-precision applications, caging provides a way to manipulate an object without needing to immobilize it. However, most caging conditions consider only 2D motions of the object in grasping. In some cases, other motions in 3D space, e.g., the pitch and roll rotations of the objects, would also possibly change a caging configuration to an uncaging configuration. This paper aims to discuss caging with frictionless contact, by taking into consideration of all the motions of the object. We first discuss the relationship between the gripper configuration and the state of the grasped object in grasping, where all the motions of the object are taken into account. We then establish the sufficient conditions to find a set of 3D caging configurations in terms of the width of the object's projection and the gap of the gripper, such that we can utilize the projection to find the feasible placements of the pins to determine the caging configuration. Furthermore, we discuss how to find the caging configuration that is capable of leading to a form-closure based on attractive region formed in the configuration space.

Recently, many computational models have been proposed to simulate visual cognition process. For example, the hierarchical Max-Pooling (HMAX) model was proposed according to the hierarchical and bottom-up structure of V1 to V4 in the ventral pathway of primate visual cortex, which could achieve position- and scale-tolerant recognition. In our previous work, we have introduced memory and association into the HMAX model to simulate visual cognition process. In this paper, we improve our theoretical framework by mimicking a more elaborate structure and function of the primate visual cortex. We will mainly focus on the new formation of memory and association in visual processing under different circumstances as well as preliminary cognition and active adjustment in the inferior temporal cortex, which are absent in the HMAX model. The main contributions of this paper are: 1) in the memory and association part, we apply deep convolutional neural networks to extract various episodic features of the objects since people use different features for object recognition. Moreover, to achieve a fast and robust recognition in the retrieval and association process, different types of features are stored in separated clusters and the feature binding of the same object is stimulated in a loop discharge manner and 2) in the preliminary cognition and active adjustment part, we introduce preliminary cognition to classify different types of objects since distinct neural circuits in a human brain are used for identification of various types of objects. Furthermore, active cognition adjustment of occlusion and orientation is implemented to the model to mimic the top-down effect in human cognition process. Finally, our model is evaluated on two face databases CAS-PEAL-R1 and AR. The results demonstrate that our model exhibits its efficiency on visual recognition process with much lower memory storage requirement and a better performance compared with the traditional purely computational methods.

In this paper, we propose a variational model for polarimetric synthetic aperture radar (PolSAR) data speckle reduction, which is based on the complex Wishart distribution of the covariance or coherency matrix and multichannel total variation (TV) regularization defined for complex-valued matrices. By assuming the TV regularization to be a prior and taking the statistical distribution of the covariance matrix in each resolution element into account, the variational model for PolSAR covariance data speckle suppression, named WisTV-C, is derived from the maximum a posteriori estimate. A similar variational model for PolSAR coherency data speckle reduction, named WisTV-T, is also obtained. As far as we know, this is the first variational model for the whole PolSAR covariance or coherency matrix data despeckling. Since the model is nonconvex, a convex relaxation iterative algorithm is designed to solve the variational problem, based on the variable splitting and alternating minimization techniques. Experimental results on both simulated and real PolSAR data demonstrate that the proposed approach notably removes speckles in the extended uniform areas and, meanwhile, better preserves the spatial resolution, the details such as edges and point scatterers, and the polarimetric scattering characteristics, compared with other methods.

Subspace selection is widely applied in data classification, clustering, and visualization. The samples projected into subspace can be processed efficiently. In this paper, we research the linear discriminant analysis (LDA) and maximum margin criterion (MMC) algorithms intensively and analyze the effects of scatters to subspace selection. Meanwhile, we point out the boundaries of scatters in LDA and MMC algorithms to illustrate the differences and similarities of subspace selection in different circumstances. Besides, the effects of outlier classes on subspace selection are also analyzed. According to the above analysis, we propose a new subspace selection method called angle linear discriminant embedding (ALDE) on the basis of angle measurement. ALDE utilizes the cosine of the angle to get new within-class and between-class scatter matrices and avoids the small sample size problem simultaneously. To deal with high-dimensional data, we extend ALDE to a two-stage ALDE (TS-ALDE). The synthetic data experiments indicate that ALDE can balance the within-class and between-class scatters and be robust to outlier classes. The experimental results based on UCI machine-learning repository and image databases show that TS-ALDE has a lower time complexity than ALDE while processing high-dimensional data.

Recently, Graphic Processing Units (GPUs) have been shown to be of value in supporting wire-speed name lookup in Named Data Networking (NDN). However, due to the computing model on GPU, the lookup latency is not so encouraging. In this paper, we shift the focus from GPU to Field-Programmable Gate Arrays (FPGA). We highlight three key challenges in accelerating name lookup using FPGA, and then present a scalable approach to address them. In our approach, a hierarchical and compact data structure is proposed to represent the name trie, which achieves not only effective pipeline mapping but also high memory efficiency. Further, it is finally implemented as a linear pipeline on the FPGA platform, enabling both fast lookup speed and low lookup latency. The experimental results show that our approach gains a reduction of memory cost over 90% compared with the referred GPU-based solution. Besides, the lookup throughput of our approach is almost 2.4 times higher, and the latency is up to 3 orders of magnitude lower.

Sparse coding has been used for target appearance modeling and applied successfully in visual tracking. However, noise may be inevitably introduced into the representation due to background clutter. To cope with this problem, we propose a saliency weighted sparse coding appearance model for visual tracking. Firstly, a spectral filtering based visual attention computational model, which combines both bottom-up and top-down visual attention, is proposed to calculate saliency map. Secondly, pooling operation in sparse coding is weighted by calculated saliency map to help target representation focus on distinctive features and suppress background clutter. Extensive experiments on a recently proposed tracking benchmark demonstrate that the proposed algorithm outperforms state-of-the-art methods in tracking objects under background clutter.

A new and effective salient region detection method based on local and global saliency information is proposed. To keep the completeness of salient regions, the input image is segmented into several regions firstly. Then for each region, local saliency and global saliency are generated respectively. The local saliency is computed by multi-scale neighborhood contrast, and the global saliency is measured according to global spatial distribution and inter-region isolation of features. Based on the local saliency and global saliency, the final saliency can be obtained by the weighted combination of them. The comparison experiment results demonstrate the effective performance of the proposed algorithm on salient region detection.

In this paper we propose the graduated nonconvexity and concavity procedure (GNCCP) as a general optimization framework to approximately solve the combinatorial optimization problems defined on the set of partial permutation matrices. GNCCP comprises two sub-procedures, graduated nonconvexity which realizes a convex relaxation and graduated concavity which realizes a concave relaxation. It is proved that GNCCP realizes exactly a type of convex-concave relaxation procedure (CCRP), but with a much simpler formulation without needing convex or concave relaxation in an explicit way. Actually, GNCCP involves only the gradient of the objective function and is therefore very easy to use in practical applications. Two typical related NP-hard problems, partial graph matching and quadratic assignment problem (QAP), are employed to demonstrate its simplicity and state-of-the-art performance.

The Fields of Experts (FoE) image prior model, a filter-based higher-order Markov Random Fields (MRF) model, has been shown to be effective for many image restoration problems. Motivated by the successes of FoE-based approaches, in this letter we propose a novel variational model for multiplicative noise reduction based on the FoE image prior model. The resulting model corresponds to a non-convex minimization problem, which can be efficiently solved by a recently published non-convex optimization algorithm. Experimental results based on synthetic speckle noise and real synthetic aperture radar (SAR) images suggest that the performance of our proposed method is on par with the best published despeckling algorithm. Besides, our proposed model comes along with an additional advantage, that the inference is extremely efficient. Our GPU based implementation takes less than 1s to produce state-of-the-art despeckling performance.

A famous biologically inspired hierarchical model (HMAX model), which was proposed recently and corresponds to V1 to V4 of the ventral pathway in primate visual cortex, has been successfully applied to multiple visual recognition tasks. The model is able to achieve a set of position- and scale-tolerant recognition, which is a central problem in pattern recognition. In this paper, based on some other biological experimental evidence, we introduce the memory and association mechanism into the HMAX model. The main contributions of the work are: 1) mimicking the active memory and association mechanism and adding the top down adjustment to the HMAX model, which is the first try to add the active adjustment to this famous model and 2) from the perspective of information, algorithms based on the new model can reduce the computation storage and have a good recognition performance. The new model is also applied to object recognition processes. The primary experimental results show that our method is efficient with a much lower memory requirement.

Control gains play an important role in the control of a natural or a technical system since they reflect how much resource is required to optimize a certain control objective. This paper is concerned with the controllability of neuronal networks with constraints on the average value of the control gains injected in driver nodes, which are in accordance with engineering and biological backgrounds. In order to deal with the constraints on control gains, the controllability problem is transformed into a constrained optimization problem (COP). The introduction of the constraints on the control gains unavoidably leads to substantial difficulty in finding feasible as well as refining solutions. As such, a modified dynamic hybrid framework (MDyHF) is developed to solve this COP, based on an adaptive differential evolution and the concept of Pareto dominance. By comparing with statistical methods and several recently reported constrained optimization evolutionary algorithms (COEAs), we show that our proposed MDyHF is competitive and promising in studying the controllability of neuronal networks. Based on the MDyHF, we proceed to show the controlling regions under different levels of constraints. It is revealed that we should allocate the control gains economically when strong constraints are considered. In addition, it is found that as the constraints become more restrictive, the driver nodes are more likely to be selected from the nodes with a large degree. The results and methods presented in this paper will provide useful insights into developing new techniques to control a realistic complex network efficiently.

Object grasping or localization is an essential stage in automatic manufacturing processes. In general, stable grasping of 3-D objects is achieved by a multifinger hand. Different from the existing approaches, in this paper, we propose a novel vision-based method for grasping 3-D objects with a simple 2-D gripper. We describe the grasping strategy, the range of 3-D objects which can be grasped, and the region of grasping orientations which can guarantee successful actions. The proposed method facilitates the freedom of grasping orientation, which endows the strategy with a larger application range. Furthermore, we also explain the potential applications of the proposed method in grasping 3-D objects with other 2-D grippers. Although the proposed method uses a 2-D gripper and 2-D vision information, it can be regarded as a 3-D grasping approach due to the following facts. 1) The objects to be grasped are 3-D. The grasping process and the grasping orientations are analyzed and given in a 3-D space. 2) The contact points between the object and the gripper are not always in a 2-D plane. In the proposed method, the concept attractive region in configuration space, which was proposed in the previous work, is introduced to analyze the whole grasping process. Compared with other traditional methods, the proposed method directly gives the range of objects which can be grasped as well as the region of grasping orientations. By establishing the relationship between our method and traditional ones, it is proved that the bottom of the attractive region corresponds to a stable grasp.

Manifold learning is a hot research topic in the held of computer science and has many applications in the real world. A main drawback of manifold learning methods is, however, that there are no explicit mappings from the input data manifold to the output embedding. This prohibits the application of manifold learning methods in many practical problems such as classification and target detection. Previously, in order to provide explicit mappings for manifold learning methods, many methods have been proposed to get an approximate explicit representation mapping with the assumption that there exists a linear projection between the high-dimensional data samples and their low-dimensional embedding. However, this linearity assumption may be too restrictive. In this paper, an explicit nonlinear mapping is proposed for manifold learning, based on the assumption that there exists a polynomial mapping between the high-dimensional data samples and their low-dimensional representations. As far as we know, this is the hrst time that an explicit nonlinear mapping for manifold learning is given. In particular, we apply this to the method of locally linear embedding and derive an explicit nonlinear manifold learning algorithm, which is named neighborhood preserving polynomial embedding. Experimental results on both synthetic and real-world data show that the proposed mapping is much more effective in preserving the local neighborhood information and the nonlinear geometry of the high-dimensional data samples than previous work.

The support vector machine (SVM) method, as a promising classification technique, has been widely used in various fields due to its high efficiency. However, SVM cannot effectively solve online classification problems since, when a new sample is misclassified, the classifier has to be retrained with all training samples plus the new sample, which is time consuming. According to the geometric characteristics of SVM, in this paper we propose an online SVM classifier called VS-OSVM, which is based on convex hull vertices selection within each class. The VS-OSVM algorithm has two steps: 1) the samples selection process, in which a small number of skeleton samples constituting an approximate convex hull in each class of the current training samples are selected and 2) the online updating process, in which the classifier is updated with newly arriving samples and the selected skeleton samples. From the theoretical point of view, the first d+1 (d is the dimension of the input samples) selected samples are proved to be vertices of the convex hull. This guarantees that the selected samples in our approach keep the greatest amount of information of the convex hull. From the application point of view, the new algorithm can update the classifier without reducing its classification performance. Experimental results on benchmark data sets have shown the validity and effectiveness of the VS-OSVM algorithm.

This paper presents a novel context-based feature for image encoding and object detection. The Oriented Gradient Context (OGC) descriptor represents the image in the context of different local area-based oriented gradient information. Both fine and coarse oriented gradients information about the image is captured, then different sizes of local areas with statistical oriented gradients are assembled into pair combinations to represent the gradient distribution context of the image. The features are comparatively simple but information-rich for utilization by classification algorithms. Based on the context information, the detection algorithm is relatively invariant to small shifts, translations of objects and changes in object appearance; even cases with partial occlusions and cluttered background are handled. The detection algorithm based on the proposed OGC features is shown to achieve good performance on pedestrian detection, comparable to other popular algorithms.

Manifold learning is an important topic in pattern recognition and computer vision. However, most manifold learning algorithms implicitly assume the data are aligned on a single manifold, which is too strict in actual applications. Isometric feature mapping (Isomap), as a promising manifold learning method, fails to work on data which distribute on clusters in a single manifold or manifolds. In this paper, we propose a new multi-manifold learning algorithm (M-Isomap). The algorithm first discovers the data manifolds and then reduces the dimensionality of the manifolds separately. Meanwhile, a skeleton representing the global structure of whole data set is built and kept in low-dimensional space. Secondly, by referring to the low-dimensional representation of the skeleton, the embeddings of the manifolds are relocated to a global coordinate system. Compared with previous methods, these algorithms can keep both of the intra and inter manifolds geodesics faithfully. The features and effectiveness of the proposed multi-manifold learning algorithms are demonstrated and compared through experiments.

The path following algorithm was proposed recently to approximately solve the matching problems on undirected graph models and exhibited a state-of-the-art performance on matching accuracy. In this paper, we extend the path following algorithm to the matching problems on directed graph models by proposing a concave relaxation for the problem. Based on the concave and convex relaxations, a series of objective functions are constructed, and the Frank-Wolfe algorithm is then utilized to minimize them. Several experiments on synthetic and real data witness the validity of the extended path following algorithm.

In this letter, we propose a semi-supervised dimensionality reduction method named Discriminative Sparsity Preserving Projection (DSPP). In order to get the feature mapping$f$which projects the high-dimensional data into a low-dimensional intrinsic space, DSPP attempts to maintain the prior low-dimensional representation constructed by the data points and the known class labels and, meanwhile, considers the complexity of$f$in the ambient space and the smoothness of$f$in preserving the sparse representation of data. On one hand, the DSPP method obtains an explicit nonlinear feature mapping for the out-of-sample extrapolation. On the other hand, the DSPP method has a high discriminative ability which is inherited from the sparse representation of data. Experiment results show the effectiveness of the proposed method.

Opportunistic routing (OR) involves multiple forwarding candidates to relay packets by taking advantage of the broadcast nature and multi-user diversity of the wireless medium. Compared with Traditional Routing (TR), OR is more suitable for the unreliable wireless link, and can evidently improve the end to end throughput of Wireless Mesh Networks (WMNs). At present, there are many achievements concerning OR in the single radio wireless network. However, the study of OR in multi radio wireless network stays the beginning stage. In this paper, we focus on OR in multi-radio multi-channel WMNs. We validate the advantage of OR in multi-radio multi-channel WMNs, and propose a Simple Channel Assignment for Opportunistic Routing (SCAOR), which assigns channel to flows. According to interference state of every node, SCAOR assigns a channel with minimum interference to each flow to balance channel load. The simulation result shows OR of dual-radio dual-channel WMNs can promote throughput evidently, specifically, 16.8% higher than throughput of TR in the dual-radio dual-channel WMNs, 87.11% and 111.8% higher than throughput of OR and TR in single-radio single-channel, respectively.

Manifold learning is a fast growing area of research recently. The main purpose of manifold learning is to search for intrinsic variables underlying high-dimensional inputs which lie on or are close to a low-dimensional manifold. Different from current theoretical works and applications of manifold learning approaches, in our work manifold learning framework is transferred to tracking feature extraction for the first time. The contributions of this article include three aspects. Firstly, in this article, we focus on tracking feature extraction for dynamic visual tracking on dynamic systems. The feature extracted in this article is based on manifold learning framework and is particular for dynamic tracking purpose. It can be directly applied to system control of dynamic systems. This is different from most traditional tracking features which are used for recognition and detection. Secondly, the proposed tracking feature extraction method has been successfully applied to three different dynamic systems: dynamic robot system, intelligent vehicle system and aircraft visual navigation system. Thirdly, experimental results have proven the validity of the tracking method based on manifold learning framework. Particularly, in the tracking experiments the vision system is dynamic. The tracking method is also compared with the well-known mean-shift tracking method, and tracking results have shown that our method outperforms the latter.

Several adaptive visual tracking algorithms have been recently proposed to capture the varying appearance of target. However, adaptability may also result in the problem of gradual drift, especially when the target appearance changes drastically. This paper gives some theoretical principles for online learning of target model, and then presents a novel adaptive tracking algorithm which is able to effectively cope with drastic variations in target appearance and resist gradual drift. Once target is localized in each frame, the patches sampled from target observation are first classified into foreground and background using an effective classifier. Then the adaptive, pure and time-continuous target model is extracted online through two processes: absorption process and rejection process, through which only the reliable features with high separability are absorbed in the new target model, while the “dangerous” features which may cause interfusion of background patterns are rejected. To minimize the influence of background and keep the temporal continuity of target model, two collaborative models dominant model and continuous model are designed. The proposed learning and generation mechanisms of target model are finally embedded in an adaptive tracking system. Experimental results demonstrate the robust performance of the proposed algorithm under challenging conditions.

Manifold learning has been a popular method in many areas such as classification and recognition. In this paper, we propose a novel algorithm for pedestrian tracking based on our previous work on manifold learning. A new kind of manifold subspace is introduced, in which the intrinsic features of the target's motion can be best preserved, and the dimensionality of feature is very low. In the proposed subspace, variations of continuous pedestrian postures can be represented well by these intrinsic features. This also validates our conjecture that the movement of pedestrians can be described by some intrinsic and low-dimensional features, which are significant for tracking. Although intrinsic features are useful for tracking, algorithms that directly apply intrinsic features could not guarantee stable performance due to the influence from a complicated background. To address this issue, a foreground extraction method is introduced to enhance tracking stability by selecting the most discriminative color features to automatically distinguish the foreground from the candidate image. This preprocessing stage is proven to promote the accuracy of low-dimensional feature representation in pedestrian tracking. The whole tracking procedure, particularly dimensionality reduction, is linear and fast without complicated calculations. The experimental results validate the effectiveness of our algorithm under challenging conditions, such as a complex background, various pedestrian postures, and even occlusion.

In general, localization is a very important step in the manufacturing process, which can be considered as a prior process of assembly, machining, transportation, etc. Localization can be achieved with or without sensors. Compared with localization with sensors, localization without sensors can be more reliable, cheaper, and can have lower requirements on the environment. For example, localization without sensors can be achieved in a dark environment. However, it is more restrictive to the condition of the system. Localization of 2-D objects without sensors has been deeply investigated. Some theoretical results on 3-D-object localization without sensors have been given. On the other hand, some work has been carried out that tries to find methods to reduce uncertainties in the 3-D orientation of a polyhedron (i.e., the orientation of the polyhedron in a 3-D space). However, to the best of our knowledge, no practical and effective methods have been proposed, so far, to localize a polyhedron from any initial 3-D orientation to a unique 3-D orientation without sensors. This paper aims to find conditions and strategies for 3-D objects to be rotated from an unknown initial stable state to a unique stable state without sensors. The main contributions of this paper are given as follows: 1) It is discovered and proved that there are two classes of 3-D objects that can be rotated into a unique state from an arbitrary initial state without sensory feedback. 2) For these two classes of objects, the practical strategies are presented, and one example for each class is given to show the validity of the strategy. 3) Based on the above results, the robotic system and localization operations are illustrated, and some experimental results are given.

Classification-based pedestrian detection systems (PDSs) are currently a hot research topic in the field of intelligent transportation. A PDS detects pedestrians in real time on moving vehicles. A practical PDS demands not only high detection accuracy but also high detection speed. However, most of the existing classification-based approaches mainly seek for high detection accuracy, while the detection speed is not purposely optimized for practical application. At the same time, the performance, particularly the speed, is primarily tuned based on experiments without theoretical foundations, leading to a long training procedure. This paper starts with measuring and optimizing detection speed, and then a practical classification-based pedestrian detection solution with high detection speed and training speed is described. First, an extended classification/detection speed metric, named feature-per-object (fpo), is proposed to measure the detection speed independently from execution. Then, an fpo minimization model with accuracy constraints is formulated based on a tree classifier ensemble, where the minimum fpo can guarantee the highest detection speed. Finally, the minimization problem is solved efficiently by using nonlinear fitting based on radial basis function neural networks. In addition, the optimal solution is directly used to instruct classifier training; thus, the training speed could be accelerated greatly. Therefore, a rapid and accurate classification-based detection technique is proposed for the PDS. Experimental results on urban traffic videos show that the proposed method has a high detection speed with an acceptable detection rate and a false-alarm rate for onboard detection; moreover, the training procedure is also very fast.

Manifold learning is a hot topic in the field of computer science, particularly since nonlinear dimensionality reduction based on manifold learning was proposed in Science in 2000. The work has achieved great success. The main purpose of current manifold-learning approaches is to search for independent intrinsic variables underlying high dimensional inputs which lie on a low dimensional manifold. In this paper, a new manifold is built up in the training step of the process, on which the input training samples are set to be close to each other if the values of their intrinsic variables are close to each other. Then, the process of dimensionality reduction is transformed into a procedure of preserving the continuity of the intrinsic variables. By utilizing the new manifold, the dynamic tracking of a human who can move and rotate freely is achieved. From the theoretical point of view, it is the first approach to transfer the manifold-learning framework to dynamic tracking. From the application point of view, a new and low dimensional feature for visual tracking is obtained and successfully applied to the real-time tracking of a free-moving object from a dynamic vision system. Experimental results from a dynamic tracking system which is mounted on a dynamic robot validate the effectiveness of the new algorithm.

Sensorless localization of 3D objects has been a significant research topic for many years. Researchers have focused on this problem from both theoretical and practical perspective where the goal is to reduce uncertainties in the orientation of a 3D object. However, to the best of our knowledge, no effective practical methods have been proposed so far to localize a polyhedron from any initial orientation to a unique orientation without sensors. In our previous work [1], two broad classes of 3D objects have been introduced, which can be localized from an arbitrary state to a unique state on a flat plane (the surface resting on the flat plane is established) without sensors. In this paper, a much broader class of polyhedra is introduced, which can be localized to a unique state without sensors. The main contributions of this paper are given as follows: • It is found that a polyhedron with an arbitrary initial state on the flat plane can be rotated to a fixed orientation (the orientation of the surface resting on the flat plane is fixed), provided that the polygon corresponding to each surface of the polyhedron can be oriented to a unique orientation in a 2D space. The method of rotating the polyhedron to a fixed orientation is given. • Base on the above result, both conditions and the strategy are given for a polyhedron to be localized to a unique state. • An example is given to show the validity of the strategy.

Tracking multiple people in crowded and cluttered dynamic scenes is a very difficult task in robotic vision due to the highly frequent occlusion and lack of visibility of objects. In this paper, we present a manifold learning based multiple people tracking approach with occlusion reasoning to solve this problem. In our previous work, a new Intrinsic Variable Preserving Manifold Learning (IVPML) method is proposed, by which the continuity of the intrinsic motion variables for tracking is preserved on a new manifold after dimensionality reduction. In this paper, the IVPML method is extended to be applied to tracking multiple people with occlusion situations. Associated with spatio-temporal continuity of tracking and IVPML method, a novel robust occlusion reasoning method is proposed during the alternations of multiple people. For occlusion recovery, region covariance representation including both spatial and statistic properties of objects are used to detect people after occlusion. The multiple people tracking method has been successfully applied to mobile robotic visual tracking system in several complicated environments. Comparisons and experimental results have shown the effectiveness of the new algorithm in various situations.

The iterative closet point (ICP) method is a dominant method for data registration that has attracted extensive attention. In this paper, a unified mathematical model of ICP based on Lie group representation is established. Under the framework, the registration problem is formulated into an optimization problem over a certain Lie group. In order to simplify the model and to reduce the dimension of parameter space, the translation part of geometric transformation is eliminated by calibrating the centers of two data sets under registration. As a result, a fast algorithm by solving an iterative linear system is designed for the optimization problem on Lie groups. Moreover, PCA and ICA methods are jointly applied to estimate the initial registration to achieve the global minimum. Finally, several illustrations and comparison experiments are presented to test the performance of the proposed algorithm.

In this paper, we are concerned with the registration of two 3D data sets with large-scale stretches and noises. First, by incorporating a scale factor into the standard iterative closest point (ICP) algorithm, we formulate the registration into a constraint optimization problem over a 7D nonlinear space. Then, we apply the singular value decomposition (SVD) approach to iteratively solving such optimization problem. Finally, we establish a new ICP algorithm, named Scale-ICP algorithm, for registration of the data sets with isotropic stretches. In order to achieve global convergence for the proposed algorithm, we propose a way to select the initial registrations. To demonstrate the performance and efficiency of the proposed algorithm, we give several comparative experiments between Scale-ICP algorithm and the standard ICP algorithm.

The ultimate purpose of a pedestrian-detection system (PDS) is to reduce pedestrian-vehicle-related injury. Most such systems tend to adopt expensive sensors, such as infrared devices, in expectation of better performance. In comparison, a low-cost optical-camera-based system has much potential practical value, including a greater detection range, and can easily be trained to detect other objects. However, such low-cost systems are difficult to design (e.g., little original information can be collected, and the scene is very complex). To address these problems, an effective and reliable classifier is needed. The classifier should have a proper structure, its features need to be well selected, and a large number of high-quality samples are necessary for training. In this paper, we present a low-cost PDS which only uses a single optical camera. We design a cascade classifier to achieve an effective and reliable detection. First, our system scans two sequential frames at each zoom scale with a sliding window. Second, with each window, both appearance and motion features are extracted. A well-trained cascade classifier, combining statistical learning with a decomposed support-vector-machine classifier, then determines whether the window contains a human body. At the same time, to provide as much information as possible about the pedestrian, a small-scale weighted template tree trained by a coevolutionary algorithm is adopted to identify each pedestrian's direction, and the distance of each from the vehicle is also provided using an estimation algorithm. During the training procedure, we select key features by using the AdaBoost algorithm and a large number of high-quality samples. Experimental results demonstrate that the system is suitable for pedestrian detection in city traffic: The detection speed is more than 10 ft/s, the detection rate reaches 80%, and the false positive rate is no more than 0.30/00.

This paper presents an efficient and robust tracking algorithm based on multiple cues fusion in the Bayesian framework. This method characterizes the object to be tracked using a MOG (mixture of Gaussians) based appearance model and a chamfer-matching based shape model. A selective updating technique for the models is employed to accommodate for appearance and illumination changes. Meantime, the mean shift algorithm is embedded as the prior information into the Bayesian framework to give a heuristic prediction in the hypotheses generation process, which also alleviates the great computational load suffered by the conventional Bayesian tracker. Experimental results demonstrate that, compared with some existing works, the proposed algorithm has a better adaptability to changes of the object as well as the environments.

By minimizing the mean square reconstruction error, multisets mixture learning (MML) provides a general approach for object detection in image. To calculate each sample reconstruction error, as the object template is represented by a set of contour points, the MML needs to inefficiently enumerate the distances between the sample and all the contour points. In this paper, we develop the line segment approximation (LSA) algorithm to calculate the reconstruction error, which is shown theoretically and experimentally to be more efficient than the enumeration method. It is also experimentally illustrated that the MML based algorithm has a better noise resistance ability than the generalized Hough transform (GHT) based counterpart.

To deal with the problem of tracking person with a mobile robot in dynamic scene with occlusion, a tracking system is presented in this paper. We focus on the individual person tracking system, which is applied on our two-wheel mobile robot with a single PTZ (pan-tilt-zoom) camera. The tracking system works reliably in the dynamic environment with partial/complete occlusion. For the sake of complete occlusion, we use several filters and spatial relation restriction to represent the target model. These filters are all based on the appearance of the target, and each filter models a part of human body which is more rigid than the entire body. To construct a tracking system, the situation-based strategy and a simple frame-to-frame tracker are involved in the system, and an improved mean-shift tracking algorithm is applied as the tracker in this paper. Finally, experimental results on tracking individual person with complete occlusion in different environments are shown, which demonstrate the robustness and effectiveness of the algorithm

Gilbert algorithm is a very popular algorithm in collision detection in robotics and also in classification in pattern recognition. However, the major drawback of Gilbert algorithm is that in many cases it becomes very slow as it approaches the final solution and the vertices selection vibrates in these cases. In this paper: a) It is proven theoretically that when the selection of vertices vibrates among several points, the algorithm will converge to the hyperplane determined by these points. b) Based on the above results, an improved Gilbert algorithm for computing the distance between two convex polytopes is presented. The algorithm can avoid the slow convergence of the original one. Numerical simulation results demonstrate the effectiveness and advantage of the improved algorithm

In a pedestrian detection system, a classifier is usually designed to recognize whether a candidate is a pedestrian. Support vector machines (SVM) has become a primary technique to train a classifier for pedestrian detection. However, it is hard to give the best training model which has a tremendous effect to the performance of a SVM classifier. In this paper, we design special code/decode scheme and evaluation function for a training model firstly; and then use genetic algorithm to optimize key parameters which represent the SVM training model. Therefore a most suitable SVM classifier can be obtained for pedestrian detection. Experiments have been carried out in a single camera based pedestrian detection system. The results show that the evolutionary SVM classifier has a better detection rate; moreover, RBF kernel is more suitable than polynomial kernel when chosen in an evolutionary SVM classifier for pedestrian detection

Mean shift, for its simplicity and efficiency, has achieved a considerable success in robot tracking. For the mean shift based tracking algorithm, the scale of the mean-shift kernel bandwidth is a crucial parameter which reflects the size of tracking window. However, in literature how to properly update or select the bandwidth remains a tough task as the size of the object under consideration changes. In this paper, a weighted average integral projection approach is proposed to extract the local information of the object, and then a multiinformation fusion strategy is suggested for the scale selection, which combines both the global and local information of the sample weight image. Moreover, a coarse-to-fine approximate approach is employed to accelerate the procedure. Experimental results demonstrate that, compared to some existing works, the strategy proposed has a better adaptability as the size of the object changes in clutter environments.

Support vector machines (SVM) decomposition methods were proposed to solve high dimensional and/or large data classification problems. Two major decomposition algorithms: Karush-kuhn-Tucker (KKT) condition based algorithm, and 'Joachims' decomposition algorithm are popularly adopted. In this paper, both these two decomposition methods are analyzed and applied into face recognition with three basic mapping kernels. Numerical results showed that: a) face recognition with SVM performs better accuracy than other existed methods; b) the decomposition methods can perform face recognition efficiently; c) Joachims' decomposition method has better accuracy than that of decomposition algorithm based on KKT condition; d) linear kernel can provide much higher recognition accuracy than polynomial and slightly better accuracy than Gaussian radial based function (RBF) kernel; Also due to the fact that the linear kernel method is much simpler than others, it is most suitable for face recognition.

In our previous work (Qiao, 2000), the concept of "generalized attractive region" in strategy investigation has been proposed. With this simple concept, a logical way to form strategies to achieve high precision tasks with minimum sensory feedback was achieved. The concept has been successfully applied to 3D high precision sensor-less robotic assembly, 2D sensor-less robotic orientation system and 2D sensor-less posing system with a four-pin gripper. The work was reported by "Advanced Manufacturing Technology Alert" (1999). In this paper, the conditions of the environment in robotic system, which can form attractive regions, are systematically analyzed in theory. Through analysis and simulation results: 1) it shows that there are always some strategic attractive regions which exist in the allowable region of the 3D polyhedral part with a flat environment; 2) therefore the pushing or grasping plan can be given directly; 3) also the vertex which touches the fence or the gripper in given orientation is given directly; and 3) it has been proven that on each rotational vector direction (which lies on any direction on XOY plane) and with the same contact vertex, the distance between the center of the part and the fence is a sinusoid function of the rotation amplitude. Furthermore, the amplitude and the initial angle of the sinusoid function is also investigated.

In this paper, a novel methodology called a reference model approach to stability analysis of neural networks is proposed. The core of the new approach is to study a neural network model with reference to other related models, so that different modeling approaches can be combinatively used and powerfully cross-fertilized. Focused on two representative neural network modeling approaches (the neuron state modeling approach and the local field modeling approach), we establish a rigorous theoretical basis on the feasibility and efficiency of the reference model approach. The new approach has been used to develop a series of new, generic stability theories for various neural network models. These results have been applied to several typical neural network systems including the Hopfield-type neural networks, the recurrent back-propagation neural networks, the BSB-type neural networks, the bound-constraints optimization neural networks, and the cellular neural networks. The results obtained unify, sharpen or generalize most of the existing stability assertions, and illustrate the feasibility and power of the new method.

In our previous work, the concept of generalized attractive region in strategy investigation has been proposed. With this simple concept, a logical way to form strategies to achieve high precision tasks with minimum sensory feedback was achieved. The concept has been successfully applied to some of robotic assembly, robotic pushing and robotic grasping systems. In this paper, the conditions of the environment, which can form attractive regions, are systematically analyzed in theory. Through simulation results, it shows that the robotic assembly, robotic grasping and robotic pushing systems satisfy these conditions and therefore there are some attractive regions formed by the environment which can be used to achieve high precision sensorless manipulation.

It is found that the attractive regions exist in the motion region of a polygonal part in 2D during a pushing process by a fence or grasping process by a parallel jaw gripper. Taking a triangular part as an example, the paper provides the strategies to move the polygonal part, from any pose on the plane to a unique pose (by grasping) or a pair of symmetric poses (by grasping) of a parallel jaw gripper. The key point is to enable more than one stable state in the last stage to stay in one local attractive region in the current stage. Furthermore the relationship between the shape of the polygonal part and the action steps can be easily deduced due to the number of attractive regions are given clearly. It is also found that there are also many attractive regions in motion regions of a polyhedral part in 3D during a pushing process by a fence or grasping process by a parallel jaw gripper. These attractive regions in the motion region of the polygonal or polyhedral part can also be used to help the identification of the objects on the same conveyer with incomplete information.

In this paper, a new concept called nonlinear measure is introduced to quantify stability of nonlinear systems in the way similar to the matrix measure for stability of linear systems. Based on the new concept, a novel approach for stability analysis of neural networks is developed. With this approach, a series of new sufficient conditions for global and local exponential stability of Hopfield type neural networks is presented, which generalizes those existing results. By means of the introduced nonlinear measure, the exponential convergence rate of the neural networks to stable equilibrium point is estimated, and, for local stability, the attraction region of the stable equilibrium point is characterized. The developed approach can be generalized to stability analysis of other general nonlinear systems.

A novel neural network is proposed in this paper for realizing associative memory. The main advantage of the neural network is that each prototype pattern is stored if and only if as an asymptotically stable equilibrium point. Furthermore, the basin of attraction of each desired memory pattern is distributed reasonably (in the Hamming distance sense), and an equilibrium point that is not asymptotically stable is really the state that cannot be recognized. The proposed network also has a high storage as well as the capability of learning and forgetting, and all its components can be implemented. The network considered is a very simple linear system with a projection on a closed convex set spanned by the prototype patterns. The advanced performance of the proposed network is demonstrated by means of simulation of a numerical example.

Most industrial grippers now in use are two-fingered. Among them the parallel-jaw gripper is the simplest. It can partially remove the pose uncertainty of an object through grasping, such as the orientation uncertainty. This paper addresses a new type of grippers with the finger configuration of four circles instead of two parallel lines. It has a number of important advantages. Especially, it achieves form-closure and confines the object to a locally unique pose, so as to remove the pose uncertainty completely. It allows the gripped object to reach this pose freely without loss of required friction in the direction perpendicular to the grasping plane. More information can be acquired for identifying the object and its grasp mode. As a result the identification can be performed at one grasp. The key parameter of a symmetric four-pin gripper is the distance (span) between two pin centers on each finger, which depends upon the object shape and impacts the closure property, Based on a new approach to the grasp geometry, selection and limitations of the span are illustrated.

This paper presents a new data representation method called direction quantization representation (DQR) which is motivated by a simplified geometric model of biological compound eye and used in describing the shape of convex hulls of objects. Advantages of DQR include high efficiency and stability in numerical computation, convenience for semidynamic maintenance, suitability for parallel implementation, and applicability to various convex set related problems. Several practical applications are presented which show the feasibility and powerfulness of DQR.

In general, it is believed that the sensor-less manipulation can only achieve low-precision or simple tasks. However, it is not always true. We point out that there are two attractive regions. In the motion space of the dynamic system with an active control system, which bases on sensory feedback, there is an attractive region, which forces the system to follow the nominal trajectory with the uncertainty of the system. At the same time, there is another attractive region, which widely exists in the configuration space of the dynamic system under constraints from environment. Both of these two attractive regions can be used to eliminate the uncertainty of the system states in order to achieve high-precision tasks. This paper focuses on the effective use of attractive regions in the environment to eliminate the uncertainty. However, in general, sensors are still useful and play an important role in many systems. Thus this work is related to the approach on the relationship between required information and strategy investigation.

An autonomous robotic manipulator can reduce the uncertainty in the locations of the objects by sensing, by motion strategies or by both. Compared with manipulation with sensors, sensor-less manipulation can provide a simple method to eliminate the uncertainty and can also avoid all the problems caused by sensors. Sensor-less motion carried out (i) to achieve high-precision tasks and (ii) in three-dimensional space is considered too difficult. The limited application range of sensor-less manipulation is regarded as its major disadvantage with respect to the strategies with sensors used. Robotic peg-hole assembly, as the most common mechanical automated task, is a long-standing problem in robotics. In our previous work, several strategies for the robotic peg-hole assembly were designed and were applied to experiments successfully. These strategies can program the robots to perform the assembly in three-dimensional space with the precision higher than that of the robots being used, without additional devices such as force sensors or flexible wrists. The approach provided a good example of the high-precision sensor-less manipulation in three-dimensional space. The purpose of this paper is not to design new strategies in robotic assembly. It provides a logical method in sensor-less motion planning and applies it to robotic assembly. Many strategies used in Inoue's (1979) work in robotic assembly with force sensors and our previous work in robotic assembly without force sensors are derived from this basic premise and other new strategies can be designed. It is expected that sensor-less motion planning can be investigated for many a wide variety of tasks based on this approach.