Weakly supervised tumor localization in whole-slide images (WSIs) remains challenging due to the absence of region level annotations and the difficulty of capturing diagnostically relevant cellular organization under gigapixel resolution. Most existing multiple instance learning (MIL) methods rely primarily on patch-level appearance features, overlooking the nuclear structural patterns that underlie pathological interpretation. We propose Nuclear Graph-Guided Multiple Instance Learning (NGG-MIL), a weakly supervised framework that explicitly incorporates nuclear-level structural information into WSI analysis. Within each patch, nuclei are represented as a Direction Weighted Nuclear Graph, where edges encode both spatial proximity and nuclear orientation consistency. Graph Neural Network (GNN) is employed to learn structure-aware patch representations, which are subsequently aggregated through Attention Based MIL for slide-level prediction. The learned patch attention scores are spatially reprojected to generate interpretable tumor probability maps without requiring region-level supervision. Extensive experiments on three public breast cancer WSI datasets (CAMELYON16, TCGA-BRCA, and BRACS) demonstrate that NGG-MIL consistently surpasses strong MIL baselines in both slide-level classification performance and interpretability of localization maps, achieving consistent AUC improvements and competitive F1 scores across datasets.

This paper centers on the distributed fault-tolerant control (DFTC) issues of time-varying delayed nonlinear multiagent systems (TVDNMASs) with switching topologies and external disturbances by considering multiple faults and event-triggered consensus strategy (ETCS). The switching topologies satisfy generally uncertain semi-Markovian switching topologies (GUSMSTs), and they contain uncertain and partially unknown semi-Markovian transition rates (TRs). In addition, the ETCS is adopted in this paper to decide the update of controllers, which alleviates the load of the correspondence network. In view of Lyapunov-Krasovskii functional (LKF), the tracking control protocols are presented to guarantee the DFTC of nonlinear multiagent systems (MASs). Moreover, the controller gain and observer gain matrices are derived through the solution of linear matrix inequalities (LMIs). Finally, a simulation example is proposed to exhibit the capability of our design technique. Note to Practitioners—Due to the complexity of engineering environment, the cooperative control of MASs has gained widespread attention. Nowadays, the MASs are generally utilized in diverse fields, such as multi-motor synchronization, drone swarm formation, and smart grids. As one of the significant research interests in cooperative control, the consensus control of MASs has become a research hotspot. However, in practical applications, due to stochastic system failures and sudden changes in the external environment, it is hard for the fixed communication topologies to cope with these unexpected situations. Therefore, the DFTC issues of delayed nonlinear MASs with GUSMSTs and external disturbances by considering multiple faults are investigated in this paper. Moreover, the mode-dependent distributed time-delay intermediate observers and active fault-tolerant consensus controllers are designed on the basis of distributed fault-tolerant control consensus protocol.

Existing active learning methods for object detection face challenges, such as the lack of ground truth labels for regression loss, insufficient representation of unlabeled instance samples information, and discrepancies in information quality between image-level and multiple anchor-level instances. To address these issues, we propose an active learning method for object detection with vectorized dual pseudo loss and multiple instance offset constraint. This method implements a two-stage framework. The first stage focuses on evaluating the information quality of detection images. We first pioneer a dual pseudo loss formulation that provides theoretically grounded regression loss estimation. The regression loss is calculated as the norm of the offset discrepancy loss vector between the enhanced and original base box vector, further constrained by the cosine value of the angle between the anchor box feature and regressor parameters vector. The distance entropy from the base box feature vector to each category's feature prototype vector is used as a weighting factor for the regression and classification information quality of instance samples. Subsequently, the second stage employs diversity-driven sampling on high-information images, leveraging instance-level cosine similarity to effectively remove redundant images. The proposed method outperforms state-of-the-art active learning approaches for object detection on PASCAL VOC and MS COCO datasets. Additionally, the proposed dual pseudo regression loss robustly captures regression information quality, demonstrating its effectiveness for active learning in object detection.

Single image novel view synthesis allows the generation of target images with different views from a single input image. Pixel generation methods are one of the main approaches for novel view synthesis, with previous methods typically using the input image to infer the target image in the new view. However, only features from input images in the source view might not be sufficient to generate a good target image, especially when only a single input image is available. In this paper, we fuse features from an input and a warped image to collaboratively generate pixels in the new view, with the warped image as an intermediate output generated by projecting pixels of the input image onto the target view via an estimated depth. Since the estimated depth and the generated warped image are not perfect, errors will be introduced when generating target pixels. To alleviate these and to ensure better channel information between the features from input and warped image, channel attention blocks are employed. In addition, in order to use skip connections for better novel view synthesis results, encoder features in different layers from the input image are transformed to the target view via multi-resolution depths. Here, instead of downsampling a single full-resolution depth to several lower-resolution depths, we adopt a multi-scale depth estimation network to predict multiple depths at different resolutions. Experimental results on benchmark datasets show that our method gives excellent view synthesis results and outperforms other state-of-the-art novel view synthesis methods.

Abstract Due to the emergence of graph convolutional networks (GCNs), the skeleton-based action recognition has achieved remarkable results. However, the current models for skeleton-based action analysis treat skeleton sequences as a series of graphs, aggregating features of the entire sequence by alternately extracting spatial and temporal features, i.e., using a 2D (spatial features) plus 1D (temporal features) approach for feature extraction. This undoubtedly overlooks the complex spatiotemporal fusion relationships between joints during motion, making it challenging for models to capture the connections between different temporal frames and joints. In this paper, we propose a Multimodal Graph Self-Attention Network (MGSAN), which combines GCNs with self-attention to model the spatiotemporal relationships between skeleton sequences. Firstly, we design graph self-attention (GSA) blocks to capture the intrinsic topology and long-term temporal dependencies between joints. Secondly, we propose a multi-scale spatio-temporal convolutional network for channel-wise topology modeling (CW-TCN) to model short-term smooth temporal information of joint movements. Finally, we propose a multimodal fusion strategy to fuse joint, joint movement, and bone flow, providing the model with a richer set of multimodal features to make better predictions. The proposed MGSAN achieves state-of-the-art performance on three large-scale skeleton-based action recognition datasets, with accuracy of 93.1% on NTU RGB+D 60 cross-subject benchmark, 90.3% on NTU RGB+D 120 cross-subject benchmark, and 97.0% on the NW-UCLA dataset. Code is available at https://github.com/lizaowo/MGSAN .

This article focuses on the issue of novel dynamic event-triggered consensus control of multiagent systems (MASs) with denial-of-service (DoS) attacks. Different from the conventional Markovian switching topologies, the generally uncertain semi-Markovian (GUSM) switching topologies with partially unknown elements and time-dependent uncertainties are constructed for the leader-following MASs by considering the equipment performance and external uncertain environment influence. To save communication resources, the novel dynamic memory event-triggered strategy (DMETS) is presented to decrease the frequency of communication between agents. Some secure consensus control criteria are established for the MASs with GUSM switching topologies and DoS attacks due to the potential system communication disruption caused by attackers. Finally, two physical system examples are designed to prove the effectiveness of the presented method.

Landslides refer to occurrences of massive ground movements due to geological (and meteorological) factors, and can have disastrous impacts on property, economy, and even lead to the loss of life. The advances in remote sensing provide accurate and continuous terrain monitoring, enabling the study and analysis of land deformation which, in turn, can be used for land deformation prediction. Prior studies either rely on predefined factors and patterns or model static land observations without considering the subtle interactions between different point locations and the dynamic changes of the surface conditions, causing the prediction model to be less generalized and unable to capture the temporal deformation characteristics. To address these issues, we present DyLand, a dynamic manifold learning framework that models the dynamic structures of the terrain surface. We contribute to the land deformation prediction literature in four directions. First, DyLand learns the spatial connections of interferometric synthetic aperture radar (InSAR) measurements and estimates the conditional distributions on a dynamic terrain manifold with a novel normalizing flow-based method. Second, instead of modeling the stable terrains, we incorporate surface permutations and capture the innate dynamics of the land surface while allowing for tractable likelihood estimations on the manifold. Third, we formulate the spatiotemporal learning of land deformations as a dynamic system and unify the learning of spatial embeddings and surface deformation. Finally, extensive experiments on curated real-world InSAR datasets (land slopes prone to landslides) show that DyLand outperforms existing benchmark models.

After several years of development, deep synthesis technology has made significant progress in image and video synthesis. Deep forgery represented by Deepfakes has become a research hotspot, which is used as a tool for disinformation attacks. The current strongly discriminative models can have good performance on specific datasets, even close to 100% accuracy. Unfortunately, since a specific discriminative method only fits a specific data distribution, and different forgery methods or datasets have different data distributions. These methods fail to achieve high performance in cross-dataset detection. In response to this problem and focusing on the actual situation, we adjust the strong generalization detection across the dataset to the generalization detection of unseen fake video. We propose Multi-Crise-Cross Attention and StyleGANv2 Generative Adversarial Network (MCS-GAN). Firstly, we built a Generative Adversarial Network (GAN) framework to learn the distribution of real face data and generate corresponding face images. Secondly, to break the high stitch between the fake region and the background, the model needs to have strong enough feature analysis and pixel restoration capabilities. Therefore, we propose a generator consisting of a Multi-Crise-Cross-Attention (MC) encoder and a StyleGANv2 (SG2) decoder. Finally, to avoid the situation where as long as a face is normal or different faces are abnormal, we set a latent space encoding discriminator and increase the ratio of latent space vector, so as to detect anomaly generated by the forgery operation acting on latent space. We conduct some model generalization experiments on videos on the Internet and some popular deepfake databases. The results show that the accuracy of our method is better compared with the best methods.

This article focuses on the event-triggered consensus control (ETCC) issue of the time-varying delayed leader-following nonlinear multiagent systems (TVDLFNMASs). In order to minimize the influence on uncertain factors of the information transmission and the data information loss, the switching topologies are constructed as the generally uncertain Markovian jumping forms whose transition rates include completely unknown elements and estimate values of uncertain elements. In addition, the event-triggered (ET) transmission strategy is given based on the threshold parameter and the ET matrix to relieve the communication burden of TVDLFNMASs. The new leader-following (LF) consensus conditions and control gains are obtained based on ET strategy. Finally, the effectiveness of the ET consensus criteria is demonstrated in the simulation section.

Video understanding has been enhanced by the use of multi-modal networks. However, recent multi-modal video analysis models have limited applicability to sports videos due to their specialised nature. This paper proposes a novel attention-based multi-modal neural network for sports event classification featuring a multi-stage fusion training strategy. The proposed multi-modal neural network integrates three modalities, including an image sequence modality, an audio modality and a newly proposed sports formation modality, to improve the sports video classification performance. Empirical results show that the proposed model outperforms the state-of-the-art transformer-based video method by 4.43% on top-1 accuracy on Soccernet-V2 dataset.

This paper develops a new approach to estimate predicted class probabilities in deep Spiking Neural Networks (SNN) that encourages faster classification. The proposed approach utilizes the temporal separation between the first spikes generated by the output neurons to estimate the predicted class probabilities which are then used with cross entropy loss for training the network. This maximizes the separation between the first spikes generated by the neuron associated with the correct class and neurons associated with other classes. Higher classification performance is obtained by maximising the tem-poral separation, which also drives the correct class neuron to spike earlier in the simulation. As a consequence, the predicted class may be determined from the first spike in the output layer, leading to quicker classification. The sensitivity factor for each neuron in the network is estimated via error-backpropagation during training. Using Spike Timing Dependent Plasticity (STDP) regulated by the estimated sensitivity factors, the network weights are updated. It results that the learning method is termed as Temporal Separation Modulated Spike Timing Dependent Plasticity (TSM-STDP). On the benchmark MNIST dataset, the performance of TSM-STDP has been assessed, and the evaluation results are compared with those of other learning methods for SNNs. Additionally, a histogram of the output layer's first spikes demonstrated that the right class neurons spiked earlier in the simulation than other class neurons, enabling faster classification. On real-world Attention Deficit Hyperactivity Disorder (ADHD) detection dataset, the effectiveness of TSM-STDP has also been assessed and compared with other available approaches. The per-formance comparison results clearly show that TSM-STDP can achieve classification performance comparable to other existing learning algorithms on benchmark and real-world datasets while requiring less time for classification.

With the widespread applications of Deep Neural Networks (DNNs), the safety of DNNs has become a significant issue. The vulnerability of the neural networks against adversarial examples deepens concerns about the safety of DNNs applications. This paper proposed a novel defence method to improve the adversarial robustness of DNN classifiers without using adversarial training. This method introduces two new loss functions. First, a zero-cross-entropy loss is used to punish overconfidence and find the appropriate confidence for different instances. Second, a logit balancing loss is proposed to protect DNNs from non-targeted attacks by regularising incorrect classes’ logits distribution. This method achieved competitive adversarial robustness compared to advanced adversarial training methods. Meanwhile, a novel robustness diagram is proposed to analyse, interpret and visualise the robustness of DNN classifiers against adversarial attacks. Furthermore, a Log-Softmax-pattern-based adversarial attack detection method is proposed. This detection method can distinguish clean inputs and multiple adversarial attacks via one multi-classification MLP. In particular, it is state-of-the-art in identifying white-box gradient-based attacks; it achieved at least 95.5% accuracy for classifying four white-box gradient-based attacks with maximum 0.1% false positive ratio.

This article focuses on the dissipativity-based consensus tracking control (DBCTC) problems of time-varying delayed leader-following nonlinear multiagent systems (LFNMASs) with the event-triggered transmission strategy. The switching topologies of the LFNMASs are subject to the uncertain and partially unknown generally Markovian jumping process. The control inputs of the following agents are updated according to the proposed event-triggered transmission strategy, which could reduce the communication burden. Based on the event-triggered transmission condition and distributed consensus protocol, some dissipativity-based criteria obtained by adopting the delay-product-term Lyapunov-Krasovskii functional (DPTLKF) and higher order polynomial-based relaxed inequality (HOPRII) are proposed to guarantee the LFNMAS consensus. The validity of the main results is verified by two simulation examples.

This article focuses on the synchronization problem of delayed inertial neural networks (INNs) with generally uncertain Markovian jumping and their applications in image encryption. The random connection weight strengths and generally uncertain Markovian are discussed in the INNs model. Compared with most existing INNs models that have constant connection weight strengths, our model is more practical because connection weight strengths of INNs may randomly vary due to the external and internal environment and human factor. The delay-range-dependent synchronization conditions (DRDSCs) could be obtained by adopting the delay-product-term Lyapunov-Krasovskii functional (DPTLKF) and higher order polynomial-based relaxed inequality (HOPRII). In addition, the desired controllers are obtained by solving a set of linear matrix inequalities. Finally, two examples are shown to demonstrate the effectiveness of the proposed results.

Graph convolutional networks (GCNs) are widely used in skeleton-based action recognition and have achieved excellent results. However, it is evident that the convolution operation can lead to losing some original input information. The incomplete utilisation of original input data limits GCNs’ ability to obtain the skeleton’s correlation. This paper proposes a graph instinctive attention convolutional network (GIAN) to solve this problem. In particular, it contains an instinctive attention module that uses self-attention to obtain the correlation within the original input skeleton. Then, parameter attention is used to further refine the relationship between different skeleton joints. Experimental results on publicly available datasets demonstrate that the GIAN outperforms most of the state-of-the-art algorithms.

Single image novel view synthesis allows the generation of target images with different views from a single input image. Pixel generation methods are one of the main approaches for novel view synthesis, with previous methods typically using the input images to infer the target image in the new view. However, only features from input images in the source view might not be sufficient to generate a good target image, especially when only a single input image is available. In this paper, we present a deep learning-based novel view synthesis approach that fuses features from an input and a warped image to collaboratively generate pixels in the new view. The warped image here is an intermediate output generated by projecting pixels of the input image onto the target view via an estimated depth. Since the estimated depth and the generated warped image are not perfect, errors will be introduced when generating target pixels. To alleviate these and to ensure better channel information between the features from input and warped image, channel attention blocks are employed. Experimental results on standard benchmark datasets show that our method produces excellent view synthesis results and outperforms other state-of-the-art methods.

In this paper we introduce a novel multi-object tracker based on the tracking-by-detection paradigm. This tracker utilises a Dynamic Bayesian Network for predicting objects’ positions through filtering and updating in real-time. The algorithm is trained and then tested using the MOTChallenge1 challenge benchmark of video sequences. After initial testing, a state-of-the-art residual neural network for extracting feature descriptors is used. This ResNet feature extractor is integrated into the tracking algorithm for object similarity estimation to further enhance tracker performance. Finally, we demonstrate the effects of object detection on tracker performance using a custom trained state of the art You Only Look Once (YOLO) V5 object detector. Results are analysed and evaluated using the MOTChallenge Evaluation Kit, followed by a comparison to state-of-the-art methods.

Abstract The essential of developing an advanced driving assistance system is to learn human-like decisions to enhance driving safety. When controlling a vehicle, joining roundabouts smoothly and timely is a challenging task even for human drivers. In this paper, we propose a novel imitation learning based decision making framework to provide recommendations to join roundabouts. Our proposed approach takes observations from a monocular camera mounted on vehicle as input and use deep policy networks to provide decisions when is the best timing to enter a roundabout. The domain expert guided learning framework can not only improve the decision-making but also speed up the convergence of the deep policy networks. We evaluate the proposed framework by comparing with state-of-the-art supervised learning methods, including conventional supervised learning methods, such as SVM and kNN, and deep learning based methods. The experimental results demonstrate that the imitation learning-based decision making framework, which ourperforms supervised learning methods, can be applied in driving assistance system to facilitate better decision-making when approaching roundabouts.

Recently, the visual quality evaluation of screen content images (SCIs) has become an important and timely emerging research theme. This article presents an effective and novel blind quality evaluation metric for SCIs by using stacked autoencoders (SAE) based on pictorial and textual regions. Since the SCI consists of not only the pictorial area but also the textual area, the human visual system (HVS) is not equally sensitive to their different distortion types. First, the textual and pictorial regions can be obtained by dividing an input SCI via an SCI segmentation metric. Next, we extract quality-aware features from the textual region and pictorial region, respectively. Then, two different SAEs are trained via an unsupervised approach for quality-aware features that are extracted from these two regions. After the training procedure of the SAEs, the quality-aware features can evolve into more discriminative and meaningful features. Subsequently, the evolved features and their corresponding subjective scores are input into two regressors for training. Each regressor can obtain one output predictive score. Finally, the final perceptual quality score of a test SCI is computed by these two predicted scores via a weighted model. Experimental results on two public SCI-oriented databases have revealed that the proposed scheme can compare favorably with the existing blind image quality assessment metrics.

We propose and implement a novel intelligent tutoring system, called RadarMath, to support intelligent and personalized learning for math education. The system provides the services including automatic grading and personalized learning guidance. Specifically, two automatic grading models are designed to accomplish the tasks for scoring the text-answer and formula-answer questions respectively. An education-oriented knowledge graph with the individual learner’s knowledge state is used as the key tool for guiding the personalized learning process. The system demonstrates how the relevant AI techniques could be applied in today's intelligent tutoring systems.

Human action recognition (HAR) from RGB videos is essential and challenging in the computer vision field due to its wide range of real-world applications in fields of human behaviour analysis, human-computer interactions, robotics and surveillance etc. Since the breakthrough and fast development of deep learning technology, the performance of HAR based on deep neural networks has been significantly improved in this decade. In this survey, we discuss the growing use of deep learning for HAR, such as representative two-stream and 3D CNNs, and particularly highlight most recent success achieved by using attention and transformers. We will provide our perspective on the new trend of designing innovative deep learning methods. In addition, we also present popular HAR datasets developed in recent years and benchmark accuracy achieved by current advancement in deep learning. This draws research attention to the challenges of HAR by identifying performance gaps when applying the deep learning methods on large HAR datasets. Further, this survey sheds light on the development of new methods and facilitates qualitative comparison with state of the art.

While full automation of road vehicles remains a future goal, shared-control and semiautonomous driving-involving transitions of control between the human and the machine-are more feasible objectives in the near term. These alternative driving modes will benefit from new research toward novel steering control devices, more suitably where machine intelligence only partially controls the vehicle. In this article, it is proposed that when the human shares the control of a vehicle with an autonomous or semiautonomous system, a force control, or nondisplacement steering wheel (i.e., a steering wheel which does not rotate but detects the applied torque by the human driver) can be advantageous under certain schemes: tight rein or loose rein modes according to the H -metaphor. We support this proposition with the first experiments to the best of our knowledge, in which human participants drove in a simulated road scene with a force control steering wheel (FCSW). The experiments exhibited that humans can adapt promptly to force control steering and are able to control the vehicle smoothly. Different transfer functions are tested, which translate the applied torque at the FCSW to the steering angle at the wheels of the vehicle; it is shown that fractional order transfer functions increment steering stability and control accuracy when using a force control device. The transition of control experiments is also performed with both: a conventional and an FCSW. This prototypical steering system can be realized via steer-by-wire controls, which are already incorporated in commercially available vehicles.

Visual information is increasingly recognized as a useful method to detect rail surface defects due to its high efficiency and stability. However, it cannot sufficiently detect a complete defect in the complex background information. The addition of surface profiles can effectively improve this by including a 3-D information of defects. However, in high-speed detection, the traditional 3-D profile acquisition is difficult and separate from the image acquisition, which cannot satisfy the above-mentioned requirements effectively. Therefore, an unsupervised stereoscopic saliency detection method based on a binocular line-scanning system is proposed in this article. This method can simultaneously obtain a highly precise image as well as profile information while also avoids the decoding distortion of the structured light reconstruction method. In our method, a global low-rank nonnegative reconstruction algorithm with a background constraint is proposed. Unlike the low-rank recovery model, the algorithm has a more comprehensive low rank and background clustering properties. Furthermore, outlier detection based on the geometric properties of the rail surface is also proposed in this method. Finally, the image saliency results and depth outlier detection results are associated with the collaborative fusion, and a dataset (RSDDS-113) containing the rail surface defects is established for the experimental verification. The experimental results demonstrate that our method can obtain a mean absolute error of 0.09 and area under the ROC curve of 0.94, better than 15 state-of-the-art algorithms.

The eight papers in this special section focus on advanced visual perception technologies.

Visual perception refers to the process of organizing, identifying, and interpreting visual information in environmental awareness and understanding. With the rapid progress of multimedia acquisition technology, research on visual perception has been a hot topic in the academical field and industrial applications. Especially after the introduction of artificial intelligence theory, intelligent visual perception has been widely used to promote the development of industrial production towards intelligence. In this article, we review the previous research and application of visual perception in different industrial fields such as product surface defect detection, intelligent agricultural production, intelligent driving, image synthesis, and event reconstruction. The applications basically cover most of the intelligent visual perception processing technologies. Through this survey, it will provide a comprehensive reference for research on this direction. Finally, this article also summarizes the current challenges of visual perception and predicts its future development trends.

In 3-D attitude angle estimation, monocular vision-based methods are often utilized for the advantages of short-time and high efficiency. However, the limitations of these methods lie in the complexity of the algorithm and the specificity of the scene, which needs to match the characteristics of the cooperation object and the scene. In this article, we propose a fully connected attitude detection network (FADN), which combines neural network and traditional algorithms for 3-D attitude angle estimation. FADN provides a whole process from the input of a single frame image in the industrial video stream to the output of the corresponding 3-D attitude angle estimation. Benefiting from the end-to-end estimation framework, FADN avoids tedious matching algorithms and thus has certain portability. A series of comparative experiments based on the rendering software 3-D Studio Max (3d Max) have been carried out to evaluate the performance of FADN. The experimental results show that FADN has high estimation accuracy and fast running speed. At the same time, the simulation results reliably prove the feasibility of FADN, and also promote the research in real scenarios.

Panoramic video and stereoscopic panoramic video are essential carriers of virtual reality content, so it is very crucial to establish their quality assessment models for the standardization of virtual reality industry. However, it is very challenging to evaluate the quality of the panoramic video at present. One reason is that the spatial information of the panoramic video is warped due to the projection process, and the conventional video quality assessment (VQA) method is difficult to deal with this problem. Another reason is that the traditional VQA method is problematic to capture the complex global time information in the panoramic video. In response to the above questions, this paper presents an end-to-end neural network model to evaluate the quality of panoramic video and stereoscopic panoramic video. Compared to other panoramic video quality assessment methods, our proposed method combines spherical convolutional neural networks (CNN) and non-local neural networks, which can effectively extract complex spatiotemporal information of the panoramic video. We evaluate the method in two databases, VRQ-TJU and VR-VQA48. Experiments show the effectiveness of different modules in our method, and our method outperforms state-of-the-art other related methods.

Herein, a combination of Local Pattern Generators (LPG) with reinforcement learning is proposed to balance a bipedal robot using minimal power consumption. This work presents the extension of Normalised Advantage Function (eNAF) algorithm to work with recurrent neural networks without sacrificing time-dependency between data in the same episode. Additionally, a hybrid controller is introduced by combining eNAF algorithm hierarchically with LPGs to provide more robustness with less computational power requirements. The system was asynchronous, as pattern generator had an activation frequency of 100Hz, while eNAF algorithm had only 1Hz and were not synchronised between them. Robot autonomy time was increased through reduction of computational load by introducing variable-ratio activation frequency between the LPGs and the eNAF algorithm. Finally, a new and novel bipedal robot design with non-conventional linear actuators was used as the basis of the simulator model. These experiments were implemented using V-Rep Edu simulator with the industrial Vortex Studio dynamic engine. The results demonstrate a fast and agile recovery by the trained robot after a push in transverse plane.

Verification and validation of deep learning algorithms is an important and challenging topic of artificial intelligence. Without approving by reliable and rigorous verification methods, deep learning algorithms, for instance, the convolutional neural networks, are not qualified to be used in real-world applications, especially in safety-critical areas. The gap between deep learning systems and the requirements in safety-critical application areas, such as autonomous robotics and self-driving vehicles, is the lack of Black-box V&V techniques that can test and evaluate the performance and the robustness of deep learning systems. To bridge this gap, we proposed a GAN based Black-box verification framework called AdversarialStyle for generating and searching adversarial examples in both targeted and non-targeted way from different styles or domains of interest. The AdversarialStyle can not only evaluate deep learning models but also can discover the robustness level of every instance in the test set. Therefore, this framework can support deep learning model designers to understand and to explore their algorithms and improve the trustworthiness of AI techniques.

We propose an efficient and robust segmentation network for automated seagrass region detection. The proposed network has a simple architecture to save computational demands as well as inference energy cost. More importantly, the scale of network can be feasibly adjusted, to balance the network computational demands and segmentation accuracy. Experimental results show that our proposed network is robust to segment the various seagrass patterns with 90.66% mIoU (mean Intersection over Union) accuracy. It had achieved 200 frames per second (FPS, 1.42 times faster than the second-best network GCN) on desktop GPU, and 18 FPS on NVIDIA Jetson TX2. It also has 3.45M parameters and 0.587 GMACs FLOPs (FLoating Point OPerations), only 14.6% and 10.8% of those in GCN respectively. To segment a single image on the Jetson TX2, our architecture requires an average energy of 0.26 Joule. This energy cost is only 46% of DeepLab, which shows the proposed network to be an energy efficient one. The proposed network demonstrates accurate and real-time segmentation capability, and it can be deployed to low-energy embedded AUVs for sea habitat protection.

With the wide application of stereoscopic video technology, the quality of stereoscopic video has attracted people's attention. Objective stereoscopic video quality assessment (SVQA) is highly challenging, but essential, particularly the no-reference (NR) SVQA method, where reference information is not needed and a large number of samples are required for training and testing sets. However, as far as we know, there are only a few samples in the established stereo video database, which is unsuitable for NR quality assessment and seriously hampers the development of NR-SVQA method. For these difficulties that we encountered, we carry out a comprehensive subjective evaluation of stereoscopic video quality in our newly established TJU-SVQA databases that contain various contents, mixed resolution coding and symmetrically/asymmetrically distorted stereoscopic videos. Furthermore, we propose a new inter-frame cross map to predict the objective quality scores. We compare and analyze the performance of several state-of-the-art 2D and 3D quality evaluation methods on our new databases. The experimental results on our established databases and a public database demonstrate that the proposed method can robustly predict the quality of stereoscopic videos.

An image is compressed or stretched during the multidevice displaying, which will have a very big impact on perception quality. In order to solve this problem, a variety of image retargeting methods have been proposed for the retargeting process. However, how to evaluate the results of different image retargeting is a very critical issue. In various application systems, the subjective evaluation method cannot be applied on a large scale. So we put this problem in the accurate objective-quality evaluation. Currently, most of the image retargeting quality assessment algorithms use simple regression methods as the last step to obtain the evaluation result, which are not corresponding with the perception simulation in the human vision system (HVS). In this paper, a deep quality evaluator for image retargeting based on the segmented stacked AutoEnCoder (SAE) is proposed. Through the help of regularization, the designed deep learning framework can solve the overfitting problem. The main contributions in this framework are to simulate the perception of retargeted images in HVS. Especially, it trains two separated SAE models based on geometrical shape and content matching. Then, the weighting schemes can be used to combine the obtained scores from two models. Experimental results in three well-known databases show that our method can achieve better performance than traditional methods in evaluating different image retargeting results.

Images taken under different camera poses are rotated or distorted, which leads to poor perception experiences. This article proposes a new framework to automatically transform the images to the conformable view setting by assembling different convolution neural networks. Specifically, a referential three-dimensional ground plane is first derived from the color image and a novel projection mapping algorithm is developed to achieve automatic viewing transformation. Extensive experimental results demonstrate that the proposed method outperforms the state-of-the-art vanishing points based methods by a large margin in terms of accuracy and robustness.

Surface defect detection is a critical task in industrial production process. Nowadays, there are lots of detection methods based on computer vision and have been successfully applied in industry, they also achieved good results. However, achieving full automation of surface defect detection remains a challenge, due to the complexity of surface defect, in intraclass. While the defects between interclass contain similar parts, there are large differences in appearance of the defects. To address these issues, this article proposes a pyramid feature fusion and global context attention network for pixel-wise detection of surface defect, called PGA-Net. In the framework, the multiscale features are extracted at first from backbone network. Then the pyramid feature fusion module is used to fuse these features into five resolutions through some efficient dense skip connections. Finally, the global context attention module is applied to the fusion feature maps of adjacent resolution, which allows effective information propagate from low-resolution fusion feature maps to high-resolution fusion ones. In addition, the boundary refinement block is added to the framework to refine the boundary of defect and improve the result of the prediction. The final prediction is the fusion of the five resolutions fusion feature maps. The results of evaluation on four real-world defect datasets demonstrate that the proposed method outperforms the state-of-the-art methods on mean intersection of union and mean pixel accuracy (NEU-Seg: 82.15%, DAGM 2007: 74.78%, MT_defect: 71.31%, Road_defect: 79.54%).

Accurate measurement of position and attitude information is particularly important. Traditional measurement methods generally require high-precision measurement equipment for analysis, leading to high costs and limited applicability. Vision-based measurement schemes need to solve complex visual relationships. With the extensive development of neural networks in related fields, it has become possible to apply them to the object position and attitude. In this paper, we propose an object pose measurement scheme based on convolutional neural network and we have successfully implemented end-to-end position and attitude detection. Furthermore, to effectively expand the measurement range and reduce the number of training samples, we demonstrated the independence of objects in each dimension and proposed subadded training programs. At the same time, we generated generating image encoder to guarantee the detection performance of the training model in practical applications.

The aim of this work is to produce and test a robustness module (ROB-M) that can be generally applied to distributed, multi-agent task allocation algorithms, as robust versions of these are scarce and not well-documented in the literature. ROB-M is developed using the Performance Impact (PI) algorithm, as this has previously shown good results in deterministic trials. Different candidate versions of the module are thus bolted on to the PI algorithm and tested using two different task allocation problems under simulated uncertain conditions, and results are compared with baseline PI. It is shown that the baseline does not handle uncertainty well; the task-allocation success rate tends to decrease linearly as degree of uncertainty increases. However, when PI is run with one of the candidate robustness modules, the failure rate becomes very low for both problems, even under high simulated uncertainty, and so its architecture is adopted for ROB-M and also applied to MIT’s baseline Consensus Based Bundle Algorithm (CBBA) to demonstrate its flexibility. Strong evidence is provided to show that ROB-M can work effectively with CBBA to improve performance under simulated uncertain conditions, as long as the deterministic versions of the problems can be solved with baseline CBBA. Furthermore, the use of ROB-M does not appear to increase mean task completion time in either algorithm, and only 100 Monte Carlo samples are required compared to 10,000 in MIT’s robust version of the CBBA algorithm. PI with ROB-M is also tested directly against MIT’s robust algorithm and demonstrates clear superiority in terms of mean numbers of solved tasks.

In this paper, a modified centralized algorithm based on particle swarm optimization (MCPSO) is presented to solve the task allocation problem in the search and rescue domain. The reason for this paper is to provide a benchmark against distributed algorithms in search and rescue application area. The hypothesis of this paper is that a centralized algorithm should perform better than distributed algorithms because it has all the available information at hand to solve the problem. Therefore, the centralized approach will provide a benchmark for evaluating how well the distributed algorithms are working and how much improvement can still be gained. Among the distributed algorithms, the consensus-based bundle algorithm (CBBA) is a relatively recent method based on the market auction mechanism, which is receiving considerable attention. Other distributed algorithms, such as PI and PI with softmax, have shown to perform better than CBBA. Therefore, in this paper, the three distributed algorithms mentioned earlier are compared against three centralized algorithms. They are particle swarm optimization, MCPSO, described in this paper, and genetic algorithms. Two experiments were conducted. The first involved comparing all the above-mentioned algorithms, both centralized and distributed, using the same set of application scenarios. It is found that MCPSO always outperforms the other five algorithms in time cost. Due to the high failure rate of CBBA and the other two centralized methods, the second experiment focused on carrying out more tests to compare MCPSO against PI and PI with softmax. All the results are shown and analyzed to determine the performance gaps between the distributed algorithms and the MCPSO. Note to Practitioners-This paper was motivated by the limitation of current distributed task allocation algorithms as they cannot achieve performances that are as good as the centralized ones. Therefore, a centralized algorithm is designed to evaluate the performance gap between the state-of-the-art distributed and centralized approaches. In the future research section, a new distributed particle swarm optimization (PSO) algorithm is proposed based on this paper as the research has shown that the proposed centralized PSO algorithm delivers the best results so it is potentially a strong candidate for adaptation.

Most of the current blind stereoscopic image quality assessment (SIQA) algorithms cannot show reliable accuracy. One reason is that they do not have the deep architectures and the other reason is that they are designed on the relatively weak biological basis, compared with findings on human visual system (HVS). In this paper, we propose a Deep Edge and COlor Signal INtegrity Evaluator (DECOSINE) based on the whole visual perception route from eyes to the frontal lobe, and especially focus on edge and color signal processing in retinal ganglion cells (RGC) and lateral geniculate nucleus (LGN). Furthermore, to model the complex and deep structure of the visual cortex, Segmented Stacked Auto-encoder (S-SAE) is used, which has not utilized for SIQA before. The utilization of the S-SAE complements weakness of deep learning-based SIQA metrics that require a very long training time. Experiments are conducted on popular SIQA databases, and the superiority of DECOSINE in terms of prediction accuracy and monotonicity is proved. The experimental results show that our model about the whole visual perception route and utilization of S-SAE are effective for SIQA.

Being able to join roundabouts correctly is crucial for an autonomous vehicle to maintain not only its own safety but also a normal traffic order for others. In order to know the right time and speed for entering roundabouts, the location, speed and direction of the approaching vehicles need to be taken into consideration. This study investigated the feasibility of leveraging computer vision and machine learning to help autonomous vehicles decide to wait or to enter when reaching roundabouts. A grid-based image processing approach with a single camera at normal roundabouts (GBIPA-SC-NR) is proposed in this paper to characterize traffic situations that can be used for machine learning algorithms to learn the roundabout joining criteria. Video road clips recorded when human drivers reach and then join various roundabouts at different locations were utilised for this learning process, with a selection of four supervised classification algorithms (i.e. the Support Vector Machines, Random Forests, K-Nearest Neighbours, and Decision Tree). The trained classifiers using the proposed approach were evaluated on 507 test videos captured at roundabouts, where the SVM showed the best performance with a 90.28% classification accuracy. This result suggests that the proposed grid-based image processing method can be applied to effectively help autonomous vehicles made the right decision when reaching a roundabout.

This paper addresses two main problems with many heuristic task allocation approaches - solution trapping in local minima and static structure. The existing distributed task allocation algorithm known as performance impact (PI) is used as the vehicle for developing solutions to these problems as it has been shown to outperform the state-of-the-art consensus-based bundle algorithm for time-critical problems with tight deadlines, but is both static and suboptimal with a tendency toward trapping in local minima. This paper describes two additional modules that are easily integrated with PI. The first extends the algorithm to permit dynamic online rescheduling in real time, and the second boosts performance by introducing an additional soft-max action-selection procedure that increases the algorithm's exploratory properties. This paper demonstrates the effectiveness of the dynamic rescheduling module and shows that the average time taken to perform tasks can be reduced by up to 9% when the soft-max module is used. In addition, the solution of some problems that baseline PI cannot handle is enabled by the second module. These developments represent a significant advance in the state of the art for multiagent, time-critical task assignment.

This paper considers the problem of maximizing the number of task allocations in a distributed multirobot system under strict time constraints, where other optimization objectives need also be considered. It builds upon existing distributed task allocation algorithms, extending them with a novel method for maximizing the number of task assignments. The fundamental idea is that a task assignment to a robot has a high cost if its reassignment to another robot creates a feasible time slot for unallocated tasks. Multiple reassignments among networked robots may be required to create a feasible time slot and an upper limit to this number of reassignments can be adjusted according to performance requirements. A simulated rescue scenario with task deadlines and fuel limits is used to demonstrate the performance of the proposed method compared with existing methods, the consensus-based bundle algorithm and the performance impact (PI) algorithm. Starting from existing (PI-generated) solutions, results show up to a 20% increase in task allocations using the proposed method.

Robust real-time obstacle detection/avoidance is a challenging problem especially for micro and small aerial vehicles due to the limited number of the on-board sensors due to the battery constraint and low payload. Usually lightweight sensors such as CMOS camera are the best choice comparing with laser or radar sensors. For real-time applications, most studies focus on using stereo cameras to reconstruct a 3D model of the obstacles or to estimate their depth. Instead, in this paper, a method that mimics the human behavior of detecting the state of the approaching obstacles using single camera is proposed. During the flight, this method is able to detect the changes of the size area of the obstacles. First, the method detects the feature points of the obstacles, and then extracts the obstacles that has probability of getting close. In addition, by comparing the changes in the area ratios of the obstacle in the image sequence, the method can decide if it is obstacle or not. Finally, by estimating the obstacle 2D position in the image and combining with the tracked waypoints, the UAV can take the action of avoidance.

Using distributed task allocation methods for cooperating multivehicle systems is becoming increasingly attractive. However, most effort is placed on various specific experimental work and little has been done to systematically analyze the problem of interest and the existing methods. In this paper, a general scenario description and a system configuration are first presented according to search and rescue scenario. The objective of the problem is then analyzed together with its mathematical formulation extracted from the scenario. Considering the requirement of distributed computing, this paper then proposes a novel heuristic distributed task allocation method for multivehicle multitask assignment problems. The proposed method is simple and effective. It directly aims at optimizing the mathematical objective defined for the problem. A new concept of significance is defined for every task and is measured by the contribution to the local cost generated by a vehicle, which underlies the key idea of the algorithm. The whole algorithm iterates between a task inclusion phase, and a consensus and task removal phase, running concurrently on all the vehicles where local communication exists between them. The former phase is used to include tasks into a vehicle's task list for optimizing the overall objective, while the latter is to reach consensus on the significance value of tasks for each vehicle and to remove the tasks that have been assigned to other vehicles. Numerical simulations demonstrate that the proposed method is able to provide a conflict-free solution and can achieve outstanding performance in comparison with the consensus-based bundle algorithm.

Rescue missions require both speed to meet strict time constraints and maximum use of resources. This study presents a Task Swap Allocation (TSA) algorithm that increases vehicle allocation with respect to the state-of-the-art consensus-based bundle algorithm and one of its extensions, while meeting time constraints. The novel idea is to enable an online reconfiguration of task allocation among distributed and networked vehicles. The proposed strategy reallocates tasks among vehicles to create feasible spaces for unallocated tasks, thereby optimizing the total number of allocated tasks. The algorithm is shown to be efficient with respect to previous methods because changes are made to a task list only once a suitable space in a schedule has been identified. Furthermore, the proposed TSA can be employed as an extension for other distributed task allocation algorithms with similar constraints to improve performance by escaping local optima and by reacting to dynamic environments.

This paper describes enhancements made to the distributed performance impact (PI) algorithm and presents the results of trials that show how the work advances the state-of-the-art in single-task, single-robot, time-extended, multiagent task assignment for time-critical missions. The improvement boosts performance by integrating the architecture with additional action selection methods that increase the exploratory properties of the algorithm (either soft max or e-greedy task selection). It is demonstrated empirically that the average time taken to perform rescue tasks can reduce by up to 8% and solution of some problems that baseline PI cannot handle is enabled. Comparison with the consensus-based bundle algorithm (CBBA) also shows that both the baseline PI algorithm and the enhanced versions are superior. All test problems center around a team of heterogeneous, autonomous vehicles conducting rescue missions in a 3-dimensional environment, where a number of different tasks must be carried out in order to rescue a known number of victims that is always more than the number of available vehicles.

This paper investigates the problem of global exponential lag synchronization of a class of switched neural networks with time-varying delays via neural activation function and applications in image encryption. The controller is dependent on the output of the system in the case of packed circuits, since it is hard to measure the inner state of the circuits. Thus, it is critical to design the controller based on the neuron activation function. Comparing the results, in this paper, with the existing ones shows that we improve and generalize the results derived in the previous literature. Several examples are also given to illustrate the effectiveness and potential applications in image encryption.

The paper presents an approach to applying a classifier ensemble to identify human body gestures, so as to control a robot to write Chinese characters. Robotic handwriting ability requires complicated robotic control algorithms. In particular, the Chinese handwriting needs to consider the relative positions of a character's strokes. This approach derives the font information from human gestures by using a motion sensing input device. Five elementary strokes are used to form Chinese characters, and each elementary stroke is assigned to a type of human gestures. Then, a classifier ensemble is applied to identify each gesture so as to recognize the characters that gestured by the human demonstrator. The classier ensemble's size is reduced by feature selection techniques and harmony search algorithm, thereby achieving higher accuracy and smaller ensemble size. The inverse kinematics algorithm converts each stroke's trajectory to the robot's motor values that are executed by a robotic arm to draw the entire character. Experimental analysis shows that the proposed approach can allow a human to naturally and conveniently control the robot in order to write many Chinese characters.

The ability of pointing is recognised as an essential skill of a robot in its communication and social interaction. This paper introduces a developmental learning approach to robotic pointing, by exploiting the interactions between a human and a robot. The approach is inspired through observing the process of human infant development. It works by first applying a reinforcement learning algorithm to guide the robot to create attempt movements towards a salient object that is out of the robot’s initial reachable space. Through such movements, a human demonstrator is able to understand the robot desires to touch the target and consequently, to assist the robot to eventually reach the object successfully. The human–robot interaction helps establish the understanding of pointing gestures in the perception of both the human and the robot. From this, the robot can collect the successful pointing gestures in an effort to learn how to interact with humans. Developmental constraints are utilised to drive the entire learning procedure. The work is supported by experimental evaluation, demonstrating that the proposed approach can lead the robot to gradually gain the desirable pointing ability. It also allows that the resulting robot system exhibits similar developmental progress and features as with human infants.

In recent years, unmanned aerial vehicles (UAVs) have gained a great importance in both military and civilian applications. In this paper, we proposed a vehicle detection method from UAVs which integrated of Scalar Invariant Feature Transform (SIFT) and Implicit Shape Model (ISM). Firstly, a set of key points was detected in the testing image by using SIFT. Secondly, feature descriptors around the key points were generated by using the ISM. Support Vector Machines (SVMs) were applied during the key points selection. The experiment used a video shoot by a UAV in a highway and the results showed the performance and the effectiveness of the method.

Motion segmentation plays an important role in human motion analysis. Understanding the intrinsic features of human activities represents a challenge for modern science. Current solutions usually involve computationally demanding processing and achieve the best results using expensive, intrusive motion capture devices. In this paper, a simple, affordable and effective method for human motion segmentation and alignment is presented. The approach follows a two-step process: first, the most salient principal components are calculated in order to reduce the dimension of the input motion data. Then, candidates of key body poses (landmarks) are inferred using multi-class, supervised machine learning techniques from a set of training samples. Finally, cluster analysis is used to refine the result. Predictions are guaranteed to be invariant to the repetitiveness and symmetry of the performance. Results show the effectiveness of the proposed approach by comparing it against the Dynamic Time Warping algorithm and Hierarchical Aligned Cluster Analysis.

This paper addresses the problem of multi-agent, multi-task assignment with multiple agent requirements on tasks for unmanned aerial vehicles by presenting the Consensus Based Grouping Algorithm. The algorithm is an extension of the Consensus Based Bundle Algorithm that converges to a conflict free, feasible solution of which previous algorithms are unable to account for. Furthermore the algorithm creates a framework to take into account task based requirements, deadlocking and a method to store assignments for a dynamical environment.

In this paper, immune systems and its relationships with multi-robot shepherding problems are discussed. The proposed algorithm is based on immune network theories that have many similarities with the multi-robot systems domain. The underlying immune-inspired cooperative mechanism of the algorithm is simulated and evaluated. The paper also describes a refinement of the memory-based immune network that enhances a robot's action-selection process. A refined model, which is based on the Immune Network T-cell-regulated — with Memory (INT-M) model, is applied to the dog–sheep scenario. The refinements involves the low-level behaviors of the robot dogs, namely shepherds' formation and shepherds' approach. These behaviors would make the shepherds form a line behind the group of sheep and also obey a safety zone of each flock, thus achieving better control of the flock and minimize flock separation occurrences. Simulation experiments are conducted on the Player/Stage robotics platform.

Advanced autonomous artificial systems will need incremental learning and adaptive abilities similar to those seen in humans. Knowledge from biology, psychology and neuroscience is now inspiring new approaches for systems that have sensory-motor capabilities and operate in complex environments. Eye/hand coordination is an important cross-modal cognitive function, and is also typical of many of the other coordinations that must be involved in the control and operation of embodied intelligent systems. This paper examines a biologically inspired approach for incrementally constructing compact mapping networks for eye/hand coordination. We present a simplified node-decoupled extended Kalman filter for radial basis function networks, and compare this with other learning algorithms. An experimental system consisting of a robot arm and a pan-and-tilt head with a colour camera is used to produce results and test the algorithms in this paper. We also present three approaches for adapting to structural changes during eye/hand coordination tasks, and the robustness of the algorithms under noise are investigated. The learning and adaptation approaches in this paper have similarities with current ideas about neural growth in the brains of humans and animals during tool-use, and infants during early cognitive development.

Range images of complex geometry presented by large point data sets almost always yield surface reconstruction imperfections. We propose a novel compact and complete mesh representation for non-uniformly sampled noisy range image data using an adaptive radial basis function network. The network is established using a heuristic learning strategy. Neurons can be inserted, removed or updated iteratively, adapting to the complexity and distribution of the underlying data. This flexibility is particularly suited to highly variable spatial frequencies, and is conducive to data compression with network representations. Experiments confirm the performance advantages of the network when applied to 3D point-cloud surface reconstruction.

In this paper, we describe a new error-driven active learning approach to self-growing radial basis function networks for early robot learning. There are several mappings that need to be set up for an autonomous robot system for sensorimotor coordination and transformation of sensory information from one modality to another, and these mappings are usually highly nonlinear. Traditional passive learning approaches usually cause both large mapping errors and nonuniform mapping error distribution compared to active learning. A hierarchical clustering technique is introduced to group large mapping errors and these error clusters drive the system to actively explore details of these clusters. Higher level local growing radial basis function subnetworks are used to approximate the residual errors from previous mapping levels. Plastic radial basis function networks construct the substrate of the learning system and a simplified node-decoupled extended Kalman filter algorithm is presented to train these radial basis function networks. Experimental results are given to compare the performance between active learning and passive learning

In order to control the continuous passive motion (CPM) machine for injured fingers, we develop an embedded control platform. We first bring forward the philosophy of function modularization design for control platforms. Then we actually begin to develop an embedded control platform of the CPM machine by using the method of function modularization. The core of the control platform consists of two main parts: the data acquisition function module and the motor control function module, both are based on the serial peripheral interface (SPI) network. The whole control platform is open-ended for new functions and applications. It can be easily expanded if we add new modules to the SPI network. Primary experiments have proved that the control platform works well and the design method of function modularization provides a new method for the design of control platforms

Advanced autonomous artificial systems will need incremental learning and adaptive abilities similar to those seen in humans. Knowledge from biology, psychology and neuro-science is now inspiring new approaches for systems that have sensory-motor capabilities and operate in complex environments. Eye/hand coordination is an important cross-modal cognitive function, and is also typical of many of the other coordinations that must be involved in the control and operation of embodied intelligent systems. This paper examines a biologically inspired approach for incrementally constructing compact mapping networks for eye/hand coordination. We present a simplified node-decoupled extended Kalman filter for radial basis function networks, and compare this with other learning algorithms. An experimental system consisting of a robot arm and a pan-and-tilt head with a color camera is used to produce results and test the algorithms in this paper. We also present three approaches for adapting to structural changes during eye/hand coordination tasks, and the robustness of the algorithms under noise are investigated. The learning and adaptation approaches in this paper have similarities with current ideas about neural growth in the brains of infants during early cognitive development

We describe an approach to sensory-motor learning and coordination that draws from psychology rather than neuroscience. The growth of the motor coordination is controlled through sequential lifting of constraints, which is inspired by Jean Piaget's developmental learning theory. Our objective is the implementation of a flexible learning framework for an embodied hand/eye system which exhibits a prolonged epigenetic developmental process. The results show how staged competence can be shaped by qualitative behavior changes produced by anatomical, computational and maturational constraints.

We propose a general algorithm for identifying an arbitrary pose of an articulated subject with sparse point features. The algorithm aims to identify a one-to-one correspondence between a model point-set and an observed point-set taken from freeform motion of the articulated subject. We avoid common assumptions such as pose similarity or small motions with respect to the model, and assume no prior knowledge from which to infer an initial or partial correspondence between the two point-sets. The algorithm integrates local segment-based correspondences under a set of affine transformations, and a global hierarchical search strategy. Experimental results, based on synthetic pose and real-world human motion data demonstrate the ability of the algorithm to perform the identification task. Reliability is increasingly compromised with increasing data noise and segmental distortion, but the algorithm can tolerate moderate levels. This work contributes to establishing a crucial self-initializing identification in model-based point-feature tracking for articulated motion.

Feature-based methods in vision analysis often encounter the problem of correspondences between features of two related patterns. The features may be points, lines, curves and surfaces/regions. Point pattern matching (PPM) is a primary and essential approach for establishing a correspondence within two related patterns. Applications exist in wide circumstances. Numerous techniques related to PPM have been studied within a rich and extensive literature, encompassing both theoretical and practical problem domains. In this paper, we provide a review of the PPM techniques and their applications from three aspects: 1) PPM under rigid/affine motion, 2) PPM under non-rigid/elastic motion, and 3) PPM in a dynamic sequence. We also anticipate the trend for the future in adapting existing techniques to novel algorithms for non-rigid PPM.

In this paper, we describe the incorporation of learning and reuse of experience into behavior-based service robot systems. Experience is context based, and the learned experience is associated with its relevant behaviors. Object constraints are obtained by virtual object movements in image space. In addition to retaining the key properties of behavior-based systems, our approach also has the planning ability to recover from errors and to imitate an object pattern shown by humans. Experiments on a real physical robot system have successfully tested the approach.

Tracking and identifying articulated objects have received growing attention in computer vision in the past decade. In market-based optical motion capture (MoCap) systems, an articulated movement of near-rigid segments is represented via a sequence of moving dots of known 3D coordinates, corresponding to the captured marker positions. We propose a segment-based articulated model-fitting algorithm to address the problem of self-initializing identification and pose estimation utilizing one frame of data in such point-feature tracking systems. It is ultimately crucial for recovering the complete motion sequence. Experimental results, based on synthetic pose and real-world human motion capture data, demonstrate the performance of the algorithm.