Multi-view spectral clustering (MVSC) has garnered growing interest across various real-world applications, owing to its flexibility in managing diverse data space structures. Nevertheless, the fusion of multiple $n\times n$n×n similarity matrices and the separate post-discretization process hinder the utilization of MVSC in large-scale tasks, where $n$n denotes the number of samples. Moreover, noise in different similarity matrices, along with the two-stage mismatch caused by the post-discretization, results in a reduction in clustering effectiveness. To overcome these challenges, we establish a novel fast multi-view discrete clustering (FMVDC) model via spectral embedding fusion, which integrates spectral embedding matrices ($n\times c$n×c, $c\ll n$c≪n) to directly obtain discrete sample categories, where $c$c indicates the number of clusters, bypassing the need for both similarity matrix fusion and post-discretization. To further enhance clustering efficiency, we employ an anchor-based spectral embedding strategy to decrease the computational complexity of spectral analysis from cubic to linear. Since gradient descent methods are incapable of discrete models, we propose a fast optimization strategy based on the coordinate descent method to solve the FMVDC model efficiently. Extensive studies demonstrate that FMVDC significantly improves clustering performance compared to existing state-of-the-art methods, particularly in large-scale clustering tasks.

This paper presents a novel Nesterov accelerated gradient-based fixed-time convergent actor-critic (NAG-FxT-AC) scheme for the optimal control of nonlinear systems. The proposed approach integrates the Nesterov accelerated gradient method with FxT concurrent learning to achieve rapid convergence while ensuring optimal performance. Actor-critic neural networks (NN) are employed to approximate the optimal value function and control policy, where the accelerated gradient mechanism introduces auxiliary variables to enhance learning efficiency. A FxT concurrent learning algorithm is developed to update the NN weights, guaranteeing convergence to bounded regions within fixed time independent of initial conditions. Lyapunov stability analysis proves that both the closed-loop system states and NN estimation errors are ultimately uniformly bounded with FxT NN weights convergence properties. Simulation results on a nonlinear system validate the effectiveness of the proposed control scheme. Compared to the baseline methods like FxT-ADP, the proposed NAG-FxT-AC reduces the state convergence time by up to 12% and the weight convergence time by 84%, demonstrating superior learning efficiency.

This paper presents EveryBrain, a method to generate electroencephalographic (EEG) signals of visual stimuli using images. Given that individuals exhibit distinct EEG responses to the same visual stimulus, EveryBrain is capable of capturing these individual characteristics during signal generation. The framework operates in two stages. By leveraging the temporal properties of EEG signals and the spatial features of images, EveryBrain presents a self-supervised framework that simultaneously reconstruct EEG signals and perform contrastive learning between image and EEG features. Furthermore, through additional training focused on individual EEG differences, Stage2 injects an ID number (representing a specific person) into image features via a cross-modal projector. The resulting personalized EEG latent codes, supervised by the Stage1 encoder, are then decoded into vivid, individualized EEG responses. Experiments validate the accuracy of EveryBrain in generating EEG signals for various individuals in response to visual stimuli. Overall, the proposed method tackles challenges in EEG generation from images, such as cross-modal alignment, individual variability, and waveform stability, yielding promising results. Additionally, the novel approach of of joint learning between images and EEG demonstrates positive effects on decoding visual neural representations. Both quantitative and qualitative evaluations demonstrate the effectiveness of methods, marking a significant step toward portable and cost-effective "image-to-thought".

To efficiently assist humans in various tasks, it is crucial to accurately decode and understand the rich information embedded in brain's visual cognition. Existing brain-driven research often fails to overcome the challenge of small target data domains, and the lack of explicit semantic, spatial, and other information constraints on feature extractors prevents brain decoding models from learning uniform cross-domain representations, leading to degradation of their performance in unseen domains. To overcome these limitations, we propose DAMind, a multimodal EEG-based model for robust visual cross-domain alignment and decoding. Our approach integrates VLM with brain-inspired cognitive mechanisms, leveraging the strong image-text representation abilities to learn both fine-grained primary visual features and high-level semantic concepts from neural signals, provide effective visual fine-tuning using the visual guidance mechanism. DAMind introduces a stepwise EEG encoding process aligned with visual processing, and employs an instruction-based learning strategy for effective cross-domain zero-shot transfer. Its robust architecture efficiently achieves good generalization performance, enabling the mapping of EEG signals from multiple domains to a unified learning domain. We construct a comprehensive EEG decoding benchmark EBench, DAMind achieves state-of-the-art results on several visual tasks, and outperforms the baseline in zero-shot setting.

Information theory has inspired numerous advancements in multi-view learning. Most multi-view methods incorporating information-theoretic principles rely an assumption called multi-view redundancy which states that common information between views is necessary and sufficient for down-stream tasks. This assumption emphasizes the importance of common information for prediction, but inherently ignores the potential of unique information in each view that could be predictive to the task. In this paper, we propose a comprehensive information-theoretic multi-view learning framework named CIML, which discards the assumption of multi-view redundancy. Specifically, CIML considers the potential predictive capabilities of both common and unique information based on information theory. First, the common representation learning maximizes Gács-Körner common information to extract shared features and then compresses this information to learn task-relevant representations based on the Information Bottleneck (IB). For unique representation learning, IB is employed to achieve the most compressed unique representation for each view while simultaneously minimizing the mutual information between unique and common representations, as well as among different unique representations. Importantly, we theoretically prove that the learned joint representation is predictively sufficient for the downstream task. Extensive experimental results have demonstrated the superiority of our model over several state-of-art methods. The code is released on CIML.

Recently, multi-view learning has witnessed a considerable interest on the research of trusted decision-making. Previous methods are mainly inspired from an important paper published by Han et al. in 2021, which formulates a Trusted Multi-view Classification (TMC) framework that aggregates evidence from different views based on Dempster's combination rule. All these methods only consider inter-view aggregation, yet lacking exploitation of intra-view information. In this paper, we propose a generalized trusted multi-view classification framework with hierarchical opinion aggregation. This hierarchical framework includes a two-phase aggregation process: the intra-view and inter-view aggregation hierarchies. In the intra aggregation, we assume that each view is comprised of common information shared with other views, as well as its specific information. We then aggregate both the common and specific information. This aggregation phase is useful to eliminate the feature noise inherent to view itself, thereby improving the view quality. In the inter-view aggregation, we design an attention mechanism at the evidence level to facilitate opinion aggregation from different views. To the best of our knowledge, this is one of the pioneering efforts to formulate a hierarchical aggregation framework in the trusted multi-view learning domain. Extensive experiments show that our model outperforms some state-of-art trust-related baselines. One can access the source code onhttps://github.com/lshi91/GTMC-HOA.

In the context of multi-view clustering, graph learning is recognized as a crucial technique, which generally involves constructing an adaptive neighbor graph based on probabilistic neighbors, and then learning a consensus graph for clustering. However, it is worth noting that these graph learning methods encounter two significant limitations. Firstly, they often rely on Euclidean distance to measure similarity when constructing the adaptive neighbor graph, which proves inadequate in capturing the intrinsic structure among data points in practice, particularly for high-dimensional data. Secondly, most of these methods focus solely on consensus graph, ignoring unique information from each view. Although a few graph-based studies have considered using specific information as well, the modelling approach employed does not exclude the noise impact from the common or specific components. To this end, we propose a novel tensor-based multi-view graph learning framework that simultaneously considers consistency and specificity, while effectively eliminating the influence of noise. Specifically, we calculate similarity using pseudo-Stiefel manifold distance to preserve the intrinsic properties of data. By making an assumption that the learned neighbor graph of each view comprises a consistent part, a specific part, and a noise part, we formulate a new tensor-based target graph learning paradigm for noise-free graph fusion. Owing to the benefits of tensor singular value decomposition (t-SVD) in uncovering high-order correlations, this model is capable of achieving a comprehensive understanding of the target graph. Furthermore, we derive an algorithm to address the optimization problem. Experiments on six datasets have demonstrated the superiority of our method. We have released the source code onhttps://github.com/lshi91/CSTGL-Code.

This work investigates the formation tracking issue for multirobot manipulator end-effectors under input constraints. A distributed formation control law is designed to guarantee the finite-time boundedness of tracking errors within the framework. To estimate the significant bias of dynamics discovered during practical multirobot collaborative manipulation tasks, a bias radial basis function neural network (RBFNN) is integrated, along with a designed adaptive updating law for expeditious approximation. In addition, an anti-windup compensator within a finite-time framework is specifically introduced to mitigate the input saturation issue arising from torque limitations in joint actuators. Finally, the system’s semi-global practical finite-time boundedness (SGPFTB) is rigorously established through Lyapunov theory. Five planar manipulators are employed in comparative computational experiments to validate the feasibility of the presented control strategy.

The self-attention mechanism of Transformer has gained considerable attention for its potential in modeling long-term temporal dependencies in electroencephalogram (EEG) signals. Despite recent advancements, Transformer-based decoding methods often neglect the explicit temporal priors inherent in EEG signals, i.e., the dependency between tokens tends to diminish as their relative temporal distance increases, which limits the efficacy of these decoding approaches. Inspired by the recent Retentive Network (RetNet), we develop a novel convolutional retentive network for EEG decoding (RetEEG), which integrates temporal priors into the self-attention mechanism to tackle the above challenge. Specifically, RetEEG incorporates a convolution module to capture local EEG features and cascades multiple bidirectional retention modules to learn global correlations within these local features. The bidirectional retention module introduces a temporal decay matrix, which imparts prior knowledge to the self-attention mechanism by accounting for variations in relative temporal distances. To validate the superiority of RetEEG, we conducted experiments on two publicly available EEG datasets. The experimental results empirically demonstrate that the proposed RetEEG achieves the state-of-the-art EEG decoding performance. Code is available at: https://github.com/kfhss/RetEEG.

Source-free domain generalization (SFDG) tackles the challenge of adapting models to unseen target domains without access to source domain data. To deal with this challenging task, recent advances in SFDG have primarily focused on leveraging the text modality of vision-language models such as CLIP. These methods involve developing a transferable linear classifier based on diverse style features extracted from the text and learned prompts or deriving domain-unified text representations from domain banks. However, both style features and domain banks have limitations in capturing comprehensive domain knowledge. In this work, we propose Prompt-Driven Text Adapter (PromptTA) method, which is designed to better capture the distribution of style features and employ resampling to ensure thorough coverage of domain knowledge. To further leverage this rich domain information, we introduce a text adapter that learns from these style features for efficient domain information storage. Extensive experiments conducted on four benchmark datasets demonstrate that PromptTA achieves state-of-the-art performance. The code is available at https://github.com/zhanghr2001/PromptTA.

The human brain utilizes spikes for information transmission and dynamically reorganizes its network structure to boost energy efficiency and cognitive capabilities throughout its lifespan. Drawing inspiration from this spike-based computation, Spiking Neural Networks (SNNs) have been developed to construct event-driven models that emulate this efficiency. Despite these advances, deep SNNs continue to suffer from over-parameterization during training and inference, a stark contrast to the brain’s ability to self-organize. Furthermore, existing sparse SNNs are challenged by maintaining optimal pruning levels due to a static pruning ratio, resulting in either under or over-pruning. In this paper, we propose a novel two-stage dynamic structure learning approach for deep SNNs, aimed at maintaining effective sparse training from scratch while optimizing compression efficiency. The first stage evaluates the compressibility of existing sparse subnetworks within SNNs using the PQ index, which facilitates an adaptive determination of the rewiring ratio for synaptic connections based on data compression insights. In the second stage, this rewiring ratio critically informs the dynamic synaptic connection rewiring process, including both pruning and regrowth. This approach significantly improves the exploration of sparse structures training in deep SNNs, adapting sparsity dynamically from the point view of compression efficiency. Our experiments demonstrate that this sparse training approach not only aligns with the performance of current deep SNNs models but also significantly improves the efficiency of compressing sparse SNNs. Crucially, it preserves the advantages of initiating training with sparse models and offers a promising solution for implementing Edge AI on neuromorphic hardware.

Behavior Cloning (BC) is a widely adopted visual imitation learning method in robot manipulation. Current BC approaches often enhance generalization by leveraging large datasets and incorporating additional visual and textual modalities to capture more diverse information. However, these methods overlook whether the learned representations contain redundant information and lack a solid theoretical foundation to guide the learning process. To address these limitations, we adopt an information-theoretic perspective and introduce mutual information to quantify and mitigate redundancy in latent representations. Building on this, we incorporate the Information Bottleneck (IB) principle into BC, which extends the idea of reducing redundancy by providing a structured framework for compressing irrelevant information while preserving task-relevant features. This work presents the first comprehensive study on redundancy in latent representations across various methods, backbones, and experimental settings, while extending the generalizability of the IB to BC. Extensive experiments and analyses on the CortexBench and LIBERO benchmarks show consistent performance improvements with IB across various settings, underscoring the importance of reducing input data redundancy and highlighting its practical value for real-world applications.

Brain disorders have been consistently associated with abnormalities in specific brain regions or neural circuits. Identifying key brain regional activities and functional connectivity patterns is essential for discovering more precise neurobiological biomarkers. However, previous studies have primarily emphasized alterations in functional connectivity while overlooking abnormal neuronal population activity within brain regions. To bridge this gap, we propose a novel Local-Global Coupling Spiking Graph Transformer (LGC-SGT) that jointly models both inter-regional connectivity differences and deviations in neuronal population firing rates within brain regions, enabling a dual-perspective neuropathological analysis. The global pathway leverages spike-based computation in LGC-SGT to model biologically plausible aberrant neural firing dynamics, while the local pathway adaptively captures abnormal graph-based representations of brain connectivity learned by local plasticity in the liquid state machine module. Furthermore, we design a shortcut-enhanced output strategy in LGC-SGT with the hybrid loss function to suppress outlier interference caused by inter-individual and inter-center variability, enabling a more robust decision boundary. Extensive experiments on three brain disorder datasets demonstrate that our model consistently outperforms state-of-the-art graph methods in brain disorder diagnosis. Moreover, it facilitates the extraction of interpretable neurobiological biomarkers by jointly analyzing regional neural activity and functional connectivity, offering a more comprehensive framework for brain disorder understanding and diagnosis.

The large-scale integration of distributed energy resources into active distribution networks may significantly intensify voltage fluctuations and increase network losses. Traditional model-based reactive power optimization approaches depend on existence of accurate system models. On the other hand, conventional reinforcement learning methods largely ignore the spatial characteristics of the active distribution networks during training, allowing agents to have an inadequate perception of the system state. To address these challenges, this paper proposes a multi-agent deep reinforcement learning approach that integrates graph learning with reinforcement learning for the learning of reactive power optimization strategies in active distribution networks. Specifically, the active distribution network is divided into multiple regions, with each region being controlled by an agent. The agents collaborate to achieve the global reactive power optimization goal. The perception capability of the agents is enhanced by adopting graph attention networks during the feature extraction phase. In the training phase, a primal-dual method is employed to manage constraints effectively. During the execution phase, each agent controls the photovoltaic inverters, electric springs, and capacitor banks based on the strategies developed in the training phase. The performance of the proposed approach is validated by a series of experiments on the IEEE-33 system, along with comparisons versus some existing data-driven deep reinforcement learning methods.

Sparse Bayesian learning has promoted many effective frameworks of brain activity decoding for the brain-computer interface, including the direct reconstruction of muscle activity using brain recordings. However, existing sparse Bayesian learning algorithms mainly use Gaussian distribution as error assumption in the reconstruction task, which is not necessarily the truth in the real-world application. On the other hand, brain recording is known to be highly noisy and contains many non-Gaussian noises, which could lead to large performance degradation for sparse Bayesian learning algorithms. The goal of this paper is to propose a novel robust implementation of sparse Bayesian learning so that robustness and sparseness can be realized simultaneously. Motivated by the exceptional robustness of maximum correntropy criterion (MCC), we proposed integrating MCC to the sparse Bayesian learning regime. To be specific, we derived the explicit error assumption inherent in the MCC, and then leveraged it for the likelihood function. Meanwhile, we utilized the automatic relevance determination technique as the sparse prior distribution. To fully evaluate the proposed method, a synthetic example and a real-world muscle activity reconstruction task with two different brain modalities were leveraged. Experimental results showed, our proposed sparse Bayesian correntropy learning framework significantly improves the robustness for the noisy regression tasks. Our proposed algorithm could realize higher correlation coefficients and lower root mean squared errors for the real-world muscle activity reconstruction scenario. Sparse Bayesian correntropy learning provides a powerful approach for brain activity decoding which will promote the development of brain-computer interface technology.

Event cameras form a fundamental foundation for visual perception in scenes characterized by high speed and a wide dynamic range. Although deep learning techniques have achieved remarkable success in estimating event-based optical flow, existing methods have not adequately addressed the significance of temporal information in capturing spatiotemporal features. Due to the dynamics of spiking neurons in SNNs, which preserve important information while forgetting redundant information over time, they are expected to outperform analog neural networks (ANNs) with the same architecture and size in sequential regression tasks. In addition, SNNs on neuromorphic hardware achieve advantages of extremely low power consumption. However, present SNN architectures encounter issues related to limited generalization and robustness during training, particularly in noisy scenes. To tackle these problems, this study introduces an innovative spike-based self-supervised learning algorithm known as SeLHIB, which leverages the information bottleneck theory. By utilizing event-based camera inputs, SeLHIB enables robust estimation of optical flow in the presence of noise. To the best of our knowledge, this is the first proposal of a self-supervised information bottleneck learning strategy based on SNNs. Furthermore, we develop spike-based self-supervised algorithms with nonlinear and high-order information bottleneck learning that employs nonlinear and high-order mutual information to enhance the extraction of relevant information and eliminate redundancy. We demonstrate that SeLHIB significantly enhances the generalization ability and robustness of optical flow estimation in various noise conditions. In terms of energy efficiency, SeLHIB achieves 90.44% and 45.70% cut down of energy consumption compared to its counterpart ANN and counterpart SNN models, while attaining 33.78% lower AEE (MVSEC), 5.96% lower RSAT (ECD) and 6.21% lower RSAT (HQF) compared to the counterpart ANN implementations with the same sizes and architectures.

The iterative Wiener filter (IWF) algorithm can achieve a fast convergence rate. However, its performance may degrade when it encounters noisy input scenarios. To tackle this problem, a novel IWF algorithm incorporating bias-compensation (BC-IWF) is proposed, which can enhance the performance of the algorithm by estimating the input noise variance. The BC-IWF algorithm optimizes the step size for each iteration and updates along the direction of the gradient. To further reduce the steady-state error, a normalized IWF by making use of the bias-compensation scheme (BC-NIWF) algorithm is proposed. Moreover, the steady-state performance of the BC-NIWF algorithm is analyzed. Simulation results demonstrate the validity of the theoretical analysis and the BC-NIWF algorithm achieves improved misadjustment compared with the state-of-the-art algorithms.

The study of the relationship between emotions and physiological signals of subjects under multimedia stimulation is an emerging field, and many important advances are made. However, there are still some challenges: 1) How to effectively utilize the complementarity among spatial-spectral-temporal domain information. 2) How to employ the heterogeneity and the correlation among multi-modal physiological signals simultaneously. 3) How to improve the robustness of the model dealing with missing channels. 4) How to model the dependency among different emotions. In this paper, we propose a novel two-stream Dynamic Heterogeneous Graph Recurrent Neural Network called DHGRNN. Specifically, DHGRNN consists of a spatial-temporal stream, a spatial-spectral stream, a fusion layer, and a multi-label classifier. Each stream is composed of a graph transformer network, evolved graph convolutional neural network, and gated recurrent units. We propose a graph-based two-stream structure to fuse the information of the spatial-spectral-temporal domain simultaneously. Graph transformer network and evolved graph convolutional neural network are used to model the heterogeneity and correlation of multi-modal physiological signals, respectively. To deal with the problem of robustness in the face of missing channel data, we transform it into the problem of dynamic graphs and use a dynamic graph neural network to improve the robustness. In addition, we propose a multi-label classifier to model the dependency among different emotion dimensions. Experiments on three public datasets demonstrate that our proposed model outperforms existing state-of-the-art methods.

Major depressive disorder (MDD) is a serious psychiatric disorder characterized by persistent feelings of sadness, hopelessness, and lack of interest or pleasure in daily activities. Yet, reliable diagnostic tools for this brain disorder remain lacking. Functional near-infrared spectroscopy (fNIRS), an optical brain imaging technique, offers a promising approach for monitoring cerebral hemodynamic activity associated with MDD. In this study, we propose a novel algorithm based on wavelet coherence and a state-pathology separation network (WCSN) to automatically detect MDD using a dual-channel fNIRS system. The fNIRS signals were first preprocessed and transformed into two-dimensional feature maps using a wavelet coherence method. Following this, a wrapped exhaustive search was applied to select the optimal subset of feature maps, which was then utilized to reconstruct the dataset. Finally, samples were classified using the state-pathology separation network that employed a dual-encoder convolutional autoencoder (DCoAE) module to separate the feature maps into state features and pathology features, while a Transformer module distinguished MDD patients from healthy controls based solely on pathology features. The WCSN algorithm achieved exceptional performance with an accuracy of 0.923 ± 0.068 and a subject accuracy of 0.918 ± 0.076. Our result highlights the WCSN algorithm's ability to isolate pure pathology features, enhancing classification robustness and generalizability under dual-channel data conditions. Taken together, the proposed WCSN algorithm is well-suited for home-based MDD screening applications.

This paper proposes a hierarchical safe reinforcement learning with prescribed performance control (HSRL-PPC) scheme to address the challenges of interconnected leader-follower systems operating in complex environments. The framework consists of two levels: at the higher level, the leader agent detects and avoids moving obstacles while planning optimal paths; at the lower level, the follower agent tracks the leader within strict prescribed performance bounds. We formulate the optimal prescribed performance safe control problem and solve it using the Hamilton-Jacobi-Bellman (HJB) equation. Due to system nonlinearity and obstacle complexity, we approximate the leader’s optimal value function using a state-following neural network that efficiently extrapolates training data to neighboring states, while employing a regular critic neural network for the follower’s value function approximation. Lyapunov stability analysis demonstrates the closed-loop system’s theoretical guarantees. Experimental results from two simulation examples and hardware tests with a quadcopter-vehicle system validate the effectiveness of the proposed approach in achieving safe navigation and precise tracking performance in dynamic environments. Note to Practitioners—Challenges exist in unpredictable obstacles and agent limitations for the interconnected leader-follower system. To provide a safe, efficient, and reliable control scheme, hierarchical safe reinforcement learning with prescribed performance control is proposed in this paper. The hierarchical structure is utilized to coordinate the leader and follower agents in the interconnected system, where the leader agent plans the optimal path and avoids obstacles, and the follower agent tracks the leader within prescribed performance bounds. Based on the proposed hierarchical structure, engineers can design efficient and safe control schemes for interconnected leader-follower systems with moving obstacles. In future work, we will address the problem of external disturbances and uncertainties in the interconnected leader-follower system.

Deep nonnegative matrix factorization (DNMF) technique has attached much great attention in recent year, since it can effectively discover the underlying hierarchical structure of complex data. However, most existing unsupervised and semi-supervised DNMF approaches not only suffer from the noisy data seriously, but also fail to enhance the decomposition quality of DNMF obviously by using the obtained supervisory information. To overcome these drawbacks, a robust semi-supervised DNMF method, called the correntropy based semi-supervised DNMF with constraint propagation (CSDCP), is proposed in this paper for learning a compact and meaningful data representation from the original data. Particularly, instead of adopting the traditional Frobenius norm, CSDCP employs the nonlinear and local similarity measure (e.g., correntropy) as the loss function in DNMF to enhance the robustness of DNMF for the noisy data. In addition, the hypergraph based constraint propagation (HCP) algorithm is adopted in CSDCP to exploit the limited supervisory information fully for capturing good data representation. Moreover, algorithm analysis of CSDCP is presented in this paper, including convergence analysis, robustness analysis supervised information analysis, and computational complexity. Extensive experimental results have illustrated that, in comparison to the most related DNMF approaches, CSDCP usually has better clustering results on six nonnegative datasets in clustering tasks.

Compared with the functional link neural network (FLNN) algorithm, the delayless multi-sampled multiband-structured subband FLNN (DMSFLNN) algorithm provides fast convergence when encountering highly auto-correlated input signals, but there is a compromise between convergence and steady-state performances. Therefore, in order to overcome this flaw, we develop an optimal DMSFLNN (ODMSFLNN) algorithm by minimizing the mean square deviation of the weight vector with respect to the subband gain vectors. Interestingly, a vectorized version is also proposed for the ODMSFLNN algorithm, which aims at reducing computational complexity. Additionally, this paper also presents a stability analysis of this algorithm. Then, considering the impulsive noise environment, we develop two robust variants of ODMSFLNN that are the R-ODMSFLNN-I and R-ODMSFLNN-II algorithms, which are based on the specified robust function and the energy constraint of the weight update increment, respectively. Finally, to resolve that the DMSFLNN algorithm may not exploit cross-terms of input samples in nonlinear active noise control scenarios, we further propose the subband second-order Volterra filter (SSOVF) framework in an analogy way and apply the R-ODMSFLNN-II learning principle to obtain the robust optimal SSOVF algorithm. Simulations in several nonlinear scenarios have shown that the proposed algorithms perform better than their competitors.

Reconstructing visual stimuli from functional Magnetic Resonance Imaging (fMRI) enables fine-grained retrieval of brain activity. However, the accurate reconstruction of diverse details, including structure, background, texture, color, and more, remains challenging. The stable diffusion models inevitably result in the variability of reconstructed images, even under identical conditions. To address this challenge, we first uncover the neuroscientific perspective of diffusion methods, which primarily involve top-down creation using pre-trained knowledge from extensive image datasets, but tend to lack detail-driven bottom-up perception, leading to a loss of faithful details. In this paper, we propose NeuralDiffuser which incorporates primary visual feature guidance to provide detailed cues in the form of gradients. This extension of the bottom-up process for diffusion models achieves both semantic coherence and detail fidelity when reconstructing visual stimuli. Furthermore, we have developed a novel guidance strategy for reconstruction tasks that ensures the consistency of repeated outputs with original images rather than with various outputs. Extensive experimental results on the Natural Senses Dataset (NSD) qualitatively and quantitatively demonstrate the advancement of NeuralDiffuser by comparing it against baseline and state-of-the-art methods horizontally, as well as conducting longitudinal ablation studies. Code can be available on https://github.com/HaoyyLi/NeuralDiffuser.

Multi-view spectral clustering has attracted considerable attention since it can explore common geometric structures from diverse views. Nevertheless, existing min-min framework-based models adopt internal minimization to find the view combination with the minimized within-cluster variance, which will lead to effectiveness loss since the real clusters often exhibit high within-cluster variance. To address this issue, we provide a novel scalable min-max multi-view spectral clustering (SMMSC) model to improve clustering performance. Besides, anchor graphs, rather than full sample graphs, are utilized to reduce the computational complexity of graph construction and singular value decomposition, thereby enhancing the applicability of SMMSC to large-scale applications. Then, we rewrite the min-max model as a minimized optimal value function, demonstrate its differentiability, and develop an efficient gradient descent-based algorithm to optimize it with linear computational complexity. Moreover, we demonstrate that the resultant solution of the proposed algorithm is the global optimum. Numerous experiments on different real-world datasets, including some large-scale datasets, demonstrate that SMMSC outperforms existing state-of-the-art multi-view clustering methods regarding clustering performance.

Bayesian learning provides a unified skeleton to solve the electrophysiological source imaging task. From this perspective, existing source imaging algorithms utilize the Gaussian assumption for the observation noise to build the likelihood function for Bayesian inference. However, the electromagnetic measurements of brain activity are usually affected by miscellaneous artifacts, leading to a potentially non-Gaussian distribution for the observation noise. Hence the conventional Gaussian likelihood model is a suboptimal choice for the real-world source imaging task. In this study, we aim to solve this problem by proposing a new likelihood model which is robust with respect to non-Gaussian noises. Motivated by the robust maximum correntropy criterion, we propose a new improper distribution model concerning the noise assumption. This new noise distribution is leveraged to structure a robust likelihood function and integrated with hierarchical prior distributions to estimate source activities by variational inference. In particular, the score matching is adopted to determine the hyperparameters for the improper likelihood model. A comprehensive performance evaluation is performed to compare the proposed noise assumption to the conventional Gaussian model. Simulation results show that, the proposed method can realize more precise source reconstruction by designing known ground-truth. The real-world dataset also demonstrates the superiority of our new method with the visual perception task. This study provides a new backbone for Bayesian source imaging, which would facilitate its application using real-world noisy brain signal.

Functional Magnetic Resonance Imaging (fMRI) presents challenges due to limited paired samples and low signal-to-noise ratios, particularly in tasks involving reconstructing natural images or decoding their semantic content. To address these challenges, we introduce BrainCLIP, an innovative fMRI-based brain decoding model. BrainCLIP leverages Contrastive Language-Image Pre-training's (CLIP) cross-modal generalization abilities to bridge brain activity, images, and text for the first time. Our experiments demonstrate CLIP's effectiveness in diverse brain decoding tasks, including zero-shot visual category decoding, fMRI-image/text alignment, and fMRI-to-image generation. The core objective of BrainCLIP is to train a mapping network that translates fMRI patterns into a unified CLIP embedding space, achieved through visual and textual supervision integration. Our experiments highlight that this approach significantly enhances performance in tasks such as fMRI-text alignment and fMRI-based image generation. Notably, BrainCLIP surpasses BraVL, a recent multi-modal method, in zero-shot visual category decoding. Moreover, BrainCLIP demonstrates strong capability in reconstructing visual stimuli with high semantic fidelity, competing favorably with state-of-the-art methods in capturing high-level semantic features during fMRI-based natural image reconstruction.

It is challenging for crowd counting models to generalize to new scenes due to domain shifts in training and test data. Although domain adaptation approaches have made notable progress in bridging the domain gap, they require target domain data. In this paper, we propose a novel framework for cross-scene crowd counting, which unifies domain generalization and adaptation. For domain generalization, we train a model only using single-domain data and the model can be generalized to any scene with satisfying performance. Regarding domain adaptation, we use both source and target domain data to further improve the performance. We first design a generation network that diversifies the generated samples to cover the unseen target domains as much as possible by minimizing mutual information. This approach simulates training data in various domains, thereby enhancing the model's generalization ability. Then we develop a pixel-wise supervised contrastive loss function that pulls the human heads in the source images and generated images closer to each other and pushes them further away from the background. This loss helps extract a domain-invariant feature representation, thus improving the model's generalization ability. Moreover, if information about the target domain is available, our generalization method can be easily applied as an adaptation method by replacing the mutual information minimization loss with the mutual information maximization loss. This can further improve cross-scene crowd counting performance. The experimental results demonstrate the strong generalizability of our method across different datasets.

As an emerging decentralized machine learning technique, federated learning organizes collaborative training and preserves the privacy and security of participants. However, untrustworthy devices, typically Byzantine attackers, pose a significant challenge to federated learning since they can upload malicious parameters to corrupt the global model. To defend against such attacks, we propose a novel robust aggregation method-maximum correntropy aggregation (MCA), which applies the maximum correntropy criterion (MCC) to derive a central value from parameters. Different from the previous use of MCC for denoising, we utilize it as a similarity metric to measure parameter distribution and aggregate a robust center. Correntropy in MCC, with all even-order moments of the parameter, contains high-order statistical properties, which allows for a comprehensive capture of parameter characteristics, thus helping to prevent interference from attackers. Meanwhile, correntropy extracts information from the parameters themselves, without requiring the proportion of malicious attackers. Through the fixed-point iteration, we solve the optimization objective, demonstrating the linear convergence of the iteration formula. Theoretical analysis reveals the robustness aggregation property of MCA and the error bound between MCA and the global optimal solution, with linear convergence to the optimal solution neighborhood. By performing independent identically distribution (IID) and non-IID experiments on three different datasets, we show that MCA exhibits significant robustness under mainstream attacks, whereas other methods cannot withstand all of them.

Brain-inspired computing technique presents a promising approach to prompt the rapid development of artificial general intelligence (AGI). As one of the most critical aspects, spiking neural networks (SNNs) have demonstrated superiority for AGI, such as low power consumption. Effective training of SNNs with high generalization ability, high robustness, and low power consumption simultaneously is a significantly challenging problem for the development and success of applications of spike-based machine intelligence. In this research, we present a novel and flexible learning framework termed high-order spike-based information bottleneck (HOSIB) leveraging the surrogate gradient technique. The presented HOSIB framework, including second-order and third-order formation, i.e., second-order information bottleneck (SOIB) and third-order information bottleneck (TOIB), comprehensively explores the common latent architecture and the spike-based intrinsic information and discards the superfluous information in the data, which improves the generalization capability and robustness of SNN models. Specifically, HOSIB relies on the information bottleneck (IB) principle to prompt the sparse spike-based information representation and flexibly balance its exploitation and loss. Extensive classification experiments are conducted to empirically show the promising generalization ability of HOSIB. Furthermore, we apply the SOIB and TOIB algorithms in deep spiking convolutional networks to demonstrate their improvement in robustness with various categories of noise. The experimental results prove the HOSIB framework, especially TOIB, can achieve better generalization ability, robustness and power efficiency in comparison with the current representative studies.

Due to its high computational complexity, graph-based methods have limited applicability in large-scale multiview clustering tasks. To address this issue, many accelerated algorithms, especially anchor graph-based methods and indicator learning-based methods, have been developed and made a great success. Nevertheless, since the restrictions of the optimization strategy, these accelerated methods still need to approximate the discrete graph-cutting problem to a continuous spectral embedding problem and utilize different discretization strategies to obtain discrete sample categories. To avoid the loss of effectiveness and efficiency caused by the approximation and discretization, we establish a discrete fast multiview anchor graph clustering (FMAGC) model that first constructs an anchor graph of each view and then generates a discrete cluster indicator matrix by solving the discrete multiview graph-cutting problem directly. Since the gradient descent-based method makes it hard to solve this discrete model, we propose a fast coordinate descent-based optimization strategy with linear complexity to solve it without approximating it as a continuous one. Extensive experiments on widely used normal and large-scale multiview datasets show that FMAGC can improve clustering effectiveness and efficiency compared to other state-of-the-art baselines.

Developing new diagnostic models based on the underlying biological mechanisms rather than subjective symptoms for psychiatric disorders is an emerging consensus. Recently, machine learning (ML)-based classifiers using functional connectivity (FC) for psychiatric disorders and healthy controls (HCs) are developed to identify brain markers. However, existing ML-based diagnostic models are prone to overfitting (due to insufficient training samples) and perform poorly in new test environments. Furthermore, it is difficult to obtain explainable and reliable brain biomarkers elucidating the underlying diagnostic decisions. These issues hinder their possible clinical applications. In this work, we propose BrainIB, a new graph neural network (GNN) framework to analyze functional magnetic resonance images (fMRI), by leveraging the famed information bottleneck (IB) principle. BrainIB is able to identify the most informative edges in the brain (i.e., subgraph) and generalizes well to unseen data. We evaluate the performance of BrainIB against three baselines and seven state-of-the-art (SOTA) brain network classification methods on three psychiatric datasets and observe that our BrainIB always achieves the highest diagnosis accuracy. It also discovers the subgraph biomarkers that are consistent with clinical and neuroimaging findings. The source code and implementation details of BrainIB are freely available at the GitHub repository (https://github.com/SJYuCNEL/brain-and-Information-Bottleneck).

Recently, despite the unprecedented success of large pre-trained visual-language models (VLMs) on a wide range of downstream tasks, the real-world unsupervised domain adaptation (UDA) problem is still not well explored. Therefore, in this paper, we first experimentally demonstrate that the unsupervised-trained VLMs can significantly reduce the distribution discrepancy between source and target domains, thereby improving the performance of UDA. However, a major challenge for directly deploying such models on downstream UDA tasks is prompt engineering, which requires aligning the domain knowledge of source and target domains, since the performance of UDA is severely influenced by a good domain-invariant representation. We further propose a Prompt-based Distribution Alignment (PDA) method to incorporate the domain knowledge into prompt learning. Specifically, PDA employs a two-branch prompt-tuning paradigm, namely base branch and alignment branch. The base branch focuses on integrating class-related representation into prompts, ensuring discrimination among different classes. To further minimize domain discrepancy, for the alignment branch, we construct feature banks for both the source and target domains and propose image-guided feature tuning (IFT) to make the input attend to feature banks, which effectively integrates self-enhanced and cross-domain features into the model. In this way, these two branches can be mutually promoted to enhance the adaptation of VLMs for UDA. We conduct extensive experiments on three benchmarks to demonstrate that our proposed PDA achieves state-of-the-art performance. The code is available at https://github.com/BaiShuanghao/Prompt-based-Distribution-Alignment.

DETR-like methods have significantly increased detection performance in an end-to-end manner. The main-stream two-stage frameworks of them perform dense self-attention and select a fraction of queries for sparse cross-attention, which is proven effective for improving performance but also introduces a heavy computational burden and high dependence on stable query selection. This paper demonstrates that suboptimal two-stage selection strategies result in scale bias and redundancy due to the mismatch between selected queries and objects in two-stage initial-ization. To address these issues, we propose hierarchical salience filtering refinement, which performs transformer encoding only on filtered discriminative queries, for a bet-ter trade-off between computational efficiency and precision. The filtering process overcomes scale bias through a novel scale-independent salience supervision. To com-pensate for the semantic misalignment among queries, we introduce elaborate query refinement modules for stable two-stage initialization. Based on above improvements, the proposed Salience DETR achieves significant improvements of +4.0% AP, +0.2% AP, +4.4% AP on three challenging task-specific detection datasets, as well as 49.2% AP on COCO 2017 with less FLOPs. The code is available at https://github.com/xiuqhou/Salience-DETR.

Cross-domain few-shot classification (CDFSC) is a challenging and tough task due to the significant distribution discrepancies across different domains. To address this challenge, many approaches aim to learn transferable representations. Multilayer perceptron (MLP) has shown its capability to learn transferable representations in various downstream tasks, such as unsupervised image classification and supervised concept generalization. However, its potential in the few-shot settings has yet to be comprehensively explored. In this study, we investigate the potential of MLP to assist in addressing the challenges of CDFSC. Specifically, we introduce three distinct frameworks incorporating MLP in accordance with three types of few-shot classification methods to verify the effectiveness of MLP. We reveal that MLP can significantly enhance discriminative capabilities and alleviate distribution shifts, which can be supported by our expensive experiments involving 10 baseline models and 12 benchmark datasets. Furthermore, our method even compares favorably against other state-of-the-art CDFSC algorithms.

Emotion recognition based on electroencephalography (EEG) signals is one of the current research challenges in this field. In order to learn the optimal graph structure information for each subject, we propose a dynamic graph attention neural network model. The model utilizes a graph attention neural network as a feature learner, dynamically learning channel connections, and enriching feature representations between channels through global attention. To verify the effectiveness of the proposed method, we conducted experiments on the publicly available emotion recognition dataset SEED. The experimental results show that the average accuracy and standard deviation of the 15 subjects are 94.6% and 4.98%, respectively. The results indicate that our proposed dynamic graphical attention neural network outperforms existing methods.

Repetitive transcranial magnetic stimulation (rTMS) is a common non-invasive treatment for medication-resistant major depressive disorder (MDD). It utilizes continuous and adjustable magnetic stimulation to modulate neural circuits implicated in the pathogenesis of depression. Nevertheless, constructing a universal and effective predictive factor for forecasting treatment outcomes remains challenging. To address this, we first collect neuroimaging data and five depression scales from 26 medication-resistant MDD patients before and after rTMS treatment. Then we propose a novel framework for predicting treatment effects precisely, which combines open-loop control and neural manifold estimation. This framework utilizes the geometric information of the manifold as a biomarker to predict the therapeutic efficacy of rTMS. Experiments based on the clinical dataset demonstrate the effectiveness and robustness of our framework.

Most previous embedding-based methods for anomaly detection directly utilize the visual features extracted from pretrained CNN network. However, there usually exists a gap of domain between pretrained data and target data in anomaly detection. To alleviate this discrepancy, we introduce ReconFA in this paper, a self-supervised domain adaptation method for anomaly detection: firstly, we design a self-supervised multi-scale features fusion method with multi-scale aggregation module (MSAM) to enhance interaction of multi-level outputs of pretrained CNN. Secondly, we train an encoder to adapt the features extracted from pretrained CNN to target domain by a feature-reconstruction task. Meanwhile, we use encoder to reduce the dimension of features to save space complexity. Extensive experiments show that our ReconFA method outperforms previous methods on MVTec AD datasets and on more challenging and sophisticated datasets MPDD.

Federated learning empowers privacy-preserving, multi-party secure model training without the necessity of sharing raw data. In recent years, knowledge distillation has emerged as a promising solution to address the significant challenge of model heterogeneity within federated learning. However, current research often overlooks the potential threats posed by Byzantine attacks, which can significantly compromise the security of federated distillation. Previous work on Byzantine attacks has been primarily focused on manipulating local gradients to compromise global model, lacking attacks on logits in knowledge distillation scenarios. In this paper, we introduce two innovative attacks, shedding light on the inherent risks in federated distillation. The proposed attacks include a top-k attack, which perturbs the top k values of logits in each column, and an impersonation attack, which emulates knowledge significantly deviating from the norm. To counter such attacks, we propose a robust aggregation strategy-FedTGD (Federated Top Guard Distillation), designed to ensure robust distillation with heterogeneous models. Specifically, FedTGD incorporates Density-Based Spatial Clustering of Applications with Noise (DBSCAN) and maximum cosine similarity on top-k values of logits to select benign knowledge. Experimental evaluations conducted on FEMNIST and CIFAR100 datasets, considering scenarios for both IID and Non-IID, reveal that top-k attack results in a substantial 27.16% accuracy reduction for FedMD. In contrast, our aggregation method shows a marginal 0.7% accuracy decrease under top-k attacks, outperforming state-of-the-art baselines.

With the advancement of neural networks, diverse methods for neural Granger causality have emerged, which demonstrate proficiency in handling complex data, and nonlinear relationships. However, the existing framework of neural Granger causality has several limitations. It requires the construction of separate predictive models for each target variable, and the relationship depends on the sparsity on the weights of the first layer, resulting in challenges in effectively modeling complex relationships between variables as well as unsatisfied estimation accuracy of Granger causality. Moreover, most of them cannot grasp full-time Granger causality. To address these drawbacks, we propose a **J**acobian **R**egularizer-based **N**eural **G**ranger **C**ausality (**JRNGC**) approach, a straightforward yet highly effective method for learning multivariate summary Granger causality and full-time Granger causality by constructing a single model for all target variables. Specifically, our method eliminates the sparsity constraints of weights by leveraging an input-output Jacobian matrix regularizer, which can be subsequently represented as the weighted causal matrix in the post-hoc analysis. Extensive experiments show that our proposed approach achieves competitive performance with the state-of-the-art methods for learning summary Granger causality and full-time Granger causality while maintaining lower model complexity and high scalability.

Electroencephalography signals are increasingly used in affective computing. Taking into account subject differences, we propose a novel Graph Transformer network model based on self-attention and channel attention. The model addresses variations in EEG channel connectivity relationships among subjects by utilizing a mapping layer, while leveraging a Transformer-based graph neural network and channel attention to learn and enhance the representational capacities of the output. To validate the effectiveness of the proposed method, we conducted experiments on each of the 15 subjects of the SEED dataset. The average accuracy and standard deviation of the proposed method are 94.35% and 5.05%, respectively. The results show that our method outperforms the existing methods.

Despite the success of Multi-Agent Reinforcement Learning (MARL) algorithms in cooperative tasks, previous works, unfortunately, face challenges in heterogeneous scenarios since they simply disable parameter sharing for agent specialization. Sequential updating scheme was thus proposed, naturally diversifying agents by encouraging agents to learn from preceding ones. However, the exploration strategy in sequential scheme has not been investigated. Benefiting from updating one-by-one, agents have the access to the information from preceding agents. Thus, in this work, we propose to exploit the preceding information to enhance exploration and heterogeneity sequentially. We present Multi-Agent Divergence Policy Optimization (MADPO), equipped with mutual policy divergence maximization framework. We quantify the policy discrepancies between episodes to enhance exploration and between agents to heterogenize agents, termed intra-agent and inter-agent policy divergence. To address the issue that traditional divergence measurements lack stability and directionality, we propose to employ the conditional Cauchy-Schwarz divergence to provide entropy-guided exploration incentives. Extensive experiments show that the proposed method outperforms state-of-the-art sequential updating approaches in two challenging multi-agent tasks with various heterogeneous scenarios. Source code is available at \url{https://github.com/hwdou6677/MADPO}.

With the advancement of artificial intelligence, deep reinforcement learning is emerging as an effective solution for distribution system service restoration. However, traditional deep reinforcement learning approaches are typically tailored for training agents in specific scenarios, limiting their ability to adapt rapidly to new environments. Furthermore, the spatial characteristics of the distribution network are largely ignored during the training, constraining the state perception capabilities of agents. To address these issues, this paper proposes a meta-based graph reinforcement learning approach that combines graph learning, meta-learning, and reinforcement learning for the learning of service restoration strategies in distribution network. The agent trained by such an approach possesses the feature perception capability of graph learning, allowing it to acquire deeper service restoration strategies from latent graph features. Moreover, the agent also has the fast adaptation ability of meta-learning, enabling it to quickly adapt to new restoration scenarios. Experimental results demonstrate that the proposed approach outperforms existing results of both specialized and generalized strategies.

Latent multi-view subspace clustering has been demonstrated to have desirable clustering performance. However, the original latent representation method vertically concatenates the data matrices from multiple views into a single matrix along the direction of dimensionality to recover the latent representation matrix, which may result in an incomplete information recovery. To fully recover the latent space representation, we in this paper propose an Enhanced Latent Multi-view Subspace Clustering (ELMSC) method. The ELMSC method involves constructing an augmented data matrix that enhances the representation of multi-view data. Specifically, we stack the data matrices from various views into the block-diagonal locations of the augmented matrix to exploit the complementary information. Meanwhile, the non-block-diagonal entries are composed based on the similarity between different views to capture the consistent information. In addition, we enforce a sparse regularization for the non-diagonal blocks of the augmented self-representation matrix to avoid redundant calculations of consistency information. Finally, a novel iterative algorithm based on the framework of Alternating Direction Method of Multipliers (ADMM) is developed to solve the optimization problem for ELMSC. Particularly, we theoretically analyze the convergence of ELMSC in detail. Extensive experiments on real-world datasets show that our proposed ELMSC is able to achieve higher clustering performance than some state-of-art multi-view clustering methods. Moreover, our experiments show that our method remains effective with randomly chosen parameters, demonstrating ELMSC’s practical potential.

Predicting the trajectory of pedestrians in crowd scenarios is indispensable in self-driving or autonomous mobile robot field because estimating the future locations of pedestrians around is beneficial for policy decision to avoid collision. It is a challenging issue because humans have different walking motions, and the interactions between humans and objects in the current environment, especially between humans themselves, are complex. Previous researchers focused on how to model human-human interactions but neglected the relative importance of interactions. To address this issue, a novel mechanism based on correntropy is introduced. The proposed mechanism not only can measure the relative importance of human-human interactions but also can build personal space for each pedestrian. An interaction module, including this data-driven mechanism, is further proposed. In the proposed module, the data-driven mechanism can effectively extract the feature representations of dynamic human-human interactions in the scene and calculate the corresponding weights to represent the importance of different interactions. To share such social messages among pedestrians, an interaction-aware architecture based on long short-term memory network for trajectory prediction is designed. Experiments are conducted on two public datasets. Experimental results demonstrate that our model can achieve better performance than several latest methods with good performance.

Crowd counting models in highly congested areas confront two main challenges: weak localization ability and difficulty in differentiating between foreground and background, leading to inaccurate estimations. The reason is that objects in highly congested areas are normally small and high-level features extracted by convolutional neural networks are less discriminative to represent small objects. To address these problems, we propose a learning discriminative features framework for crowd counting, which is composed of a masked feature prediction module (MPM) and a supervised pixel-level contrastive learning module (CLM). The MPM randomly masks feature vectors in the feature map and then reconstructs them, allowing the model to learn about what is present in the masked regions and improving the model's ability to localize objects in high-density regions. The CLM pulls targets close to each other and pushes them far away from background in the feature space, enabling the model to discriminate foreground objects from background. Additionally, the proposed modules can be beneficial in various computer vision tasks, such as crowd counting and object detection, where dense scenes or cluttered environments pose challenges to accurate localization. The proposed two modules are plug-and-play, incorporating the proposed modules into existing models can potentially boost their performance in these scenarios.

Proper monitoring of anesthesia stages can guarantee the safe performance of clinical surgeries. In this study, different anesthesia stages were classified using near-infrared spectroscopy (NIRS) signals with machine learning. The cerebral hemodynamic variables of right proximal oxyhemoglobin (HbO2) in maintenance (MNT), emergence (EM) and the consciousness (CON) stage were collected and then the differences between the three stages were compared by phase-amplitude coupling (PAC). Then combined with time-domain including linear (mean, standard deviation, max, min and range), nonlinear (sample entropy) and power in frequency-domain signal features, feature selection was performed and finally classification was performed by support vector machine (SVM) classifier. The results show that the PAC of the NIRS signal was gradually enhanced with the deepening of anesthesia level. A good three-classification accuracy of 69.27% was obtained, which exceeded the result of classification of any single category feature. These results indicate the fesibility of NIRS signals in performing three or even more anesthesia stage classifications, providing insight into the development of new anesthesia monitoring modalities.

Incomplete multi-view clustering (IMVC) aims to leverage complementary information from multi-view data with missing instances to enhance clustering performance. Many existing IMVC methods exhibit limitations in effectively exploiting hidden information and addressing distribution differences between views and modules. To address these challenges, we present a novel IMVC framework that leverages the proposed stack feature-based matrix completion to impute the missing instances, enhancing the exploitation of underlying information. We also incorporate graph consensus to integrate graph structures learned from both completed and observed data. Additionally, we introduce correntropy-induced metric as a flexible measurement to adaptively assign different constraints to various views and modules. Furthermore, we derive an efficient iterative algorithm based on Fenchel conjugate and accelerated block coordinate update (BCU) to solve the joint learning problem. Experimental results on eight benchmark datasets demonstrate the superior performance of our method compared to state-of-the-art IMVC methods across various metrics.

A novel model called error loss network (ELN) is proposed to build an error loss function for supervised learning. The ELN is similar in structure to a radial basis function (RBF) neural network, but its input is an error sample and output is a loss corresponding to that error sample. That means the nonlinear input-output mapper of the ELN creates an error loss function. The proposed ELN provides a unified model for a large class of error loss functions, which includes some information-theoretic learning (ITL) loss functions as special cases. The activation function, weight parameters, and network size of the ELN can be predetermined or learned from the error samples. On this basis, we propose a new machine learning paradigm where the learning process is divided into two stages: first, learning a loss function using an ELN; second, using the learned loss function to continue to perform the learning. Experimental results are presented to demonstrate the desirable performance of the new method.

Multichannel electroencephalogram (EEG) is an array signal that represents brain neural networks and can be applied to characterize information propagation patterns for different emotional states. To reveal these inherent spatial graph features and increase the stability of emotion recognition, we propose an effective emotion recognition model that performs multicategory emotion recognition with multiple emotion-related spatial network topology patterns (MESNPs) by learning discriminative graph topologies in EEG brain networks. To evaluate the performance of our proposed MESNP model, we conducted single-subject and multisubject four-class classification experiments on two public datasets, MAHNOB-HCI and DEAP. Compared with existing feature extraction methods, the MESNP model significantly enhances the multiclass emotional classification performance in the single-subject and multisubject conditions. To evaluate the online version of the proposed MESNP model, we designed an online emotion monitoring system. We recruited 14 participants to conduct the online emotion decoding experiments. The average online experimental accuracy of the 14 participants was 84.56%, indicating that our model can be applied in affective brain-computer interface (aBCI) systems. The offline and online experimental results demonstrate that the proposed MESNP model effectively captures discriminative graph topology patterns and significantly improves emotion classification performance. Moreover, the proposed MESNP model provides a new scheme for extracting features from strongly coupled array signals.

Scene Graph Generation (SGG) aims to capture the semantic information in an image and build a structured representation, which facilitates downstream tasks. The current challenge in SGG is to tackle the biased predictions caused by the long-tailed distribution of predicates. Since multiple predicates in SGG are coupled in an image, existing data re-balancing methods cannot completely balance the head and tail predicates. In this work, a decoupled learning framework is proposed for unbiased scene graph generation by using attribute-guided predicate features to construct a balanced training set. Specifically, the predicate recognition is decoupled into Predicate Feature Representation Learning (PFRL) and predicate classifier training with a class-balanced predicate feature set, which is constructed by our proposed Attribute-guided Predicate Feature Generation (A-PFG) model. In the A-PFG model, we first define the class labels of and corresponding visual feature as attributes to describe a predicate. Then the predicate feature and the attribute embedding are mapped into a shared hidden space by a dual Variational Auto-encoder (VAE), and finally the synthetic predicate features are forced to learn the contextual information in the attributes via cross reconstruction and distribution alignment. To demonstrate the effectiveness of our proposed method, our decoupled learning framework and A-PFG model are applied to various SGG models. The empirical results show that our method is substantially improved on all benchmarks and achieves new state-of-the-art performance for unbiased scene graph generation. Our code is available at https://github.com/wanglei0618/A-PFG.

Graph Neural Networks (GNNs) have been widely applied to graph-structured data. However, the lack of interpretability impedes its practical deployment especially in high-risk areas such as medical diagnosis. Recently, the Information Bottleneck (IB) principle has been extended to GNNs to identify a compact subgraph that is most informative to class labels, which significantly improves the interpretability on decision. However, existing Graph Information Bottleneck (GIB) models are either unstable during the training (due to the difficulty of mutual information estimation) or only focus on a special kind of graph (e.g., brain networks) that suffer from poor generalization to general graph datasets with varying graph sizes. In this work, we extend the recently developed Brain Information Bottleneck (BrainIB) to general graphs by introducing matrix-based Rényi’s α-order mutual information to stablize the training; and by designing a novel mask strategy to deal with varying graph sizes such that the new method can also be used for social networks, molecules, etc. Extensive experiments on different types of graph datasets demonstrate the superior stability and generality of our model.

Previous approaches to the problem of generalization for out-of-distribution (OOD) data usually assume that data from each environment is available simultaneously, which is unrealistic in real-world applications. In this paper, we develop a new framework termed the sequential invariant information bottleneck (seq-IIB) to improve the generalization ability of learning agents in sequential environments. Our main idea is to combine the merits of the famed Information Bottleneck (IB) principle with the Invariant Risk Minimization (IRM), such that the learning agent can gradually remove spurious features and remain invariant and compact task-relevant information in a sequential manner. Experimental results on three MNIST-like datasets show the effectiveness of our method.

Sparseness and robustness are two important properties for many machine learning scenarios. In the present study, regarding the maximum correntropy criterion (MCC) based robust regression algorithm, we investigate to integrate the MCC method with the automatic relevance determination (ARD) technique in a Bayesian framework, so that MCC-based robust regression could be implemented with ‘adaptive sparseness’. To be specific, we use an inherent noise assumption from the MCC to derive an explicit likelihood function, and realize the maximum a posteriori (MAP) estimation with the ARD prior by variational Bayesian inference. Compared to the existing robust and sparse$L_{1}$-regularized MCC regression, the proposed MCC-ARD regression can eradicate the troublesome tuning for the regularization hyper-parameter which controls the regularization strength. Further, MCC-ARD achieves superior prediction performance and feature selection capability than$L_{1}$-regularized MCC, as demonstrated by a noisy and high-dimensional simulation study. The codes of the proposed method are shared at https://sites.google.com/view/liyuanhao/codes.

Defect inspection, also known as defect detection, is significant in mobile screen quality control. There are some challenging issues brought by the characteristics of screen defects, including the following: (1) the problem of interclass similarity and intraclass variation, (2) the difficulty in distinguishing low contrast, tiny-sized, or incomplete defects, and (3) the modeling of category dependencies for multi-label images. To solve these problems, a graph reasoning module, stacked on a classification module, is proposed to expand the feature dimension and improve low-quality image features by exploiting category-wise dependency, image-wise relations, and interactions between them. To further improve the classification performance, the classifier of the classification module is redesigned as a cosine similarity function. With the help of contrastive learning, the classification module can better initialize the category-wise graph of the reasoning module. Experiments on the mobile screen defect dataset show that our two-stage network achieves the following best performances: 97.7% accuracy and 97.3% F -measure. This proves that the proposed approach is effective in industrial applications.

The maximum correntropy criterion (MCC) algorithm depends upon two fundamental parameters, i.e., step-size and kernel width. Previous studies of parameter optimization in the MCC mainly focus on a single parameter (mainly the kernel width), lacking optimization research concerning both parameters. To this end, this letter investigates a novel optimization scheme simultaneously involving step-size and kernel width. The optimization framework is based on making the power of weight error vector undergo the steepest attenuation. Under the premise of maintaining the same evolutionary trend for time-varying step-size and kernel width, we formulate a constrained parameter optimization problem, where the step-size is subject to a kernel width induced constraint. By taking this approach, the original bivariate optimization can be transformed into a univariate optimization problem, which facilitates optimization solving. We further develop an existing reset scheme to make it suitable for kernel width to ensure a good tracking capability. In addition, we investigate the convergence behavior of the optimized algorithm. Simulation results demonstrate that the developed optimization scheme is beneficial for performance improvement, and the resulting algorithm outperforms some state-of-art MCC-based algorithms.

In real-world crowd counting applications, the crowd densities in an image vary greatly. When facing density variation, humans tend to locate and count the targets in low-density regions, and reason the number in high-density regions. We observe that CNN focus on the local information correlation using a fixed-size convolution kernel and the Transformer could effectively extract the semantic crowd information by using the global self-attention mechanism. Thus, CNN could locate and estimate crowds accurately in low-density regions, while it is hard to properly perceive the densities in high-density regions. On the contrary, Transformer has a high reliability in high-density regions, but fails to locate the targets in sparse regions. Neither CNN nor Transformer can well deal with this kind of density variation. To address this problem, we propose a CNN and Transformer Adaptive Selection Network (CTASNet) which can adaptively select the appropriate counting branch for different density regions. Firstly, CTASNet generates the prediction results of CNN and Transformer. Then, considering that CNN/Transformer is appropriate for low/high-density regions, a density guided adaptive selection module is designed to automatically combine the predictions of CNN and Transformer. Moreover, to reduce the influences of annotation noise, we introduce a Correntropy based optimal transport loss. Extensive experiments on four challenging crowd counting datasets have validated the proposed method.

Nonnegative matrix factorization (NMF) based multiview technique has been commonly used in multiview data clustering tasks. However, previous NMF based multiview clustering approaches fail to take advantage of a small amount of supervisory information to effectively improve the clustering performance, and are easily affected by the additional post-processing method in clustering tasks. To cope with these issues, a novel framework named multiview clustering via hypergraph induced semi-supervised symmetric NMF (MVCHSS) is proposed in this paper for multiview data clustering applications. Specifically, the proposed method has the following features: 1) a new multiview based hypergraph pairwise constraints propagation (MHPCP) algorithm is developed in MVCHSS to construct a set of informative similarity matrices, revealing the high-order relationships effectively and fully utilizing the limited pairwise constraint supervisory information among samples of each view data; 2) the obtained similarity matrices with much supervisory information are not only enforced into the symmetric NMF (SNMF) model, but also incorporated into the graph regularization for each view data; 3) the optimization problem of MVCHSS is formulated for multiview data clustering tasks to acquire a more discriminative clustering indicator matrix (or called consensus assignment matrix) without additional post-processing method. Moreover, the proof of convergence and the computational complexity for MVCHSS are presented. Extensive experiments on five multiview datasets demonstrate that the proposed MVCHSS framework outperforms several state-of-the-art multiview clustering methods.

Partial discharge (PD) location techniques are a useful tool for condition monitoring of electrical apparatus in power systems. However, the noisy PD measurements may significantly degrade the performance of location algorithms. This article deals with the PD location problem by using adaptive filtering techniques. Heretofore, scarce literature focuses on addressing the PD location based on such method. A novel adaptive algorithm, termed as total least-squares (TLS)-Matérn kernel (TLS-MK), is proposed. Benefiting from the merits of the Matérn kernel, the TLS model can effectively suppress the noise from the direct and reflected waves of the PD source. Meanwhile, the TLS-MK algorithm is used to estimate the time difference, which is used in the PD location. Moreover, the convergence behavior of the TLS-MK algorithm is analyzed. Simulations and experiments show that the proposed algorithm can enhance the location accuracy as compared to state-of-the-art methods for various PD signals.

Counting objects in crowded scenes remains a challenge to computer vision. The current deep learning based approach often formulate it as a Gaussian density regression problem. Such a brute-force regression, though effective, may not consider the annotation displacement properly which arises from the human annotation process and may lead to different distributions. We conjecture that it would be beneficial to consider the annotation displacement in the dense object counting task. To obtain strong robustness against annotation displacement, generalized Gaussian distribution (GGD) function with a tunable bandwidth and shape parameter is exploited to form the learning target point annotation probability map, PAPM. Specifically, we first present a hand-designed PAPM method (HD-PAPM), in which we design a function based on GGD to tolerate the annotation displacement. For end-to-end training, the hand-designed PAPM may not be optimal for the particular network and dataset. An adaptively learned PAPM method (AL-PAPM) is proposed. To improve the robustness to annotation displacement, we design an effective transport cost function based on GGD. The proposed PAPM is capable of integration with other methods. We also combine PAPM with P2PNet through modifying the matching cost matrix, forming P2P-PAPM. This could also improve the robustness to annotation displacement of P2PNet. Extensive experiments show the superiority of our proposed methods.

One of the hottest topics in unsupervised learning is how to efficiently and effectively cluster large amounts of unlabeled data. To address this issue, we propose an orthogonal conceptual factorization (OCF) model to increase clustering effectiveness by restricting the degree of freedom of matrix factorization. In addition, for the OCF model, a fast optimization algorithm containing only a few low-dimensional matrix operations is given to improve clustering efficiency, as opposed to the traditional CF optimization algorithm, which involves dense matrix multiplications. To further improve the clustering efficiency while suppressing the influence of the noises and outliers distributed in real-world data, an efficient correntropy-based clustering algorithm (ECCA) is proposed in this article. Compared with OCF, an anchor graph is constructed and then OCF is performed on the anchor graph instead of directly performing OCF on the original data, which can not only further improve the clustering efficiency but also inherit the advantages of the high performance of spectral clustering. In particular, the introduction of the anchor graph makes ECCA less sensitive to changes in data dimensions and still maintains high efficiency at higher data dimensions. Meanwhile, for various complex noises and outliers in real-world data, correntropy is introduced into ECCA to measure the similarity between the matrix before and after decomposition, which can greatly improve the clustering effectiveness and robustness. Subsequently, a novel and efficient half-quadratic optimization algorithm was proposed to quickly optimize the ECCA model. Finally, extensive experiments on different real-world datasets and noisy datasets show that ECCA can archive promising effectiveness and robustness while achieving tens to thousands of times the efficiency compared with other state-of-the-art baselines.

This article proposes a novel low-rank matrix factorization model for semisupervised image clustering. In order to alleviate the negative effect of outliers, the maximum correntropy criterion (MCC) is incorporated as a metric to build the model. To utilize the label information to improve the clustering results, a constraint graph learning framework is proposed to adaptively learn the local structure of the data by considering the label information. Furthermore, an iterative algorithm based on Fenchel conjugate (FC) and block coordinate update (BCU) is proposed to solve the model. The convergence properties of the proposed algorithm are analyzed, which shows that the algorithm exhibits both objective sequential convergence and iterate sequential convergence. Experiments are conducted on six real-world image datasets, and the proposed algorithm is compared with eight state-of-the-art methods. The results show that the proposed method can achieve better performance in most situations in terms of clustering accuracy and mutual information.

Spiking neural networks (SNNs) have garnered increased attention in the field of artificial general intelligence (AGI) research due to their low power consumption, high computational efficiency, and low latency induced by their event-driven and sparse communication features. However, efficiently and robustly training an SNN presents a challenge. In this study, we introduce a novel framework for spike-based machine learning called spike-based nonlinear information bottleneck (SNIB). This framework utilizes an information-theoretic learning (ITL) approach and a surrogate gradient learning (SGL) method to achieve robust, accurate, and low-power performance. The proposed SNIB framework includes three variants: 1) squared information bottleneck (SIB); 2) cubic information bottleneck (CIB); and 3) quartic information bottleneck (QIB) strategies, which use a mapping mechanism to compress spiking representations. We systematically evaluate these strategies using different types of input noise and neuromorphic hardware noise. Our experimental results demonstrate that all three strategies effectively enhance the robustness of SGL in SNN architectures. Furthermore, SNIB can significantly reduce the power consumption of SNNs. As a result, SNIB offers a new and significant perspective for hardware-constrained general mobile devices for embedded edge intelligence and represents a progressive step toward realizing AGI.

By "intelligently" fuse the complementary information across different views, multi-view learning is able to improve the performance of classification task. In this work, we extend the information bottleneck principle to supervised multi-view learning scenario and use the recently proposed matrix-based Rényi’s α-order entropy functional to optimize the resulting objective directly, without the necessity of variational approximation or adversarial training. Empirical results in both synthetic and real-world datasets suggest that our method enjoys improved robustness to noise and redundant information in each view, especially given limited training samples. Code is available at https://github.com/archy666/MEIB.

Abstract This paper proposes a state estimation approach ‘robust strong tracking unscented Kalman filter with unknown inputs’ that can be applied to non‐linear systems with unknown inputs. Specifically, the non‐linear state and measurement equations are linearised by statistical linearisation. Then, the estimation equation of the unknown input is derived based on the weighted least squares method. The multiple suboptimal fading factor is introduced into a priori error covariance matrix to improve the tracking ability for the inaccuracy of the system model and the abrupt change of state variables caused by unknown inputs. Finally, based on the unbiased minimum variance estimation, the unbiased state estimation and the error covariance matrix are derived. Singular value decomposition is performed on the error covariance matrix to improve the stability of the algorithm. Simulated results validate the effectiveness of the proposed method.

This paper proposes a unified sparsity-aware robust recursive least-squares RLS (S-RRLS) algorithm for the identification of sparse systems under impulsive noise. The proposed algorithm generalizes multiple algorithms only by replacing the specified criterion of robustnessand sparsity-aware penalty. Furthermore, by jointly optimizing the forgetting factor and the sparsity penalty parameter, we develop the jointly-optimized S-RRLS (JO-S-RRLS) algorithm, which not only exhibits low misadjustment but also can track well sudden changes of a sparse system. Simulations in impulsive noise scenarios demonstrate that the proposed S-RRLS and JO-S-RRLS algorithms outperform existing techniques.

In this paper, we propose a general robust subband adaptive filtering (GR-SAF) scheme against impulsive noise by minimizing the mean square deviation under the random-walk model with individual weight uncertainty. Specifically, by choosing different scaling factors such as from the M-estimate and maximum correntropy robust criteria in the GR-SAF scheme, we can easily obtain different GR-SAF algorithms. Importantly, the proposed GR-SAF algorithm can be reduced to a variable regularization robust normalized SAF algorithm, thus having fast convergence rate and low steady-state error. Simulations in the contexts of system identification with impulsive noise and echo cancellation with double-talk have verified that the proposed GR-SAF algorithms outperforms its counterparts.

The random Fourier filter-based filtered-x least mean square (RF-FxLMS) algorithm has been proposed for the nonlinear active noise control (NANC) system to reduce the computational burden of the kernel filter. However, the RF-FxLMS algorithm markedly fluctuates when dealing with impulsive noise. In addition, the computing cost for the RF-FxLMS algorithm is still pricey in practice. In this work, a random Fourier filter based filtered-x generalized hyperbolic secant function (RF-FxGHSF) algorithm is presented to deal with impulsive noise. In virtue of the bilinear scheme, a cascaded random Fourier filter model is designed for concise computations, and the cascaded RF-FxGHSF (CRF-FxGHSF) algorithm is derived. Moreover, the steady-state convergence conditions are analyzed. The calculation complexity of the proposed algorithms is compared, and experiments emphatically analyze the principle for the presented model. Numerical simulations with α-stable noise and real noise carried out in different nonlinear path scenarios verify the convergence ability of proposed algorithms.

Multi-view clustering has received widespread attention owing to its effectiveness by integrating multi-view data appropriately, but traditional algorithms have limited applicability to large-scale real-world data due to their high computational complexity and low robustness. Focusing on the aforementioned issues, we propose an efficient and robust multi-view clustering algorithm with anchor graph regularization (ERMC-AGR). In this work, a novel anchor graph regularization (ARG) is designed to improve the quality of the learned embedded anchor graph (EAG), and the obtained EAG is decomposed by nonnegative matrix factorization (NMF) under correntropy criterion to acquire clustering results directly. Different from the traditional graph regularization that needs to construct a large-scale Laplacian matrix pertaining to the all-sample graph, our lightweight AGR, constructed from the perspective of anchors, can reduce the computational complexity significantly while improving the EAG quality. Moreover, a factor matrix of NMF is constrained to be the cluster indicator matrix to omit additional k-means after optimization. Subsequently, correntropy is utilized to improve the effectiveness and robustness of ERMC-AGR owing to its promising performance to complex noises and outliers. Extensive experiments on real-world datasets and noisy datasets show that ERMC-ARG can improve the clustering efficiency and robustness while ensuring comparable or even better effectiveness.

As a novel similarity measure that is defined as the expectation of a kernel function between two random variables, correntropy has been successfully applied in robust machine learning and signal processing to combat large outliers. The kernel function in correntropy is usually a zero-mean Gaussian kernel. In a recent work, the concept of mixture correntropy (MC) was proposed to improve the learning performance, where the kernel function is a mixture Gaussian kernel, namely, a linear combination of several zero-mean Gaussian kernels with different widths. In both correntropy and MC, the center of the kernel function is, however, always located at zero. In the present work, to further improve the learning performance, we propose the concept of multikernel correntropy (MKC), in which each component of the mixture Gaussian kernel can be centered at a different location. The properties of the MKC are investigated and an efficient approach is proposed to determine the free parameters in MKC. Experimental results show that the learning algorithms under the maximum MKC criterion (MMKCC) can outperform those under the original maximum correntropy criterion (MCC) and the maximum MC criterion (MMCC).

The minimum error entropy (MEE) criterion is a powerful approach for non-Gaussian signal processing and robust machine learning. However, the instantiation of MEE on robust classification is a rather vacancy in the literature. The original MEE purely focuses on minimizing Renyi's quadratic entropy of the prediction errors, which could exhibit inferior capability in noisy classification tasks. To this end, we analyze the optimal error distribution with adverse outliers and introduce a specific codebook for restriction, which optimizes the error distribution toward the optimal case. Half-quadratic-based optimization and convergence analysis of the proposed learning criterion, called restricted MEE (RMEE), are provided. The experimental results considering logistic regression and extreme learning machine on synthetic data and UCI datasets, respectively, are presented to demonstrate the superior robustness of RMEE. Furthermore, we evaluate RMEE on a noisy electroencephalogram dataset, so as to strengthen its practical impact.

Deep hashing methods have shown their superiority to traditional ones. However, they usually require a large amount of labeled training data for achieving high retrieval accuracies. We propose a novel transductive semisupervised deep hashing (TSSDH) method which is effective to train deep convolutional neural network (DCNN) models with both labeled and unlabeled training samples. TSSDH method consists of the following four main ingredients. First, we extend the traditional transductive learning (TL) principle to make it applicable to DCNN-based deep hashing. Second, we introduce confidence levels for unlabeled samples to reduce adverse effects from uncertain samples. Third, we employ a Gaussian likelihood loss for hash code learning to sufficiently penalize large Hamming distances for similar sample pairs. Fourth, we design the large-margin feature (LMF) regularization to make the learned features satisfy that the distances of similar sample pairs are minimized and the distances of dissimilar sample pairs are larger than a predefined margin. Comprehensive experiments show that the TSSDH method can produce superior image retrieval accuracies compared to the representative semisupervised deep hashing methods under the same number of labeled training samples.

Perturbation has a positive effect, as it contributes to the stability of neural systems through adaptation and robustness. For example, deep reinforcement learning generally engages in exploratory behavior by injecting noise into the action space and network parameters. It can consistently increase the agent's exploration ability and lead to richer sets of behaviors. Evolutionary strategies also apply parameter perturbations, which makes network architecture robust and diverse. Our main concern is whether the notion of synaptic perturbation introduced in a spiking neural network (SNN) is biologically relevant or if novel frameworks and components are desired to account for the perturbation properties of artificial neural systems. In this work, we first review part of the locality-sensitive hashing (LSH) of similarity search, the FLY algorithm, as recently published in Science, and propose an improved architecture, time-shifted spiking LSH (TS-SLSH), with the consideration of temporal perturbations of the firing moments of spike pulses. Experiment results show promising performance of the proposed method and demonstrate its generality to various spiking neuron models. Therefore, we expect temporal perturbation to play an active role in SNN performance.

Kernel-based extreme learning machine (KELM), as a natural extension of ELM to kernel learning, has achieved outstanding performance in addressing various regression and classification problems. Compared with the basic ELM, KELM has a better generalization ability owing to no needs of the number of hidden nodes given beforehand and random projection mechanism. Since KELM is derived under the minimum mean square error (MMSE) criterion for the Gaussian assumption of noise, its performance may deteriorate under the non-Gaussian cases, seriously. To improve the robustness of KELM, this article proposes a mixture correntropy-based KELM (MC-KELM), which adopts the recently proposed maximum mixture correntropy criterion as the optimization criterion, instead of using the MMSE criterion. In addition, an online sequential version of MC-KELM (MCOS-KELM) is developed to deal with the case that the data arrive sequentially (one-by-one or chunk-by-chunk). Experimental results on regression and classification data sets are reported to validate the performance superiorities of the new methods.

In this article, a novel normal-skew mixture (NSM) distribution is presented to model the normal and/or heavy-tailed and/or skew nonstationary distributed noises. The NSM distribution can be formulated as a hierarchically Gaussian presentation by leveraging a Bernoulli distributed random variable. Based on this, a novel robust Kalman filtering framework can be developed utilizing the variational Bayesian method, where the one-step prediction and measurement-likelihood densities are modeled as NSM distributions. For implementation, several exemplary robust Kalman filters (KFs) are derived based on some specific cases of NSM distribution. The relationships between some existing robust KFs and the presented framework are also revealed. The superiority of the proposed robust Kalman filtering framework is validated by a target tracking simulation example.

The multivariate autoregressive (MVAR) model is widely used in describing the dynamics of nonlinear systems, in which the estimates of model parameters and underlying states can be achieved by dual extended Kalman filter (DEKF). However, when the measurements are corrupted by complicated non-Gaussian noises, the DEKF based on the minimum mean-square error (MMSE) criterion may provide biased estimates. In the present article, we develop a novel dual Kalman-type filter, referred to as DEKF under minimum error entropy (MEE) with fiducial points (MEEFs-DEKF) to deal with the non-Gaussian noises. First, the equivalent state-space model and parameter-space model are presented based on the MVAR model. Then, an optimality criterion based on MEE with fiducial points (MEEFs) is applied in the batch-mode regression equation to improve the robustness. Finally, a fixed-point iteration algorithm gives the posterior estimates of state and parameter. Simulation results confirm that the proposed MEEF-DEKF can achieve excellent performance in various noises with different distributions, especially in heavy-tailed and multimodal noises.

Recently, the motion averaging method has been introduced as an effective means to solve the multi-view registration problem. This method aims to recover global motions from a set of relative motions, where the original method is sensitive to outliers due to using the Frobenius norm error in the optimization. Accordingly, this paper proposes a novel robust motion averaging method based on the maximum correntropy criterion (MCC). Specifically, the correntropy measure is used instead of utilizing Frobenius norm error to improve the robustness of motion averaging against outliers. According to the half-quadratic technique, the correntropy measure based optimization problem can be solved by the alternating minimization procedure, which includes operations of weight assignment and weighted motion averaging. Further, we design a selection strategy of adaptive kernel width to take advantage of correntropy. Experimental results on benchmark data sets illustrate that our method has superior performance on accuracy and robustness for multi-view registration. What’s more, it can be applied to robot mapping.

Due to the effectiveness in learning the subspace structures, low-rank representation (LRR) and its variations have been widely applied in various fields, such as computer vision and pattern recognition. However, in real applications, it is a challenge to handle the complex noises. To address this problem, we propose a novel robust LRR method based on kernel risk-sensitive loss (KRSL) with high-order manifold constraint, called RHLRR, in which the KRSL is introduced to deal with the noises and the multiple hypergraph regularization term is used as a high order manifold constraint to effectively capture the locality, similarity and the intrinsic geometric information in data. Besides, an iterative algorithm based on the half-quadratic (HQ) and the accelerated block coordinate update (BCU) is developed. The experimental results demonstrate that the proposed method can outperform other state-of-the-art LRR variants.

Multiview subspace clustering, which aims to cluster the given data points with information from multiple sources or features into their underlying subspaces, has a wide range of applications in the communities of data mining and pattern recognition. Compared with the single-view subspace clustering, it is challenging to efficiently learn the structure of the representation matrix from each view and make use of the extra information embedded in multiple views. To address the two problems, a novel correntropy-based multiview subspace clustering (CMVSC) method is proposed in this article. The objective function of our model mainly includes two parts. The first part utilizes the Frobenius norm to efficiently estimate the dense connections between the points lying in the same subspace instead of following the standard compressive sensing approach. In the second part, the correntropy-induced metric (CIM) is introduced to characterize the noise in each view and utilize the information embedded in different views from an information-theoretic perspective. Furthermore, an efficient iterative algorithm based on the half-quadratic technique (HQ) and the alternating direction method of multipliers (ADMM) is developed to optimize the proposed joint learning problem, and extensive experimental results on six real-world multiview benchmarks demonstrate that the proposed methods can outperform several state-of-the-art multiview subspace clustering methods.

Robust point set registration is a challenging problem, especially in the cases of noise, outliers, and partial overlapping. Previous methods generally formulate their objective functions based on the mean-square error (MSE) loss and, hence, are only able to register point sets under predefined constraints (e.g., with Gaussian noise). This article proposes a novel objective function based on a bidirectional kernel mean p -power error (KMPE) loss, to jointly deal with the above nonideal situations. KMPE is a nonsecond-order similarity measure in kernel space and shows a strong robustness against various noise and outliers. Moreover, a bidirectional measure is applied to judge the registration, which can avoid the ill-posed problem when a lot of points converges to the same point. In particular, we develop two effective optimization methods to deal with the point set registrations with the similarity and the affine transformations, respectively. The experimental results demonstrate the effectiveness of our methods.

As an effective and efficient discriminative learning method, broad learning system (BLS) has received increasing attention due to its outstanding performance in various regression and classification problems. However, the standard BLS is derived under the minimum mean square error (MMSE) criterion, which is, of course, not always a good choice due to its sensitivity to outliers. To enhance the robustness of BLS, we propose in this work to adopt the maximum correntropy criterion (MCC) to train the output weights, obtaining a correntropy-based BLS (C-BLS). Due to the inherent superiorities of MCC, the proposed C-BLS is expected to achieve excellent robustness to outliers while maintaining the original performance of the standard BLS in the Gaussian or noise-free environment. In addition, three alternative incremental learning algorithms, derived from a weighted regularized least-squares solution rather than pseudoinverse formula, for C-BLS are developed. With the incremental learning algorithms, the system can be updated quickly without the entire retraining process from the beginning when some new samples arrive or the network deems to be expanded. Experiments on various regression and classification data sets are reported to demonstrate the desirable performance of the new methods.

To date, most linear and nonlinear Kalman filters (KFs) have been developed under the Gaussian assumption and the well-known minimum mean square error (MMSE) criterion. In order to improve the robustness with respect to impulsive (or heavy-tailed) non-Gaussian noises, the maximum correntropy criterion (MCC) has recently been used to replace the MMSE criterion in developing several robust Kalman-type filters. To deal with more complicated non-Gaussian noises such as noises from multimodal distributions, in this article, we develop a new Kalman-type filter, called minimum error entropy KF (MEE-KF), by using the minimum error entropy (MEE) criterion instead of the MMSE or MCC. Similar to the MCC-based KFs, the proposed filter is also an online algorithm with the recursive process, in which the propagation equations are used to give prior estimates of the state and covariance matrix, and a fixed-point algorithm is used to update the posterior estimates. In addition, the MEE extended KF (MEE-EKF) is also developed for performance improvement in the nonlinear situations. The high accuracy and strong robustness of MEE-KF and MEE-EKF are confirmed by experimental results.

Recently, maximum correntropy criterion (MCC) has been widely and successfully used in robust signal processing and machine learning, in which the correntropy is maximized instead of minimizing the popular mean square error (MSE) to improve the robustness with respect to outliers or impulsive noises. A lot of efforts have been devoted to derive different adaptive algorithms under MCC, but to date, little insight has been gained as to how the MCC solution will be influenced by outliers. In this paper, we investigate this problem and our focus is mainly on the parameter estimation of a simple linear errors-in-variables (EIVs) model with scalar variables. Under some conditions, we derive an upper bound on the absolute value of the estimation error and show that the MCC solution can get very close to the true value of the unknown parameter even with arbitrarily large outliers in both the input and output variables. Illustrative examples are provided to verify and clarify the theory.

The extended Kalman filter (EKF) is a method extensively applied in many areas, particularly, in nonlinear target tracking. The optimization criterion commonly used in EKF is the celebrated minimum mean square error (MMSE) criterion, which exhibits excellent performance under Gaussian noise assumption. However, its performance may degrade dramatically when the noises are heavy tailed. To cope with this problem, this paper proposes two new nonlinear filters, namely the linear regression maximum correntropy EKF (LRMCEKF) and nonlinear regression maximum correntropy EKF (NRMCEKF), by applying the maximum correntropy criterion (MCC) rather than the MMSE criterion to EKF. In both filters, a regression model is formulated, and a fixed-point iterative algorithm is utilized to obtain the posterior estimates. The effectiveness and robustness of the proposed algorithms in target tracking are confirmed by an illustrative example.

The sparse channel estimation problem is drawing increasing attention in broadband wireless communication. Sparsity nature in structure and noise is one of the most important issues in such problems. Researchers have devised several sparsity penalty terms such as zero-attracting (ZA) and correntropy induced metric (CIM) to exploit potential sparse structure information and improve sparse channel estimate accuracy. To combat impulsive (sparse) noises, several adaptive filtering algorithms based on the maximum correntropy criterion (MCC) have been developed, which can achieve excellent performance, especially in heavy-tailed noises. Recently, the concept of mixture correntropy and the maximum mixture correntropy criterion (MMCC) were proposed to further promote the robustness of the MCC against impulsive noises. In this paper, a new robust and sparse adaptive filtering algorithm is developed to estimate the sparse channels with impulsive noises by combining the MMCC and CIM penalty. Thanks to the desirable property of the mixture correntropy, the proposed method behaves quite well with excellent convergence performance. Simulation results show that the new method can outperform several existing methods in sparse channel estimation including the MCC based methods.

Nonnegative matrix factorization (NMF) has attracted increasing attention in data mining and machine learning . However, existing NMF methods have some limitations. For example, some NMF methods seriously suffer from noisy data contaminated by outliers, or fail to preserve the geometric information of the data and guarantee the sparse parts-based representation. To overcome these issues, in this paper, a robust and sparse NMF method, called correntropy based dual graph regularized nonnegative matrix factorization with local coordinate constraint (LCDNMF) is proposed. Specifically, LCDNMF incorporates the geometrical information of both the data manifold and the feature manifold, and the local coordinate constraint into the correntropy based objective function. The half-quadratic optimization technique is utilized to solve the nonconvex optimization problem of LCDNMF, and the multiplicative update rules are obtained. Furthermore, some properties of LCDNMF including the convergence, relation with gradient descent method , robustness, and computational complexity are analyzed. Experiments of clustering demonstrate the effectiveness and robustness of the proposed LCDNMF method in comparison to several state-of-the-art methods on six real world image datasets.

To better model the non-Gaussian heavy-tailed measurement noise with unknown and time-varying bias, a new Student's t-inverse-Wishart (STIW) distribution is presented. The STIW distribution is firstly written as a Gaussian, inverse-Wishart and normal-Gamma hierarchical form, from which a new robust Kalman filter is then derived based on the variational Bayesian method. Simulation results illustrate the potentials of the new derived robust Kalman filter for addressing the above measurement noise.

This article studies the mean and mean-square behaviors of the M-estimate based normalized subband adaptive filter algorithm (M-NSAF) with robustness against impulsive noise. Based on the contaminated-Gaussian noise model, the stability condition, transient and steady-state results of the algorithm are formulated analytically. These analysis results help us to better understand the M-NSAF performance in impulsive noise. To further obtain fast convergence and low steady-state estimation error, we derive a variable step size (VSS) M-NSAF algorithm. This VSS scheme is also generalized to the proportionate M-NSAF variant for sparse systems. Computer simulations on the system identification in impulsive noise and the acoustic echo cancellation with double-talk are performed to demonstrate our theoretical analysis and the effectiveness of the proposed algorithms.

As a robust nonlinear similarity measure, the maximum correntropy criterion (MCC) has been successfully applied to active noise control (ANC) for impulsive noise. The default kernel function of the filtered-x maximum correntropy criterion (FxMCC) algorithm is the Gaussian kernel, which is desirable in many cases for its smooth and strict positive-definite. However, it is not always the best choice. In this study, a filtered-x generalized maximum correntropy criterion (FxGMCC) algorithm is proposed, which adopts the generalized Gaussian density (GGD) function as its kernel. The FxGMCC algorithm has greater robust ability against non-Gaussian environments, but, it still adopts a single error norm which exhibits poor convergence rate and noise reduction performance. To surmount this problem, an improved FxGMCC (IFxGMCC) algorithm with continuous mixed Lp-norm is proposed. Moreover, to make a trade-off between fast convergence rate and low steady-state misalignment, a convexly combined IFxGMCC (C-IFxGMCC) algorithm is further developed. The stability mechanism and computational complexity of the proposed algorithms are analyzed. Simulation results in the context of different impulsive noises as well as the real noise signals verify that the proposed algorithms are superior to most of the existing robust adaptive algorithms.

Pedestrian detection with high detection and localization accuracy is increasingly important for many practical applications. Due to the flexible structure of the human body, it is hard to train a template-based pedestrian detector that achieves a high detection rate and a good localization accuracy simultaneously. In this paper, we utilize human pose estimation to improve the detection and localization accuracy of pedestrian detection. We design two kinds of pose-indexed features that can considerably improve the discriminability of the detector. In addition to employing a two-stage pipeline to carry out these two tasks, we unify pose estimation and pedestrian detection into a cascaded decision forest in which they can cooperate sufficiently. To prevent irregular positive examples, such as truncated ones, from distracting the pedestrian detection and the pose regression, we clean the positive training data by realigning the bounding boxes and rejecting the wrong positive samples. Experimental results on the Caltech test dataset demonstrate the effectiveness of our proposed method. Our detector achieves 11.1% MR-2, outperforming all existing detectors without using the convolutional neural network (CNN). Moreover, our method can be assembled with other detectors based on CNNs to improve detection and localization performance. By collaborating with the recent CNN-based method, our detector achieves 5.5% MR-2 on the Caltech test dataset, outperforming the state-of-the-art methods.

Recently, pedestrian detection has made significant advances benefiting from the region-based convolutional neural networks (R-CNN). However, training R-CNN with a holistic intersection over union (IoU) always brings many flawed positive samples. This paper introduces a strict matching metric, which is beneficial to selecting well-aligned positive samples. Specifically, this matching metric is defined on a set of region-IoUs instead of a holistic IoU, which considers the alignments of different part regions in a whole bounding box simultaneously. A positive sample matches a ground truth only if all its region-IoUs are bigger than a threshold. Secondly, an improved negative example selection strategy using both the classification and localization information is proposed to mine hard negative examples, which can further suppress the false positive detections near the pedestrians. Based on the proposed sample selection strategy, a cascade compact convolutional neural network (CC-CNN) is proposed for accurate pedestrian detection. Each stage of the CC-CNN is constructed with a compact network that only consists of a small number of parameters, thus making the detector suitable to be implemented on onboard embedded systems. Experimental results on two widely used pedestrian datasets demonstrate that the proposed training strategy and the CC-CNN based detector can effectively improve the detection rate and the localization accuracy using fewer parameters.

The distributions of input data are very important for learning machines, such as the convex universal learning machines (CULMs). The CULMs are a family of universal learning machines with convex optimization. However, the computational complexity is a crucial problem in CULMs, because the dimension of the nonlinear mapping layer (the hidden layer) of the CULMs is usually rather large in complex system modeling. In this article, we propose an efficient quantization method called Probability density Rank-based Quantization (PRQ) to decrease the computational complexity of CULMs. The PRQ ranks the data according to the estimated probability densities and then selects a subset whose elements are equally spaced in the ranked data sequence. We apply the PRQ to kernel ridge regression (KRR) and random Fourier feature recursive least squares (RFF-RLS), which are two typical algorithms of CULMs. The proposed method not only keeps the similarity of data distribution between the code book and data set but also reduces the computational cost by using the kd-tree. Meanwhile, for a given data set, the method yields deterministic quantization results, and it can also exclude the outliers and avoid too many borders in the code book. This brings great convenience to practical applications of the CULMs. The proposed PRQ is evaluated on several real-world benchmark data sets. Experimental results show satisfactory performance of PRQ compared with some state-of-the-art methods.

Concept factorization (CF) has shown its great advantage for both clustering and data representation and is particularly useful for image representation. Compared with nonnegative matrix factorization (NMF), CF can be applied to data containing negative values. However, the performance of CF method and its extensions will degenerate a lot due to the negative effects of outliers, and CF is an unsupervised method that cannot incorporate label information. In this article, we propose a novel CF method, with a novel model built based on the maximum correntropy criterion (MCC). In order to capture the local geometry information of data, our method integrates the robust adaptive embedding and CF into a unified framework. The label information is utilized in the adaptive learning process. Furthermore, an iterative strategy based on the accelerated block coordinate update is proposed. The convergence property of the proposed method is analyzed to ensure that the algorithm converges to a reliable solution. The experimental results on four real-world image data sets show that the new method can almost always filter out the negative effects of the outliers and outperform several state-of-the-art image representation methods.

Common spatial pattern (CSP) is a classic method commonly used in multichannel electroencephalogram (EEG) signal processing, which aims to extract effective features for binary classification by solving spatial filters that maximize the ratio of filtered dispersion between two classes. The aim of this paper is to improve the performance of the conventional CSP method, which will be badly influenced by noises. The recently proposed quantized minimum error entropy (QMEE) criterion is applied to structure a new objective function instead of the L2-norm in the conventional CSP. Quantization is utilized to reduce the computational complexity. The new objective function is optimized by a gradient-based iterative algorithm. The desirable performance of the QMEE-based CSP method, namely CSP-QMEE, is demonstrated with a toy example and two real EEG datasets, including Dataset IIb of the brain-computer interfaces (BCIs) Competition IV (three channels) and Dataset IIIa of the BCI Competition III (60 channels). The new method can achieve satisfactory performance compared to existing methods on all datasets. The promising results in this paper suggest that the CSP-QMEE may become a powerful tool for BCIs.

Recently, by employing the stacked extreme learning machine (ELM) based autoencoders (ELM-AE) and sparse AEs (SAE), multilayer ELM (ML-ELM) and hierarchical ELM (H-ELM) has been developed. Compared to the conventional stacked AEs, the ML-ELM and H-ELM usually achieve better generalization performance with a significantly reduced training time. However, the ℓ1-norm based SAE may suffer the overfitting problem and it is unable to provide analytical solution leading to long training time for big data. To alleviate these deficiencies, we propose an enhanced H-ELM (EH-ELM) with a novel random sparse matrix based AE (SMA) in this paper. The contributions are in two aspects, 1) utilizing the random sparse matrix, the sparse features can be obtained; 2) the proposed SMA can provide an analytical solution so that the high computational complexity issue in SAE can be addressed. Experimental results on benchmark datasets show that the proposed EH-ELM achieves a higher recognition rate and a faster training speed than H-ELM and ML-ELM.

Nonnegative matrix factorization (NMF) has been a state-of-the-art data representation method, since it contains the psychological and physiological evidence for parts-based representation in the human brain. However, many existing NMF methods fail to ensure the decomposed results to be sparse, or ignore some useful geometrical structure information in the data. In this paper, a sparse NMF method, called dual graph regularized nonnegative matrix factorization with l1-norm sparsity constraint (l1-DNMF) is proposed to solve the two problems together. In addition, to satisfy the locality condition and sparsity constraint simultaneously, we also propose the dual graph regularized nonnegative matrix factorization with local coordinate constraint (LDNMF). By using the multiplicative update algorithm to solve the optimization problems of l1-DNMF and LDNMF, we derive two efficient alternating iterative methods. Experimental results on four image datasets demonstrate the promising performance of the new methods compared with several related methods for clustering applications.

Linear regression model (LRM) based on mean square error (MSE) criterion is widely used in Granger causality analysis (GCA), which is the most commonly used method to detect the causality between a pair of time series. However, when signals are seriously contaminated by non-Gaussian noises, the LRM coefficients will be inaccurately identified. This may cause the GCA to detect a wrong causal relationship. Minimum error entropy (MEE) criterion can be used to replace the MSE criterion to deal with the non-Gaussian noises. But its calculation requires a double summation operation, which brings computational bottlenecks to GCA especially when sizes of the signals are large. To address the aforementioned problems, in this letter, we propose a new method called GCA based on the quantized MEE (QMEE) criterion (GCA-QMEE), in which the QMEE criterion is applied to identify the LRM coefficients and the quantized error entropy is used to calculate the causality indexes. Compared with the traditional GCA, the proposed GCA-QMEE not only makes the results more discriminative, but also more robust. Its computational complexity is also not high because of the quantization operation. Illustrative examples on synthetic and EEG datasets are provided to verify the desirable performance and the availability of the GCA-QMEE.

Correntropy is a local similarity measure defined in kernel space, and the maximum correntropy criterion (mcc) has been successfully applied in many areas of signal processing and machine learning in recent years. The kernel function in correntropy is usually restricted to the Gaussian function with the center located at zero. However, the zero-mean Gaussian function may not be a good choice for many practical applications. In this letter, we propose an extended version of correntropy, whose center can be located at any position. Accordingly, we propose a new optimization criterion called maximum correntropy criterion with variable center (MCC-VC). We also propose an efficient approach to optimize the kernel width and center location in the MCC-VC. Simulation results of regression with linear-in-parameter (LIP) models confirm the desirable performance of the new method.

In order to efficiently utilize the information in the data and eliminate the negative effects of outliers in the principal component analysis (PCA) method, in this paper, we propose a novel robust sparse PCA method based on maximum correntropy criterion (MCC) with high-order manifold constraints called the RHSPCA. Compared with the traditional PCA methods, the proposed RHSPCA has the following benefits: 1) the MCC regression term is more robust to outliers than the MSE-based regression term; 2) thanks to the high-order manifold constraints, the low-dimensional representations can preserve the local relations of the data and greatly improve the clustering and classification performance for image processing tasks; and 3) in order to further counteract the adverse effects of outliers, the MCC-based samples' mean is proposed to better centralize the data. We also propose a new solver based on the half-quadratic technique and accelerated block coordinate update strategy to solve the RHSPCA model. Extensive experimental results show that the proposed method can outperform the state-of-the-art robust PCA methods on a variety of image processing tasks, including reconstruction, clustering, and classification, on outliers contaminated datasets.

High-dimensional data contain not only redundancy but also noises produced by the sensors. These noises are usually non-Gaussian distributed. The metrics based on Euclidean distance are not suitable for these situations in general. In order to select the useful features and combat the adverse effects of the noises simultaneously, a robust sparse subspace learning method in unsupervised scenario is proposed in this paper based on the maximum correntropy criterion that shows strong robustness against outliers. Furthermore, an iterative strategy based on half quadratic and an accelerated block coordinate update is proposed. The convergence analysis of the proposed method is also carried out to ensure the convergence to a reliable solution. Extensive experiments are conducted on real-world data sets to show that the new method can filter out the outliers and outperform several state-of-the-art unsupervised feature selection methods.

In this paper, we propose a novel prognostic approach for remaining useful life estimation (RUL) with evolving joint prediction of continuous and discrete states which represent the signals and health states of systems respectively. The predictors are built with evolving capability of adapting structures and parameters online to capture the dynamic characteristics of systems during runtime. Moreover, the discrete states can be determined dynamically during the construction of the predictors for systems operating under different environments. In the testing phase, the optimum predictor for predicting continuous and discrete states jointly is chosen under the error and distance criteria. The RULs are estimated conveniently once the predicted signals fall into failure mode based on a distance metric. In order to validate the performance of the proposed approach, the widely used turbofan engine datasets are taken into consideration. Experimental results demonstrate the reasonability and superiority of the proposed approach compared to other approaches.

Due to the existence of various anomalies such as non-Gaussian process and measurement noises, gross measurement errors, and sudden changes of system status, the robust forecasting-aided state estimation is pivotal for power system stability. This paper develops a novel unscented Kalman filter (UKF) with the generalized correntropy loss (GCL) (termed as GCL-UKF) to estimate power system state with forecasting aid. The GCL is used to replace the mean square error loss in the original UKF framework. The advantage of such an approach is that it combines the strength of the GCL developed in robust information theoretic learning for addressing the non-Gaussian interference and the strength of the UKF in handling strong model nonlinearities. In addition, we take into account the nontrivial influences of the bad data for the innovation vector. An enhanced GCL-UKF method is established by introducing an exponential function of the innovation vector to adjust a covariance matrix so as to improve the GCL-UKF-based state estimation accuracy under the change of gain matrix caused by bad factors. Numerical simulation results carried out on IEEE 14-bus, 30-bus, and 57-bus test systems validate the efficacy of the proposed methods for state estimation under various types of measurement.

The extreme learning machine (ELM) has attracted much attention over the past decade due to its fast learning speed and convincing generalization performance. However, there still remains a practical issue to be approached when applying the ELM: the randomly generated hidden node parameters without tuning can lead to the hidden node outputs being nonuniformly distributed, thus giving rise to poor generalization performance. To address this deficiency, a novel activation function with an affine transformation (AT) on its input is introduced into the ELM, which leads to an improved ELM algorithm that is referred to as an AT-ELM in this paper. The scaling and translation parameters of the AT activation function are computed based on the maximum entropy principle in such a way that the hidden layer outputs approximately obey a uniform distribution. Application of the AT-ELM algorithm in nonlinear function regression shows its robustness to the range scaling of the network inputs. Experiments on nonlinear function regression, real-world data set classification, and benchmark image recognition demonstrate better performance for the AT-ELM compared with the original ELM, the regularized ELM, and the kernel ELM. Recognition results on benchmark image data sets also reveal that the AT-ELM outperforms several other state-of-the-art algorithms in general.

Comparing with traditional learning criteria, such as mean square error, the minimum error entropy (MEE) criterion is superior in nonlinear and non-Gaussian signal processing and machine learning. The argument of the logarithm in Renyi's entropy estimator, called information potential (IP), is a popular MEE cost in information theoretic learning. The computational complexity of IP is, however, quadratic in terms of sample number due to double summation. This creates the computational bottlenecks, especially for large-scale data sets. To address this problem, in this paper, we propose an efficient quantization approach to reduce the computational burden of IP, which decreases the complexity from O(N2) to O(MN) with M ≪ N. The new learning criterion is called the quantized MEE (QMEE). Some basic properties of QMEE are presented. Illustrative examples with linear-in-parameter models are provided to verify the excellent performance of QMEE.

Random feature kernel least mean square RF KLMS) algorithms, like the random Fourier feature KLMS (RFF-KLMS), can effectively reduce the computation and storage burdens of the KLMS algorithm in the process of update. However, little work has been done to perform the convergence analysis for such algorithms. To this end, in this paper, we present a unified framework of RF-KLMS algorithms, and based on which, a universal model for convergence analysis is given. As two examples, the RFF-KLMS and the random Gaussian feature KLMS (RGF-KLMS) are discussed detailedly. Simulations demonstrate the validity of the theoretical analysis.

The linear model uses the space defined by the input to project the target or desired signal and find the optimal set of model parameters. When the problem is nonlinear, the adaption requires nonlinear models for good performance, but it becomes slower and more cumbersome. In this paper, we propose a linear model called Augmented Space Linear Model (ASLM), which uses the full joint space of input and desired signal as the projection space and approaches the performance of nonlinear models. This new algorithm takes advantage of the linear solution, and corrects the estimate for the current testing phase input with the error assigned to the input space neighborhood in the training phase. This algorithm can solve the nonlinear problem with the computational efficiency of linear methods, which can be regarded as a trade off between accuracy and computational complexity. Making full use of the training data, the proposed augmented space model may provide a new way to improve many modeling tasks.

Generally, electroencephalogram (EEG) signals recorded from the brain computer interface (BCI) systems are very noisy and non-stationary, which may affect the online performance of classifiers established from the prior session heavily. In order to address such problems, the classifiers should also be capable of adapting the change of EEG automatically during the processing of evaluation. In this paper, we propose an incremental adaptive EEG classification scheme. In this scheme, an extended sequential adaptive fuzzy inference system (ESAFIS) is used to evolve its structure dynamically and adapts the classifier automatically online to address the non-stationarity of the EEG signals. ESAFIS is an evolving system, wherein the fuzzy rules are evolved based on the modified influence of the rule. This paper presents the classification of 2-class motor imagery EEG based on ESAFIS with adaptive strategy. Simulations are conducted based on two datasets: one is the BCI Competition IV dataset 2b and the other one is recorded from our own BCI experiments. Compared to other methods such as ELM and LDA, the simulation results demonstrate that the proposed scheme produces better classification results.

Traditional locality preserving projection (LPP) is an excellent linear dimensionality reduction method that can preserve the local structure of the data. The objective function of LPP is based on L2-norm criterion, which results in obvious sensitivity to the outliers. In order to solve this problem, researchers proposed some LPP variants based on the L1-norm (LPP-L1) and the maximum correntropy criterion (LPP-MCC). In this paper, we propose a more robust version of LPP, called LPP-KRSL, whose objective function is based on the kernel risk-sensitive loss (KRSL). The objective function can be efficiently solved via a half-quadratic optimization procedure. The experimental results on both synthetic and real-world data demonstrate that LPP-KRSL is more robust and effective than other LPP methods.

In this paper, we propose a robust method for non-linear system identification that incorporates robustness to echo state networks (ESNs). In particular, the ESNs utilize generalized correntropy as a loss function to get optimal solutions. Generalized correntropy is a more flexible extension of correntropy in information theoretic learning (ITL). Generalized correntropy induced metric (GCIM) is robust to outliers with a proper shape parameter. The ESNs with GCIM can provide the anti-noise capacity and are insensitive outliers which are prevalent in real-world tasks. They also inherit the basic architecture of echo state network but replaces the commonly used mean square error (MSE) criterion with GCIM. The stochastic gradient descent method is adopted to optimize the generalized correntropy-based cost function. Numerical simulations are given to show that the proposed algorithm is robust to the non-Gaussian noise and outliers.

Nonnegative matrix factorization (NMF) is a significant matrix decomposition technique for learning parts-based, linear representation of nonnegative data, which has been widely used in a broad range of practical applications such as document clustering, image clustering, face recognition and blind spectral unmixing. Traditional NMF methods, which mainly minimize the square of the Euclidean distance or the Kullback-Leibler (KL) divergence, seriously suffer the outliers and non-Gaussian noises. In this paper, we propose a robust sparse nonnegative matrix factorization algorithm, called l1-norm nonnegative matrix factorization based on maximum correntropy criterion (11-CNMF). Specifically, l1-CNMF is derived from the traditional NMF algorithm by incorporating the l1 sparsity constraint and maximum correntropy criterion. Numerical experiments on the Yale database and the ORL database with and without apparent outliers show the effectiveness of the proposed algorithm for image clustering compared with other existing related methods.

In this paper, we propose a robust point set registration algorithm which combines correntropy and point-to-plane distance, which can register rigid point sets with noises and outliers. Firstly, as correntropy performs well in handling data with non-Gaussian noises, we introduce it to model rigid point set registration problem based on point-to-plane distance; Secondly, we propose an iterative algorithm to solve this problem, which repeats to compute correspondence and transformation parameters respectively in closed form solutions. Simulated experimental results demonstrate the high precision and robustness of the proposed algorithm. In addition, LiDAR based localization experiments on automated vehicle performs satisfactory for localization accuracy and time consumption.

Iterative closest point (ICP) algorithm, as its accuracy and efficiency, is widely used in rigid registration. However, ICP algorithm is easily failed when point sets lack of structure variety, such as semicircles. To solve this problem, a precise point set registration method for RGB-D data is proposed. Firstly, the color information provides a new information for registration, and the correntropy is introduced to deal with the noises and outliers. With color assisted and correntropy, a more robust objective function is built. Secondly, a variant ICP algorithm is used to deal with optimization problem via multiple iterations. Finally, as shown in the experimental results and scene reconstruction, our method obtains more precise results than other ICP algorithms.

As an important problem in point set registration, partial registration has been solved by some variants of Iterative Closest Point (ICP) algorithm under good initial values. However, the initial parameters remained to be solved for partial registration. This paper presents a parameter initialization algorithm based on matching triangles for partial registration. Experimental results demonstrate that the proposed initialization method can find an appropriate initial transformation for next accurate registration, even the initial rotation angle between two sets is large. Based on the initialization of two point sets, the partial registration can be accomplished by auto trimmed ICP (ATICP) algorithm.

In order to improve the performances of recently presented improved normalised subband adaptive filter (INSAF) and proportionate INSAF algorithms for highly noisy system, this study proposes their set‐membership versions by exploiting the theory of set‐membership filtering. Apart from obtaining smaller steady‐state error, the proposed algorithms significantly reduce the overall computational complexity. In addition, to further improve the steady‐state performance for the algorithms, their smooth variants are developed by using the smoothed absolute subband output errors to update the step sizes. Simulation results in the context of acoustic echo cancellation have demonstrated the superiority of the proposed algorithms.

Human information processing depends mainly on billions of neurons which constitute a complex neural network, and the information is transmitted in the form of neural spikes. In this paper, we propose a spiking neural network (SNN), named MD-SNN, with three key features: (1) using receptive field to encode spike trains from images; (2) randomly selecting partial spikes as inputs for each neuron to approach the absolute refractory period of the neuron; (3) using groups of neurons to make decisions. We test MD-SNN on the MNIST data set of handwritten digits, and results demonstrate that: (1) Different sizes of receptive fields influence classification results significantly. (2) Considering the neuronal refractory period in the SNN model, increasing the number of neurons in the learning layer could greatly reduce the training time, effectively reduce the probability of over-fitting, and improve the accuracy by 8.77%. (3) Compared with other SNN methods, MD-SNN achieves a better classification; compared with the convolution neural network, MD-SNN maintains flip and rotation invariance (the accuracy can remain at 90.44% on the test set), and it is more suitable for small sample learning (the accuracy can reach 80.15% for 1000 training samples, which is 7.8 times that of CNN).

Correntropy is a second order statistical measure in kernel space, which has been successfully applied in robust learning and signal processing. In this paper, we define a nonsecond order statistical measure in kernel space, called the kernel mean- power error (KMPE), including the correntropic loss (C-Loss) as a special case. Some basic properties of KMPE are presented. In particular, we apply the KMPE to extreme learning machine (ELM) and principal component analysis (PCA), and develop two robust learning algorithms, namely ELM-KMPE and PCA-KMPE. Experimental results on synthetic and benchmark data show that the developed algorithms can achieve better performance when compared with some existing methods.

In this paper, a correntropy-based evolving fuzzy neural system (CEFNS) is proposed for approximation of nonlinear systems. Different from the commonly used mean-square error criterion, correntropy has a strong outliers rejection ability through capturing the higher moments of the error distribution. Considering the merits of correntropy, this paper brings contributions to build evolving fuzzy neural system (EFNS) based on the correntropy concept to achieve a more stable evolution of the rule base and update of the rule parameters instead of the commonly used mean-square error criterion. The correntropy-EFNS (CEFNS) begins with an empty rule base, and all rules are evolved online based on the correntropy criterion. The consequent part parameters are tuned based on the maximum correntropy criterion, where the correntropy is used as the cost function so as to improve the non-Gaussian noise rejection ability. The steady-state convergence performance of the CEFNS is studied through the calculation of the steady-state excess mean square error (EMSE) in two cases: Gaussian noise; and non-Gaussian noise. Finally, the CEFNS is validated through a benchmark system identification problem, a Mackey-Glass time series prediction problem as well as five other real-world benchmark regression problems under both noise-free and noisy conditions. Compared with other EFNSs, the simulation results show that the proposed CEFNS produces better approximation accuracy using the least number of rules and training time and also owns superior non-Gaussian noise handling capability.

The minimum error entropy (MEE) is an important and highly effective optimization criterion in information theoretic learning (ITL). For regression problems, MEE aims at minimizing the entropy of the prediction error such that the estimated model preserves the information of the data generating system as much as possible. In many real world applications, the MEE estimator can outperform significantly the well-known minimum mean square error (MMSE) estimator and show strong robustness to noises especially when data are contaminated by non-Gaussian (multimodal, heavy tailed, discrete valued, and so on) noises. In this brief, we present some theoretical results on the robustness of MEE. For a one-parameter linear errors-in-variables (EIV) model and under some conditions, we derive a region that contains the MEE solution, which suggests that the MEE estimate can be very close to the true value of the unknown parameter even in presence of arbitrarily large outliers in both input and output variables. Theoretical prediction is verified by an illustrative example.

This paper investigates the online recorded data-based composite neural control of uncertain strict-feedback systems using the backstepping framework. In each step of the virtual control design, neural network (NN) is employed for uncertainty approximation. In previous works, most designs are directly toward system stability ignoring the fact how the NN is working as an approximator. In this paper, to enhance the learning ability, a novel prediction error signal is constructed to provide additional correction information for NN weight update using online recorded data. In this way, the neural approximation precision is highly improved, and the convergence speed can be faster. Furthermore, the sliding mode differentiator is employed to approximate the derivative of the virtual control signal, and thus, the complex analysis of the backstepping design can be avoided. The closed-loop stability is rigorously established, and the boundedness of the tracking error can be guaranteed. Through simulation of hypersonic flight dynamics, the proposed approach exhibits better tracking performance.

In this paper, a novel quaternion adaptive filtering algorithm is proposed for a unified processing of 3D and 4D data, called quaternion least mean kurtosis (QLMK) algorithm. Multi-dimensional signals exhibit a complex nonlinear relationship and couple among different components. Considering that quaternion has huge advantage in terms of the representation of 3D and 4D signal, quaternion algebra is employed to derive the quaternion least mean square (QLMS) algorithm for hypercomplex signal processes. However, QLMS originates from the least mean square (LMS) algorithm, which may result in performance degradation when the signal is non-Gaussian. Due to the desirable performance of the least mean kurtosis (LMK) algorithm in non-Gaussian situation, in the present work we extend the original LMK algorithm to quaternion domain to manage the 3D and 4D signal processes. The analysis shows that QLMK provides a solution that is responsive to dynamically changing environments. Simulations on prediction of 4D Saito's chaotic circuit and 3D Lorenz attractor confirm the desirable performance of the proposed method.

A bias-compensated normalized least mean absolute deviation (NLMAD) algorithm is developed for system identification under impulsive output measurement noise and noisy input environment, which takes the advantage of the NLMAD to resist impulsive output noises. Considering biased estimation caused by the noisy input, we employ an unbiasedness criterion to obtain a bias-compensated vector for NLMAD algorithm which can compensate the bias available. Simulation results show that the proposed algorithm can achieve better steady-state performance than the least mean absolute deviation (LAD) and NLMAD when the system meets the a-stable output measurement noise and noisy input case.

In this paper, we investigate the convergence performance of a sparsified kernel least mean square (KLMS) algorithm in which the input is added into the dictionary only when the prediction error in amplitude is larger than a preset threshold. Under certain conditions, we derive an approximate value of the steady-state excess mean square error (EMSE). Simulation results confirm the theoretical predictions and provide some interesting findings, showing that the sparsification can not only be used to constrain the network size (hence reduce the computational burden) but also be used to improve the steady-state performance in some cases.

Kernel least mean square (KLMS) algorithm has been successfully applied in fields of adaptive filtering and online learning due to their ability to solve sequentially nonlinear problems by implicitly mapping the input signal to a high-dimensional reproducing kernel Hilbert space (RKHS). In this paper, we propose a novel adaptive algorithm called KLMS based on conjugate gradient (KLMS-CG), which uses the orthogonal search directions, instead of using the traditional steepest descent approach, to improve the convergence speed. Further, the quantized KLMS based on conjugate gradient (QKLMS-CG) is proposed to curb the growth of network. Simulation results indicate that the new algorithm can converge faster than the original KLMS while maintaining excellent accuracy.

In this paper, we investigate the nonlinear, finite dimensional and data independent random Fourier feature expansions that can approximate the popular Gaussian kernel. With recursive least squares algorithm, we develop the Random Fourier Feature Recursive Least Squares algorithm (RFF-RLS), which shows significant performance improvements in simulations when compared with several other online kernel learning algorithms such as Kernel Least Mean Square (KLMS) and Kerne Recursive Least Squares (KRLS). Our results confirm that the RFF-RLS can achieve desirable performance with low computational cost. As for the random Fourier features, the randomization generally results in redundancy. We use an algorithm, namely, Vector Quantization with Information Theoretic Learning (VQIT) to decrease the dictionary size. The resulting sparse dictionary can match the original data distribution well. The RFF-RLS with VQIT can outperform the RFF-RLS without VQIT.

In this paper, we propose a novel point set matching algorithm to improve the matching precision in the presence of non-Gaussian noises and outliers. In our method, a non-second order similarity measure known as Kernel Mean p-Power Error (KMPE) loss is employed as the matching cost function. We introduce a local optimal solution for computing the rigid transform by repeating the correspondence estimation and parameter updating processes. This new algorithm assigns a non-linear distance evaluation in kernel space according to the current estimation of the correspondence to yield a more accurate matching result between two point sets in practice. Experimental results demonstrate that our algorithm is more robust and accurate than the traditional ICP and the state-of-the-art algorithms.

The long-term goal of artificial intelligence (AI) is to make machines learn and think like human beings. Due to the high levels of uncertainty and vulnerability in human life and the open-ended nature of problems that humans are facing, no matter how intelligent machines are, they are unable to completely replace humans. Therefore, it is necessary to introduce human cognitive capabilities or human-like cognitive models into AI systems to develop a new form of AI, that is, hybrid-augmented intelligence. This form of AI or machine intelligence is a feasible and important developing model. Hybrid-augmented intelligence can be divided into two basic models: one is human-in-the-loop augmented intelligence with human-computer collaboration, and the other is cognitive computing based augmented intelligence, in which a cognitive model is embedded in the machine learning system. This survey describes a basic framework for human-computer collaborative hybrid-augmented intelligence, and the basic elements of hybrid-augmented intelligence based on cognitive computing. These elements include intuitive reasoning, causal models, evolution of memory and knowledge, especially the role and basic principles of intuitive reasoning for complex problem solving, and the cognitive learning framework for visual scene understanding based on memory and reasoning. Several typical applications of hybrid-augmented intelligence in related fields are given.

Constrained adaptive filtering algorithms have been extensively studied in many applications. Most existing constrained adaptive filtering algorithms are developed under the mean square error (MSE) criterion, which is an ideal optimality criterion under Gaussian noises. This assumption however fails to model the behavior of non-Gaussian noises found in practice. Motivated by the robustness and simplicity of maximum correntropy criterion (MCC) for non-Gaussian impulsive noises, this paper proposes a new adaptive filtering algorithm called constrained maximum correntropy criterion (CMCC). Specifically, CMCC incorporates a linear constraint into a MCC filter to solve a constrained optimization problem explicitly. The proposed adaptive filtering algorithm is easy to implement, has low computational complexity, and can significantly outperform those MSE based constrained adaptive algorithms in heavy-tailed impulsive noises. Additionally, the mean square convergence behaviors are studied under energy conservation relation, and a sufficient condition to ensure the mean square convergence and the steady-state mean square deviation (MSD) of the CMCC algorithm are obtained. Simulation results confirm the theoretical predictions under both Gaussian and non-Gaussian noises, and demonstrate the excellent performance of the novel algorithm by comparing it with other conventional methods.

In hyperspectral images, some spectral bands suffer from low signal-to-noise ratio due to noisy acquisition and atmospheric effects, thus requiring robust techniques for the unmixing problem. This paper presents a robust supervised spectral unmixing approach for hyperspectral images. The robustness is achieved by writing the unmixing problem as the maximization of the correntropy criterion subject to the most commonly used constraints. Two unmixing problems are derived: the first problem considers the fully constrained unmixing, with both the nonnegativity and sum-to-one constraints, while the second one deals with the nonnegativity and the sparsity promoting of the abundances. The corresponding optimization problems are solved using an alternating direction method of multipliers (ADMM) approach. Experiments on synthetic and real hyperspectral images validate the performance of the proposed algorithms for different scenarios, demonstrating that the correntropy-based unmixing with ADMM is particularly robust against highly noisy outlier bands.

Based on the knowledge that input data distribution is important for learning, a data density-dependent quantization scheme (DQS) is proposed for sparse input data representation. The usefulness of the representation scheme is demonstrated by using it as a data preprocessing unit attached to the well-known least squares support vector machine (LS-SVM) for application on big data sets. Essentially, the proposed DQS adopts a single shrinkage threshold to obtain a simple quantization scheme, which adapts its outputs to input data density. With this quantization scheme, a large data set is quantized to a small subset where considerable sample size reduction is generally obtained. In particular, the sample size reduction can save significant computational cost when using the quantized subset for feature approximation via the Nyström method. Based on the quantized subset, the approximated features are incorporated into LS-SVM to develop a data density-dependent quantized LS-SVM (DQLS-SVM), where an analytic solution is obtained in the primal solution space. The developed DQLS-SVM is evaluated on synthetic and benchmark data with particular emphasis on large data sets. Extensive experimental results show that the learning machine incorporating DQS attains not only high computational efficiency but also good generalization performance.

This paper investigates the disturbance observer-based composite fuzzy control of a class of uncertain nonlinear systems with unknown dead zone. With fuzzy logic system approximating the unknown nonlinearities, composite learning is constructed on the basis of a serial–parallel identifier. By introducing the intermediate signal, the disturbance observer is developed to provide efficient learning of the compounded disturbance which includes the effect of time-varying disturbance, fuzzy approximation error, and unknown dead zone. Based on the disturbance estimation and fuzzy approximation, the adaptive fuzzy controller is synthesized with novel updating law. The stability analysis of the closed-loop system is rigorously established via Lyapunov approach. The performance of the proposed controller is verified via simulation that faster convergence and higher precision are obtained.

Currently, colorectal cancer (CRC) already becomes one of the most common cancers worldwide. Though the prognosis of CRC patients is dramatically improved due to the new advanced treatments and medical improvements, the 5-year survival rate for the CRC patient is still low. Thus, we hypothesize that CRC may result from the complicated reasons related to both genetic and environmental factors. For this reason, this study collects such big CRC data with information of genetic variations and environmental exposure for the CRC patients and cancer-free controls that are employed to train and test the predictive CRC model. Our results demonstrate that (1) the explored genetic and environmental biomarkers are validated to cause the CRC by the manually reviewed experimental evidences, (2) the model can efficiently predict the risk of CRC after parameter optimization by the big CRC-related data, (3) our innovated generalized kernel recursive maximum correntropy(GKRMC) algorithm has high predictive power. Finally, we discuss why the GKRMC can outperform the classical regression algorithms and the related future study.

Pose variation remains one of the major factors that adversely affect the accuracy of person re-identification. Such variation is not arbitrary as body parts (e.g. head, torso, legs) have relative stable spatial distribution. Breaking down the variability of global appearance regarding the spatial distribution potentially benefits the person matching. We therefore learn a novel similarity function, which consists of multiple sub-similarity measurements with each taking in charge of a subregion. In particular, we take advantage of the recently proposed polynomial feature map to describe the matching within each subregion, and inject all the feature maps into a unified framework. The framework not only outputs similarity measurements for different regions, but also makes a better consistency among them. Our framework can collaborate local similarities as well as global similarity to exploit their complementary strength. It is flexible to incorporate multiple visual cues to further elevate the performance. In experiments, we analyze the effectiveness of the major components. The results on four datasets show significant and consistent improvements over the state-of-the-art methods.

Independent component analysis (ICA) by an information measure has seen wide applications in engineering. Different from traditional probability density function based information measures, a probability survival distribution based Cauchy-Schwartz information measure for multiple variables is proposed in this paper. Empirical estimation of survival distribution is parameter-free which is inherited by the estimation of the new information measure. This measure is proved to be a valid statistical independence measure and is adopted as an objective function to develop an ICA algorithm which is validated by an experiment. This work shows promising potential regarding the use of survival distribution based information measure for ICA.

The kernel least mean square (KLMS) algorithm is an efficient non-linear adaptive filter that operates in the reproducing kernel Hilbert space (RKHS). In realistic applications of system identification or time series prediction, there are usually multiple inputs that demand multiple kernels or kernel parameters. This paper proposes to use a tensor product kernel for KLMS that accommodates multiple inputs. Furthermore, instead of arbitrarily setting kernel parameters, appropriate kernel sizes can be chosen by a gradient descent based adaptive algorithm that minimizes the square of instant error, which helps KLMS to better capture the underlying system mechanism. Effectiveness of the proposed algorithm is shown by experiments conducted for both simulated dataset and an important real-world problem - Beijing PM2.5 prediction.

Owing to their universal approximation capability and online learning manner, kernel adaptive filters have been widely used in nonlinear systems modeling. Under Gaussian assumption, traditional kernel adaptive algorithms utilize the well-known mean square error(MSE) as a cost function to get optimal solutions. For non-Gaussian situations, MSE will not properly represent the statistics of the error, and hence degrade the performance. In recent years, an information theoretic learning(ITL) based criterion called Maximum Correntropy Criterion(MCC) has been proposed and applied in robust adaptive filtering. The correntropy is a generalized correlation measure in kernel space, which uses Gaussian kernel as a default kernel function. Of course, Gaussian kernel is not always the best choice. Recently, a more flexible definition of correntropy, called generalized correntropy, has been proposed. With a proper shape parameter, the generalized correntropy may get better performance than original correntropy with Gaussian kernel. In this paper, we take advantages of both kernel methods and generalized correntropy to develop a new kernel adaptive algorithm called Generalized Kernel Maximum Correntropy(GKMC) algorithm. We analyze theoretically the stability and steady-state performance of the new algorithm. In addition, we propose a Quantized GKMC(QGKMC) algorithm to curb the growth of the network size in GKMC while maintaining the performance. Simulation results confirm the theoretical expectations and show superior performance compared with existing methods.

As a nonlinear extension of Kalman filter, the extended Kalman filter (EKF) is also based on the minimum mean square error (MMSE) criterion. In general, the EKF performs well in Gaussian noises. But its performance may deteriorate substantially when the system is disturbed by heavy-tailed impulsive noises. In order to improve the robustness of EKF against impulsive noises, a new filter for nonlinear systems is proposed in this paper, namely the maximum correntropy extended Kalman filter (MCEKF), which adopts the maximum correntropy criterion (MCC) as the optimization criterion instead of using the MMSE. In MCEKF, the state mean and covariance matrix propagation equation are used to obtain a prior estimation of the state and covariance matrix, and then a fixed-point algorithm is used to update the posterior estimates. The robustness of the new filter is confirmed by simulation results.

Survival information potential (SIP) is defined by the survival distribution function instead of the probability density function (PDF) of a random variable. SIP can be used as a risk function equipped with learning error compensation ability while this SIP based risk function does not involve the estimation of PDF. This is desirable for a robust learning application in view of the error compensation ability. The learning error compensation scheme provided by SIP requires rank information of learning errors. The accuracy of error compensation desires a large number of input data but is computationally expensive. It is shown that the error compensation can be approximated by an error-related distribution. Based on this approximation, a dictionary based error compensation scheme is proposed to obtain a fixed-budget recursive online learning method. This proposed method is compared with several well-known online learning methods including least-mean-square method, least absolute deviation method, affine projection algorithm, recursive least-mean-square method, and sliding window based SIP method. Simulation results validate the outstanding smooth and consistent convergence performance of the proposed method particularly in α-stable-noise environments.

Kernel least mean square is a simple and effective adaptive algorithm, but dragged by its unlimited growing network size. Many schemes have been proposed to reduce the network size, but few takes the distribution of the input data into account. Input data distribution is generally important in view of both model sparsification and generalization performance promotion. In this paper, we introduce an online density-dependent vector quantization scheme, which adopts a shrinkage threshold to adapt its output to the input data distribution. This scheme is then incorporated into the quantized kernel least mean square (QKLMS) to develop a density-dependent QKLMS (DQKLMS). Experiments on static function estimation and short-term chaotic time series prediction are presented to demonstrate the desirable performance of DQKLMS.

This paper proposes an obstacle avoidance method for navigation of a mobile robot in uncertain environments based on a novel neural learning algorithm, namely least mean p-norm extreme learning machine (LMP-ELM) and Q-learning. The proposed obstacle avoidance method comprises of two behavior modules, Viz., an avoidance behavior and goal-seeking behavior. At the learning phase, the two modules are independently designed using the proposed LMP-ELM and Q-Learning. And then they are combined to navigate the mobile to the goal position without colliding with obstacles based on a switching function at the running phase. The LMP-ELM is used to realize the state-action mapping of the Q-learning. In the novel LMP-ELM, the computationally simple extreme learning machine architecture is maintained but a novel error criterion, namely the least mean p-power (LMP) error criterion provides a mechanism to update the output weights sequentially. The LMP error criterion aims to minimize the mean p-power of the error that is the generalization of the mean square error criterion used in the ELM. The effectiveness of the proposed method is verified by a series of simulations.

The correntropy provides a robust criterion for outlier-insensitive machine learning, and its maximisation has been increasingly investigated in signal and image processing. In this paper, we investigate the problem of unmixing hyperspectral images, namely decomposing each pixel/spectrum of a given image as a linear combination of other pixels/spectra called endmembers. The coefficients of the combination need to be estimated subject to the nonnegativity and the sum-to-one constraints. In practice, some spectral bands suffer from low signal-to-noise ratio due to acquisition noise and atmospheric effects, thus requiring robust techniques for the unmixing problem. In this work, we cast the unmixing problem as the maximization of a correntropy criterion, and provide a relevant solution using the alternating direction method of multipliers (ADMM) method. Finally, the relevance of the proposed approach is validated on synthetic and real hyperspectral images, demonstrating that the correntropy-based unmixing is robust to outlier bands.

The combination of diffusion strategies and least-mean-square (LMS) algorithm provides many advantages for adaptive-filter to solve distributed optimization, estimation and inference problems. However, suffering from high computation complexity, software implementation of diffusion LMS algorithm is unsuitable for real-time and portable applications. In order to extend its availability, we design a reconfigurable parallel FPG accelerator by exploring multiple dimensions of parallelism, including: parallel execution of agents state updating, data combining, data training and multi-stages pipeline to speedup the execution time. The accelerator for networks with various number of agents and different input dimensions is implemented. Results demonstrate that, it can achieve a speedup of three orders of magnitude at 100Mhz compared with C implementation for a 32-nodes network with 16-dimensional input-data.

The authors consider the joint admission and power control problem in a dense small cell network, which contains multiple interference links. The goal is to mainly maximise the number of the admitted links, and at the same time minimise the transmit power. The authors formulate the admission control and power control problem as a joint optimisation problem, which is however non‐deterministic polynomial hard (NP‐hard). Such NP‐hard problem can be relaxed to a p ‐norm problem (0 < p < 1) by using the correntropy induced metric. The correntropy is a novel non‐linear similarity measure, which has been successfully used in the robust and spares signal processing, especially when the data contain large outliers. Thus, in this work the authors propose a new correntropy induced joint power and admission control algorithm. To achieve a faster convergence speed, the authors also propose an adaptive kernel size method, in which the kernel size is determined by the error so that the convergence speed is the fastest during the iterations. Simulation results show that the proposed approach can achieve much better results than the existing works.

This paper develops a novel decentralized dimensionality reduction algorithm for the distributed tensor data across sensor networks. The main contributions of this paper are as follows. First, conventional centralized methods, which utilize entire data to simultaneously determine all the vectors of the projection matrix along each tensor mode, are not suitable for the network environment. Here, we relax the simultaneous processing manner into the one-vector-by-one-vector (OVBOV) manner, i.e., determining the projection vectors (PVs) related to each tensor mode one by one. Second, we prove that in the OVBOV manner each PV can be determined without modifying any tensor data, which simplifies corresponding computations. Third, we cast the decentralized PV determination problem as a set of subproblems with consensus constraints, so that it can be solved in the network environment only by local computations and information communications among neighboring nodes. Fourth, we introduce the null space and transform the PV determination problem with complex orthogonality constraints into an equivalent hidden convex one without any orthogonality constraint, which can be solved by the Lagrange multiplier method. Finally, experimental results are given to show that the proposed algorithm is an effective dimensionality reduction scheme for the distributed tensor data across the sensor networks.

This paper proposes a 128-way parallel FPGA platform to accelerate the kernel least mean square (KLMS) algorithm. With the adoption of a quantized method and pipeline technology, this platform which works at 200MHz is 4827 times faster, on average, than the Matlab code running on a 3GHz Intel(R) Core(TM) i5-2320 CPU.

In this paper, we implement a reconfigurable FPGA accelerator which could compute multiscale permutation entropy for 128 EEG signals simultaneously in real time. When it works at 150MHz and the window size is 256, compared with C code running on a 3GHz Intel(R) Core(TM) i5-2320 CPU, the average speedup is 3748.

Sparse estimation has received a lot of attention due to its broad applicability. In sparse channel estimat ion, the parameter vector with sparsity characteristic can be well estimated from noisy measurements through sparse adaptive filters. In previous studies, most works use the mean square error (MSE) based cost to develop sparse filters, which is rat ional under the assumption of Gaussian distributions. However, Gaussian assumption does not always hold in real-world environments. To address this issue, we incorporate in this work l1-norm and reweighted l1-norm into the minimum error entropy (MEE) criterion to develop new sparse adaptive filters, which may perform much better than the MSE based methods especially in non-Gaussian situations, since the error entropy can capture higher-order statistics of the errors . Furthermore, a new approximator of l0-norm based on the Correntropy Induced Metric (CIM) is also used as a sparsity penalty term (SPT). Simulation results show the excellent performance of the proposed algorithms.

Color composition is an important property for many computer vision tasks like image retrieval and object classification. In this paper we address the problem of inferring the color composition of the intrinsic reflectance of objects, where the shadows and highlights may change the observed color dramatically. We achieve this through color label propagation without recovering the intrinsic reflectance beforehand. Specifically, the color labels are propagated between regions sharing the same reflectance, and the direction of propagation is promoted to be from regions under full illumination and normal view angles to abnormal regions. We detect shadowed and highlighted regions as well as pairs of regions that have similar reflectance. A joint inference process is adopted to trim the inconsistent identities and connections. For evaluation we collect three datasets of images under noticeable highlights and shadows. Experimental results show that our model can effectively describe the color composition of real-world images.

As a robust measure of similarity, C-Loss can be successfully used for data fitting such as regression and classification, especially when data contain large outliers. In this paper, we propose a modified C-Loss function, called exponential C-Loss (EC-Loss), which is defined as an exponential function of the C-Loss. The EC-Loss inherits the robustness and smoothness of the C-Loss but may have a better performance surface that favors the usage of a gradient-based learning algorithm, particularly at a region far from the optimal solution. In order to avoid the flatness of the performance surface near the optimal solution and obtain a fast convergence speed during the overall adaptation process, we also propose a novel switching strategy between C-Loss and EC-Loss. A simple simulation example is presented to demonstrate the performance surface and desirable performance of the new method.

The mean square convergence of the kernel least mean square (KLMS) algorithm has been studied in a recent paper [B. Chen, S. Zhao, P. Zhu, J. C. Principe, Mean square convergence analysis of the kernel least mean square algorithm, Signal Processing, vol. 92, pp. 2624-2632, 2012]. In this paper, we continue this study and focus mainly on the initial convergence behavior. Two measures of the convergence performance are considered, namely the weight error power (WEP) and excess mean square error (EMSE). The analytical expressions of the initial decreases of the WEP and EMSE are derived, and several interesting facts about the initial convergence are presented. An illustration example is given to support our observation.

Correntropy, a novel localized similarity measure defined in kernel space, has been successfully used as a cost function in adaptive system training. The adaptive algorithms under the maximum correntropy criterion (MCC) have been shown to be robust to impulsive non-Gaussian noises. However, they may converge slowly especially at a region far from the optimal solution. In this paper, we propose a new MCC algorithm with a variable step-size (VSS) called the VSS-MCC algorithm, which may achieve a much faster convergence speed while maintaining similar steady-state performance. In the new algorithm, the step-size is updated based on an approximation for the curvature of performance surface. Simulation results demonstrate the superior performance of VSS-MCC compared with the original MCC algorithm.

Correntropy has been successfully applied in non-Gaussian signal processing, but the superior performance achieved is depends on appropriate selection of the kernel width. How to select a proper kernel width is a crucial problem in correntropy applications. In this paper, we propose an adaptive algorithm to update the kernel width, which is set at a maximum between the absolute value of instantaneous error divided by square root of 2 and a predetermined kernel width. The new algorithm involves no extra free parameters and keeps the simplicity and robustness of the original maximum correntropy criterion (MCC) algorithm. Simulation results confirm that the proposed algorithm can achieve excellent performance in channel estimation under impulsive noises.

fMRI has been a popular way for encoding and decoding human visual cortex activity. A previous research reconstructed binary image using a sparse logistic regression (SLR) with fMRI activity patterns as its input. In this article, based on SLR, we propose a new sparse logistic regression with a tunable regularization parameter (SLR-T), which includes the SLR and maximum likelihood regression (MLR) as two special cases. By choosing a proper regularization parameter in SLR-T, it may yield a better performance than both SLR and MLR. An fMRI visual image reconstruction experiment is carried out to verify the performance of SLR-T.

The maximum correntropy criterion (MCC) has received increasing attention in signal processing and machine learning due to its robustness against outliers (or impulsive noises). Some gradient based adaptive filtering algorithms under MCC have been developed and available for practical use. The fixed-point algorithms under MCC are, however, seldom studied. In particular, too little attention has been paid to the convergence issue of the fixed-point MCC algorithms. In this letter, we will study this problem and give a sufficient condition to guarantee the convergence of a fixed-point MCC algorithm.

A weighted least squares scheme based on an empirical survival error potential function is proposed in this paper. The empirical survival error potential function provides an error compensation scheme for noise distributions far from being Gaussian. This error compensation procedure is efficiently implemented via a weighted least squares formulation where an analytical solution form is obtained. The performance of the developed scheme is extensively tested on 16 benchmark data sets where the results show promising potential of the proposed empirical survival error distribution compensation scheme for binary pattern classification.

In this work, we put forward a new adaptation criterion, namely the hybrid criterion (HC), which is a mixture of the traditional mean square error (MSE) and the maximum correntropy criterion (MCC). The HC criterion is developed from the viewpoint of the least trimmed squares (LTS) estimator, a high breakdown estimator that can avoid undue influence from outliers. In the LTS estimator, the data are divided (by ranking) into two categories: the normal data and the outliers, and the outlier data are purely discarded. In order to improve the robustness of the LTS, some data with large values, which may contain some useful information, are also thrown away. Instead of purely throwing away those data, the new criterion applies the robust MCC criterion on the large data, and hence can efficiently utilize them to further improve the performance. We apply the HC criterion to adaptive filtering and develop the hybrid affine projection algorithm (HAPA) and kernel hybrid affine projection algorithm (KHAPA). Simulation results show that the proposed algorithms perform very well.

Feature selection techniques play a crucial role in machine learning tasks such as regression and classification. Many filter methods of feature selection are based on the mutual information (e.g. MIFS, MIFS-U, NMIFS, and mRMR methods). In this work, a new mutual information is defined based on the cross survival information potential (CSIP) and Cauchy-Schwartz divergence (CSD), called the survival Cauchy-Schwartz mutual information (SCS-MI). We apply this new mutual information to select an informative subset of features for a SVM classifier. Experimental results illustrate the desirable performance of the new method.

A new kernel adaptive filtering (KAF) algorithm, namely the sparse kernel recursive least squares (SKRLS), is derived by adding a ℓ1-norm penalty on the center coefficients to the least squares (LS) cost (i.e. the sum of the squared errors). In each iteration, the center coefficients are updated by a fixed-point sub-iteration. Compared with the original KRLS algorithm, the proposed algorithm can produce a much sparser network, in which many coefficients are negligibly small. A much more compact structure can thus be achieved by pruning these negligible centers. Simulation results show that the SKRLS performs very well, yielding a very sparse network while preserving a desirable performance.

The least trimmed squares (LTS) estimator is a robust estimator as it can avoid undue influence from outliers. The exact solution of the LTS estimation is however hard to And and if the number of data is large then the method is unfeasible. In this work, we apply the LTS criterion to adaptive Altering and develop the trimmed affine projection algorithm (TAPA) and kernel trimmed affine projection algorithm (KTAPA). The proposed adaptive algorithms are very robust to outliers and have low computational complexity. Simulation results conflrm their excellent and robust performance.

Kernel methods are powerful for developing a nonlinear learning algorithm in a high-dimensional linear space. The least mean square (LMS) and the least absolute deviation (LAD) are two well-known linear adaptive filtering algorithms. The former performs very well when the noise is Gaussian, while the later possesses desirable performance when the noise has a long-tailed distribution (e.g. alpha-stable distribution). The combination of the LMS and LAD yields a robust mixed-norm (RMN) algorithm. In this paper, we combine the popular kernel methods and the RMN algorithm to develop a new kernel adaptive filtering algorithm, namely the kernel RMN (KRMN) algorithm, which is a robust adaptive algorithm in reproducing kernel Hilbert space (RIOTS). The mean square convergence is analyzed, and the excellent and robust performance of the new algorithm is demonstrated by the simulation results of nonlinear time series prediction.

Fast and accurate machine learning algorithms are needed in many physical applications. However, the learning efficiency is badly subjected to the intensive computation. Knowing that hardware implementation could speed up computation effectively, we use a FPGA hardware platform to implement an on-line kernel learning algorithm, namely the kernel least mean square (KLMS) which adopts the simple survival kernel as the Mercer kernel. By using an on-line quantization method and pipeline technology, the requirement of hardware resources and computation burden can be reduced significantly and the data processing speed can be accelerated apparently without losing accuracy. Finally, a 128-way parallel FPGA platform which works at 200MHz is implemented. It could achieve an average speedup of 6553 versus Matlab running on a 3GHz Intel(R) Core(TM) i5-2320 CPU.

The steady-state excess mean square error (EMSE) of the adaptive filtering under the maximum correntropy criterion (MCC) has been studied. For Gaussian noise case, we establish a fixed-point equation to solve the exact value of the steady-state EMSE, while for non-Gaussian noise case, we derive an approximate analytical expression for the steady-state EMSE, based on a Taylor expansion approach. Simulation results agree with the theoretical calculations quite well.

In this paper, we define a new Mercer kernel, namely survival kernel, which is closely related to our recently proposed survival information potential (SIP). The new kernel function is parameter free, simple in calculation, and strictly positive-definite (SPD) over ℝm+, hence it has potential utility in machine learning especially in online kernel learning. In this work we apply the survival kernel to kernel adaptive filtering, in particular the kernel least mean square (KLMS) algorithm. Simulation results show that KLMS with survival kernel may achieve satisfactory performance with little computational time and without the choice of free parameters.

Since its introduction, kernel adaptive filtering (KAF) has attracted considerable attention in recent years. Its main advantages include universal nonlinear approximation using kernel methods, linearity with convex learning in the Reproducing Kernel Hilbert Space (RKHS), and online adaptation with moderate complexity. Among its applications, the kernel least mean square (KLMS) algorithm deserves particular attention due to its simplicity and effectiveness for learning complex systems. A major drawback of current implementations of KAF is the lack of a simple determination of the certainty of each estimate. In this paper, we present a novel kernel adaptive filtering architecture with confidence intervals. By introducing an auxiliary filter, the variance of each estimate can be computed using stochastic gradient descent in O(N). Results show that the proposed algorithm produces comparable estimates of the mean and the variance functions, using only a fraction of the computation associated with the Gaussian process (GP) prediction, and is more versatile in the cases of time-varying noise variance or heteroskedasticity.

In a recent paper, we developed a novel quantized kernel least mean square algorithm, in which the input space is quantized (partitioned into smaller regions) and the network size is upper bounded by the quantization codebook size (number of the regions). In this paper, we propose the quantized kernel least squares regression, and derive the optimal solution. By incorporating a simple online vector quantization method, we derive a recursive algorithm to update the solution, namely the quantized kernel recursive least squares algorithm. The good performance of the new algorithm is demonstrated by Monte Carlo simulations.

In a recent work, we have proposed the quantized kernel least mean square (QKLMS) algorithm, which is quite effective in online learning sequentially a nonlinear mapping with a slowly growing radial basis function (RBF) structure. In this paper, in order to further reduce the network size, we propose a sparse QKLMS algorithm, which is derived by adding a sparsity inducing l 1 norm penalty of the coefficients to the squared error cost. Simulation examples show that the new algorithm works efficiently, and results in a much sparser network while preserving a desirable performance.

Correntropy, as an adaptive criterion of Information Theoretic Learning (ITL), has been successfully used in signal processing and machine learning. How to appropriately select the kernel width of correntropy is a crucial problem in correntropy applications. Existing kernel width selection methods are not suitable enough for this problem. In this paper, we develop an adaptive method for kernel width selection in correntropy. Based on the Middleton's non-Gaussian models, this method utilizes the kurtosis as a ratio to adjust the standard deviation of the prediction error to obtain the kernel width online. The superior performance of the new method has been demonstrated by simulation examples in the noisy frequency doubling and echo cancelation problems.

As a new measure of similarity, the correntropy can be used as an objective function for many applications. In this letter, we study Bayesian estimation under maximum correntropy (MC) criterion. We show that the MC estimation is, in essence, a smoothed maximum a posteriori (MAP) estimation, including the MAP and the minimum mean square error (MMSE) estimation as the extreme cases. We also prove that under a certain condition, when the kernel size in correntropy is larger than some value, the MC estimation will have a unique optimal solution lying in a strictly concave region of the smoothed posterior distribution.

In this paper, we propose a quantization approach, as an alternative of sparsification, to curb the growth of the radial basis function structure in kernel adaptive filtering. The basic idea behind this method is to quantize and hence compress the input (or feature) space. Different from sparsification, the new approach uses the "redundant" data to update the coefficient of the closest center. In particular, a quantized kernel least mean square (QKLMS) algorithm is developed, which is based on a simple online vector quantization method. The analytical study of the mean square convergence has been carried out. The energy conservation relation for QKLMS is established, and on this basis we arrive at a sufficient condition for mean square convergence, and a lower and upper bound on the theoretical value of the steady-state excess mean square error. Static function estimation and short-term chaotic time-series prediction examples are presented to demonstrate the excellent performance.

Kernel adaptive filters have drawn increasing attention due to their advantages such as universal nonlinear approximation with universal kernels, linearity and convexity in Reproducing Kernel Hilbert Space (RKHS). Among them, the kernel least mean square (KLMS) algorithm deserves particular attention because of its simplicity and sequential learning approach. Similar to most conventional adaptive filtering algorithms, the KLMS adopts the mean square error (MSE) as the adaptation cost. However, the mere second-order statistics is often not suitable for nonlinear and non-Gaussian situations. Therefore, various non-MSE criteria, which involve higher-order statistics, have received an increasing interest. Recently, the correntropy, as an alternative of MSE, has been successfully used in nonlinear and non-Gaussian signal processing and machine learning domains. This fact motivates us in this paper to develop a new kernel adaptive algorithm, called the kernel maximum correntropy (KMC), which combines the advantages of the KLMS and maximum correntropy criterion (MCC). We also study its convergence and self-regularization properties by using the energy conservation relation. The superior performance of the new algorithm has been demonstrated by simulation experiments in the noisy frequency doubling problem.

In this paper, a novel methodology is proposed to solve the state estimation problem combining the extended Kalman filter (EKF) with a kernel recursive least squares (KRLS) algorithm (EKF-KRLS). The EKF algorithm estimates hidden states in the input space, while the KRLS algorithm estimates the measurement model. The algorithm works well without knowing the linear or nonlinear measurement model. We apply this algorithm to vehicle tracking, and compare the performances with traditional Kalman filter, EKF and KRLS algorithms. Results demonstrate that the performance of the EKF-KRLS algorithm outperforms these existing algorithms. Especially when nonlinear measurement functions are applied, the advantage of the EKF-KRLS algorithm is very obvious.

Recently, the minimum error entropy (MEE) criterion has been used as an information theoretic alternative to traditional mean-square error criterion in supervised learning systems. MEE yields nonquadratic, nonconvex performance surface even for adaptive linear neuron (ADALINE) training, which complicates the theoretical analysis of the method. In this paper, we develop a unified approach for mean-square convergence analysis for ADALINE training under MEE criterion. The weight update equation is formulated in the form of block-data. Based on a block version of energy conservation relation, and under several assumptions, we carry out the mean-square convergence analysis of this class of adaptation algorithm, including mean-square stability, mean-square evolution (transient behavior) and the mean-square steady-state performance. Simulation experimental results agree with the theoretical predictions very well.

Conventional cost functions of adaptive filtering are usually related to the error’s dispersion, such as error’s moments or error’s entropy, but neglect the shape aspects (peaks, kurtosis, tails, etc.) of the error distribution. In this work, we propose a new notion of filtering (or estimation) in which the error’s probability density function (PDF) is shaped into a desired one. As PDFs contain all the probabilistic information, the proposed method can be used to achieve the desired error variance or error entropy, and is expected to be useful in the complex signal processing and learning systems. In our approach, the information divergence between the actual errors and the desired errors is used as the cost function. By kernel density estimation, we derive the associated stochastic gradient algorithm for the finite impulse response (FIR) filter. Simulation results emphasize the effectiveness of this new algorithm in adaptive system training.

As an important category of clustering methods, subspace clustering algorithms have arisen particular attention during the last decade. Most subspace clustering algorithms are designed by first constructing a similarity matrix and then using spectral clustering algorithms to perform clustering. How to learn a suitable representation matrix to construct the similarity matrix is essential to the clustering performance. In most existing algorithms, the representation matrix is solved by norm-minimization, which commonly enforces the error matrix with nuclear norm or sparsity norm. However, these methods may fail to achieve satisfactory performance for real data contaminated by complex noise. To this end, we propose a novel robust subspace clustering method based on the Logarithmic Hyperbolic Cosine Function (LHCF). We theoretically analyze the grouping effect, as well as the convergence behavior, which illustrates that highly correlated samples can be grouped into the same cluster. Experimental results conducted on the Extended Yale B dataset show that the newly proposed algorithm yields better clustering performance compared with some advanced methods.