Recently, the usage of Contrastive Representation Learning (CRL) as a pre-training technique improves the performance of learning with noisy labels (LNL) methods. However, instead of pre-training, when trivially combining CRL loss with LNL methods as an end-to-end framework, the empirical experiments show severe degeneration of the performance. We verify through experiments that this issue is caused by optimization conflicts of losses and propose an end-to-end PLReMix framework by introducing a Pseudo-Label Relaxed (PLR) contrastive loss. This PLR loss constructs a reliable negative set of each sample by filtering out its inappropriate negative pairs, alleviating the loss conflicts by trivially combining these losses. The proposed PLR loss is pluggable and we have integrated it into other LNL methods, observing their improved performance. Furthermore, a two-dimensional Gaussian Mixture Model is adopted to distinguish clean and noisy samples by leveraging semantic information and model outputs simultaneously. Experiments on multiple benchmark datasets demonstrate the effectiveness of the proposed method. Code is available at https://github.com/lxysl/PLReMix.

Identifying structures of complex networks based on time series of nodal data is of considerable interest and significance in many fields of science and engineering. This article presents a sparse Bayesian learning (SBL) method for identifying structures of community-bridge networks, where nodes are grouped to form communities connected via bridges. Using the structural information of such networks with unknown nodal dynamics and community formations, network structure identification is tackled similar to sparse signal reconstruction with mixed sparsity mode. The proposed method is theoretically proved to be convergent. Its superiority to mainstream baselines is demonstrated via extensive experiments without the need for manual adjustment of regularization parameters.

As a well-known machine learning algorithm, sparse Bayesian learning (SBL) can find sparse representations in linearly probabilistic models by imposing a sparsity-promoting prior on model coefficients. However, classical SBL algorithms lack the essential theoretical guarantees of global convergence. To address this issue, we propose an iterative Min-Min optimization method to solve the marginal likelihood function (MLF) of SBL based on the concave-convex procedure. The method can optimize the hyperparameters related to both the prior and noise level analytically at each iteration by re-expressing MLF using auxiliary functions. Particularly, we demonstrate that the method globally converges to a local minimum or saddle point of MLF. With rigorous theoretical guarantees, the proposed novel SBL algorithm outperforms classical ones in finding sparse representations on simulation and real-world examples, ranging from sparse signal recovery to system identification and kernel regression.

Learning dynamical networks based on time series of nodal states is of significant interest in systems science, computer science, and control engineering. Despite recent progress in network identification, most research focuses on static structures rather than switching ones. Therefore, this article develops a method for identifying the structures of switching networks by exploring and leveraging both temporal and spatial structural information that characterizes the switching process. The proposed method employs a new sparse Bayesian learning algorithm based on coupled hyperblocks to estimate unknown switching instants. Experimental results on benchmark artificial and real networks are elaborated to demonstrate the effectiveness and superiority of the proposed method.

Multivariate time series anomaly detection is one of the most indispensable yet troublesome links in complex industrial processes. The main challenge lies in discovering the representative patterns for collective or contextual anomalies among interconnected sensory data streams, which has been largely hampered by inefficient spatial-temporal feature extraction and suboptimal decision criteria under the scarcity of positive training samples. This article goes beyond the common limitations of the existing methods, and novelly proposes Hierarchical Spatial-Temporal grAph Representation (HiSTAR). It processes the data with strong structural inductive biases through latent spatial-temporal graph modeling, yet requiring no predefined topological priors for the sensor network. A discriminative decision boundary is constructed by learning hierarchical normality-enclosing hyperspheres on the produced graph-structure representations. In this way, HiSTAR not only presents superior anomaly detection performance, but also provides consistent anomaly localization results. The efficacy of the proposed method is experimentally corroborated through three industrial case studies.

Maintenance of machinery equipment in smart manufacturing requires real-time health monitoring, strongly supported by the rapid evolution of Artificial Intelligence (AI) technologies. Most AI-based health monitoring systems are powered by advanced modeling methods and intensive high-quality monitoring data. Such monitoring systems center on high-accuracy predictive performance but cannot necessarily convey reliability, such as satisfactory resistance to strong noises, credible uncertainty analysis, and model interpretability. This article novelly proposes noise-aware sparse Gaussian processes (NASGP) within the Bayesian inference framework. NASGP are capable of consistent high-performance and credible uncertainty assessment under strong noises. Based on NASGP, we then develop an explainable generalized additive model to bridge the gap between latent inference mechanism and domain expert knowledge. The efficacy of the proposed approach is corroborated through two case studies including remaining useful life prognosis and fault diagnosis for rolling bearings.

Attention has turned in recently to clustering of collections networks where each network is characterized by a sparse inverse covariance matrix. Here we consider a mixture model of sparse Gaussian graphical models and develop an exact EM algorithm that improves considerably on a previous approximation. We provide numerical examples to show its superior performance.

Time-series clustering involves grouping homogeneous time series together based on certain similarity measures. The mixture AR model (MxAR) has already been developed for time series clustering, as has an associated EM algorithm. However, this EM clustering algorithm fails to perform satisfactorily in large-scale applications due to its high computational complexity. This paper proposes a new algorithm, k-ARs, which is a limiting version of the existing EM algorithm. It shows remarkably good computational performance when applied to large-scale clustering problems as illustrated on some benchmark simulations motivated by some real applications.

While there is now a significant literature on sparse inverse covariance estimation, all that literature, with only a couple of exceptions, has dealt only with univariate (or scalar) networks where each node carries a univariate signal. However in many, perhaps most, applications, each node may carry multivariate signals representing multi-attribute data, possibly of different dimensions. Modelling such multivariate (or vector) networks requires fitting block-sparse inverse covariance matrices. Here we achieve maximal block sparsity by maximizing a block-l0-sparse penalized likelihood. There is only one previous algorithm that already does this, but it does not scale. Here we address key computational bottlenecks and develop a new algorithm which is much faster and has massively reduced requirements on matrix conditioning. A benchmark study shows a computational speed-up by many orders of magnitude.