Semi-supervised learning suffers from the imbalance of labeled and unlabeled training data in the video surveillance scenario. In this paper, we propose a new semi-supervised learning method called SIAVC for industrial accident video classification. Specifically, we design a video augmentation module called the Super Augmentation Block (SAB). SAB adds Gaussian noise and randomly masks video frames according to historical loss on the unlabeled data for model optimization. Then, we propose a Video Cross-set Augmentation Module (VCAM) to generate diverse pseudo-label samples from the high-confidence unlabeled samples, which alleviates the mismatch of sampling experience and provides high-quality training data. Additionally, we construct a new industrial accident surveillance video dataset with frame-level annotation, namely ECA9, to evaluate our proposed method. Compared with the state-of-the-art semi-supervised learning based methods, SIAVC demonstrates outstanding video classification performance, achieving 88.76% and 89.13% accuracy on ECA9 and Fire Detection datasets, respectively. The source code and the constructed dataset ECA9 will be released inhttps://github.com/AlchemyEmperor/SIAVC.

Atrial fibrillation anomaly detection is increasingly significant today as the incidence of cardiovascular disease continues to rise. However, most of the existing supervised learning based methods for computer-aided diagnosis of atrial fibrillation heavily rely on labeled data, which is not applicable because of the scarcity of atrial fibrillation ECG data. While unsupervised methods training solely with normal samples may result in blurred decision boundaries and inadequate discriminability. In this paper, we propose a method for atrial fibrillation anomaly detection based on multimodal time-frequency pseudo anomalies, which learns pseudo anomalies rectified time-frequency hypersphere under better ECG representations. Specifically, we propose an atrial fibrillation ECG generation method that considers the rhythm and wave characteristics to construct pseudo anomalies ECG signals. These pseudo anomalies signals are then used to optimize the time-frequency hypersphere boundary, which is learned from the features of normal ECG signals in both time and frequency domains, leading to more effective atrial fibrillation anomaly detection. Extensive experiments have been conducted on multiple ECG datasets to validate the effectiveness of the proposed method.

In unsupervised graph anomaly detection, existing methods usually focus on detecting outliers by learning local context information of nodes, while often ignoring the importance of global context. However, global context information can provide more comprehensive relationship information between nodes in the network. By considering the structure of the entire network, detection methods are able to identify potential dependencies and interaction patterns between nodes, which is crucial for anomaly detection. Therefore, we propose an innovative graph anomaly detection framework, termed CoCo (Context Correlation Discrepancy Analysis), which detects anomalies by meticulously evaluating variances in correlations. Specifically, CoCo leverages the strengths of Transformers in sequence processing to effectively capture both global and local contextual features of nodes by aggregating neighbor features at various hops. Subsequently, a correlation analysis module is employed to maximize the correlation between local and global contexts of each normal node. Unseen anomalies are ultimately detected by measuring the discrepancy in the correlation of nodes’ contextual features. Extensive experiments conducted on six datasets with synthetic outliers and five datasets with organic outliers have demonstrated the significant effectiveness of CoCo compared to existing methods.

Electrocardiogram(ECG) signal analysis is crucial in healthcare service computing. Ensuring accurate assessment of ECG signal quality is vital to prevent wastage of transmission bandwidth and ineffective analysis caused by noise. This enables the efficient utilization of service resources. However, existing ECG signal quality assessment(SQA) methods primarily focus on single-modal learning, overlooking the interrelation of ECG in a multimodal feature space and failing to effectively exploit available information for pattern mining. In this paper, we model the SQA for ECG as an anomaly detection problem and propose a multimodal unsupervised SQA method. It jointly explores the boundaries between high-quality ECG and noise in both the time and frequency domains by introducing time-frequency pseudo anomalies. Specifically, we first simulate real ECG noise from the time-domain using a combination of a series of noises and convert it to the frequency-domain to form time-frequency pseudo-anomalies. Next, we map the time-frequency pseudo anomalies onto hyperspheres and jointly refine the hyperspheres learned only from high-quality ECG samples in both feature spaces. Finally, the noise score is defined as the distance from the joint time-frequency features to the center of the hypersphere. Multiple experiments on various real-world ECG datasets validate the superior performance of our proposed method.

Enhancing the accuracy of dense classification with limited labeled data and abundant unlabeled data, known as semi-supervised semantic segmentation, is an essential task in vision comprehension. Due to the lack of annotation in unlabeled data, additional pseudo-supervised signals, typically pseudo-labeling, are required to improve the performance. Although effective, these methods fail to consider the internal representation of neural networks and the inherent class-imbalance in dense samples. In this work, we propose an information transfer theory, which establishes a theoretical relationship between shallow and deep representations. We further apply this theory at both the semantic and pixel levels, referred to as IIT-SP, to align different types of information. The proposed IIT-SP optimizes shallow representations to match the target representation required for segmentation. This limits the upper bound of deep representations to enhance segmentation performance. We also propose a momentum-based Cluster-State bar that updates class status online, along with a HardClassMix augmentation and a loss weighting technique to address class imbalance issues based on it. The effectiveness of the proposed method is demonstrated through comparative experiments on PASCAL VOC and Cityscapes benchmarks, where the proposed IIT-SP achieves state-of-the-art performance, reaching mIoU of 68.34% with only 2% labeled data on PASCAL VOC and mIoU of 64.20% with only 12.5% labeled data on Cityscapes.

Capturing the complex interaction between the node attribute and the network structure is important for attributed network embedding and anomaly detection. However, there are few methods to explicitly model the correlation between these two views of the node attribute and the network structure. In this article, we propose an attributed network anomaly detection (CaCo) method based on the canonical correlation analysis, which assumes that there should be a strong correlation between the attribute and structure features of normal nodes, and a weak correlation one between those abnormal nodes, in the attributed networks. Consequently, a joint learning mechanism is designed in CaCo to explicitly measure the correlation between two views in the latent space. Specifically, the backbone of a weight-sharing graph convolutional network is employed to encode the node feature from two views of attribute and structure in the latent space, respectively. Then, a Kullback–Leibler divergence regularization is used to align the distributions of the two views. Finally, the parameters of CaCo are optimized by maximizing the correlation between attribute and structure features of normal nodes in the training phase, and anomalies can be detected by measuring the correlation between two views in the testing phase. Extensive experiments on six real-world datasets demonstrate the effectiveness of the proposed method compared to the state-of-the-art techniques.

Anomaly detection has been widely explored by training an out-of-distribution detector with only normal data for medical images. However, detecting local and subtle irregularities without prior knowledge of anomaly types brings challenges for lung CT-scan image anomaly detection. In this paper, we propose a self-supervised framework for learning representations of lung CT-scan images via both multi-scale cropping and simple masked attentive predicting, which is capable of constructing a powerful out-of-distribution detector. Firstly, we propose CropMixPaste, a self-supervised augmentation task for generating density shadow-like anomalies that encourage the model to detect local irregularities of lung CT-scan images. Then, we propose a self-supervised reconstruction block, named simple masked attentive predicting block (SMAPB), to better refine local features by predicting masked context information. Finally, the learned representations by self-supervised tasks are used to build an out-of-distribution detector. The results on real lung CT-scan datasets demonstrate the effectiveness and superiority of our proposed method compared with state-of-the-art methods.

The detection of cracks is essential for assessing and maintaining building and road safety. However, the large appearance variations and the complex topological structures of cracks bring challenges to automatic crack detection. To alleviate the above challenges, we propose a deep multi-scale crack feature learning model called DeepCrackAT for crack segmentation, which is based on an encoder–decoder network with feature tokenization mechanism and attention mechanism. Specifically, we use hybrid dilated convolutions in the first three layers of the encoder–decoder to increase the network’s receptive field and capture more crack information. Then, we introduce a tokenized multilayer perceptron (Tok-MLP) in the last two layers of the encoder–decoder to tokenize and project high-dimensional crack features into low-dimensional space. This helps to reduce parameters and enhance the network’s ability of noise resistance. Next, we concatenate the features corresponding to the encoder–decoder layers and introduce the convolutional block attention module (CBAM) to enhance the network’s perception of the critical crack region. Finally, the five-layer features are fused to generate a binary segmentation map of the crack image. We conducted extensive experiments and ablation studies on two real-world crack datasets, and DeepCrackAT achieved 97.41% and 97.25% accuracy on these datasets, respectively. The experimental results show that the proposed method outperforms the current state-of-the-art methods.

Semi-supervised learning (SSL) provides a powerful paradigm to mitigate the reliance on large labeled data by leveraging unlabeled data during model training. However, for time series data, few SSL models focus on the underlying temporal structure of time series, which results in a suboptimal representation learning quality on unlabeled time series. In this article, we propose a framework of semisupervised time series classification by investigating irregular time sampling (iTimes), which learns the underlying temporal structure of unlabeled time series in a self-supervised manner to benefit semisupervised time series classification. Specifically, we propose four different irregular time sampling functions to transform the original time series into different transformations. Then, iTimes employs a supervised module to classify labeled time series directly and employs a self-supervised module on unlabeled time series by predicting the transformation type of irregular time sampling. Finally, the underlying temporal structure pattern of unlabeled time series can be captured in the self-supervised module. The feature spaces between labeled data and unlabeled data can be aligned by jointly training the supervised and self-supervised modules which boost the ability of model learning and the representation quality. Extensive experimental results on multiple real-world datasets demonstrate the effectiveness of iTimes compared with the state-of-the-art baselines.

Unsupervised domain adaptation-based fault diagnosis methods have been extensively studied due to their powerful knowledge transferability under different working conditions. Despite their encouraging performance, most of them cannot sufficiently account for the temporal dimension of the vibration signal, resulting in incomplete feature information used in the domain alignment procedure. To alleviate the limitation, we present a self-supervised domain adaptation fault diagnosis network (SDAFDN), which considers two temporal dependencies to improve the transferability of the learned representations. Specifically, we first design a down-sampling and interaction network that considers the temporal dependency among subsequences with low temporal resolution in feature space. Then, we combine domain adversarial learning with feature mapping to achieve domain alignment. Finally, we introduced a self-supervised learning module, which considers the temporal dependency between the past and future temporal segments via classification tasks. Extensive experiments on public Paderborn University and PHM data sets demonstrate the superiority of the proposed SDAFDN and the effectiveness of considering temporal dependencies in domain alignment.

Image matting aims to extract specific objects, deployed in many applications. Generally, the automatic matting methods need an extra before overcome the intricate details and the diverse appearances. Recently, the matting community has paid more attentions to the investigation of trimap-free matting direction to address the dependency of priors. Most trimap-free approaches divide the matting task into global segmentation and detail matting subtasks. Unfortunately, these methods suffer from stagewise modeling, uncorrectable errors, or subtasks bottleneck problems. To address these issues, we propose a new set of matting subtasks, including foreground segmentation, background segmentation, and disambiguation. And we present a novel Foreground–Background Decoupling Matting (FBDM) network motivated by the new subtasks. Specifically, we first design a nested attention mechanism to decouple the backbone features. Then, we utilize two independent progressive semantic decoders by the decoupling features to complete the foreground and background segmentation subtasks. Finally, we utilize multiple of the proposed frequency division local disambiguation modules to achieve the disambiguation subtask. Besides, we establish a challenging potted plant (PPT) benchmark which contains 100 potted plants images in the real world for the matting community. Extensive experiments on several public benchmarks and the PPTs benchmark demonstrate that the proposed FBDM generates the best results compared with the state-of-the-art trimap-free methods.

Networks are ubiquitous in the real world such as social networks and communication networks, and anomaly detection on networks aims at finding nodes whose structural or attributed patterns deviate significantly from the majority of reference nodes. However, most of the traditional anomaly detection methods neglect the relation structure information among data points and therefore cannot effectively generalize to the graph structure data. In this paper, we propose an end-to-end model of Deep Dual Support Vector Data description based Autoencoder (Dual-SVDAE) for anomaly detection on attributed networks, which considers both the structure and attribute for attributed networks. Specifically, Dual-SVDAE consists of a structure autoencoder and an attribute autoencoder to learn the latent representation of the node in the structure space and attribute space, respectively. Then, a dual-hypersphere learning mechanism is imposed on them to learn two hyperspheres of normal nodes from the structure and attribute perspectives, respectively. Moreover, to achieve joint learning between the structure and attribute of the network, we fuse the structure embedding and attribute embedding as the final input of the feature decoder to generate the node attribute. Finally, abnormal nodes can be detected by measuring the distance of nodes to the learned center of each hypersphere in the latent structure space and attribute space, respectively. Extensive experiments on the real-world attributed networks show that Dual-SVDAE consistently outperforms the state-of-the-arts, which demonstrates the effectiveness of the proposed method.

Link prediction aims at inferring missing links or predicting future ones based on the currently observed network. This topic is important for many applications such as social media, bioinformatics and recommendation systems. Most existing methods focus on homogeneous settings and consider only low-order pairwise relations while ignoring either the heterogeneity or high-order complex relations among different types of nodes, which tends to lead to a sub-optimal embedding result. This paper presents a method named Heterogeneous Hypergraph Variational Autoencoder (HeteHG-VAE) for link prediction in heterogeneous information networks (HINs). It first maps a conventional HIN to a heterogeneous hypergraph with a certain kind of semantics to capture both the high-order semantics and complex relations among nodes, while preserving the low-order pairwise topology information of the original HIN. Then, deep latent representations of nodes and hyperedges are learned by a Bayesian deep generative framework from the heterogeneous hypergraph in an unsupervised manner. Moreover, a hyperedge attention module is designed to learn the importance of different types of nodes in each hyperedge. The major merit of HeteHG-VAE lies in its ability of modeling multi-level relations in heterogeneous settings. Extensive experiments on real-world datasets demonstrate the effectiveness and efficiency of the proposed method.

Semi-supervised learning (SSL) has proven to be a powerful algorithm in different domains by leveraging unlabeled data to mitigate the reliance on the tremendous annotated data. However, few efforts consider the underlying temporal relation structure of unlabeled time series data in the semi-supervised learning paradigm. In this work, we propose a simple and effective method of Semi-supervised Time series classification architecture (termed as SemiTime) by gaining from the structure of unlabeled data in a self-supervised manner. Specifically, for the labeled time series, SemiTime conducts the supervised classification directly under the supervision of the annotated class label. For the unlabeled time series, the segments of past-future pair are sampled from time series, where two segments of pair from the same time series candidate are in positive temporal relation, while two segments from the different candidates are in negative temporal relation. Then, the temporal relation between those segments is predicted by SemiTime in a self-supervised manner. Finally, by jointly classifying labeled data and predicting the temporal relation of unlabeled data, the useful representation of unlabeled time series can be captured by SemiTime. Extensive experiments on multiple real-world datasets show that SemiTime consistently out-performs the state-of-the-arts, which demonstrates the effectiveness of the proposed method. Code and data are publicly available at https://haoyfan.github.io.

In this paper, we propose a novel entropy minimization based semi-supervised method for semantic segmentation. Entropy minimization has proven to be an effective semi-supervised method for realizing the cluster assumption, where the decision boundary should lie in low-density regions. Inspired by the existing consistency training semi-supervised segmentation networks with encoder-decoder architecture, we found that there tend to be more large gradient values at the object edges than other positions in the feature map of the encoder, and therefore propose a feature gradient map regularization to enlarge inter-class distance in the feature space for low-entropy of segmentation prediction. Additionally, we introduce an adaptive sharpening scheme with aleatoric uncertainty, and a class consistency constraint regularization, to alleviate the interference of noise with pseudo labels. Extensive experiments on PASCAL VOC, PASCAL-Context, and Leukocyte datasets show that the proposed method achieves state-of-the-art semi-supervised semantic segmentation performance without almost additional calculations and network structures.

Anomaly detection on attributed networks aims at finding nodes whose patterns deviate significantly from the majority of reference nodes, which is pervasive in many applications such as network intrusion detection and social spammer detection. However, most existing methods neglect the complex cross-modality interactions between network structure and node attribute. In this paper, we propose a deep joint representation learning framework for anomaly detection through a dual autoencoder (AnomalyDAE), which captures the complex interactions between network structure and node attribute for high-quality embeddings. Specifically, Anoma-lyDAE consists of a structure autoencoder and an attribute autoencoder to learn both node embedding and attribute embedding jointly in latent space. Moreover, attention mechanism is employed in structure encoder to learn the importance between a node and its neighbors for an effective capturing of structure pattern, which is important to anomaly detection. Besides, by taking both the node embedding and attribute embedding as inputs of attribute decoder, the cross-modality interactions between network structure and node attribute are learned during the reconstruction of node attribute. Finally, anomalies can be detected by measuring the reconstruction errors of nodes from both the structure and attribute perspectives. Extensive experiments on real-world datasets demonstrate the effectiveness of the proposed method.

Summary Tongue segmentation is a key step of automatic tongue diagnosis, and the major challenges for the effective segmentation lie in the large appearance variations of tongue caused by different diseases, for example, tongue coating and tongue texture. Moreover, the limited labeled data also hinders traditional supervised methods from their powerful learning ability. To alleviate these challenges, in this work, we propose a reconstruction enhanced probabilistic model for semisupervised tongue segmentation, named SemiTongue, in which, image reconstruction constraint combined with adversarial learning is used to improve the accuracy of tongue segmentation. Specifically, based on a shared feature encoder that served as an inference model, two separate branches in SemiTongue as the generative model, which are composed of a segmentation decoder and a reconstruction decoder, are utilized to generate the tongue segmentation and reconstruct original tongue image respectively. Then, a discriminator is employed to differentiate the generated segmentation map from the ground truth segmentation distribution. Moreover, semisupervised learning is conducted through discriminator by discovering the reliable region in the generated segmentation map of unlabeled images, which is further utilized to supervise the segmentation branch. Experimental results compared with state‐of‐the‐art methods on real‐world datasets demonstrate the effectiveness of SemiTongue.