Pre-training techniques significantly enhance the performance of semantic segmentation tasks with limited training data. However, the efficacy under a large domain gap between pre-training (e.g. RGB) and fine-tuning (e.g. infrared) remains underexplored. In this study, we first benchmark the infrared semantic segmentation performance of various pre-training methods and reveal several phenomena distinct from the RGB domain. Next, our layerwise analysis of pre-trained attention maps uncovers that: (1) There are three typical attention patterns (local, hybrid, and global); (2) Pre-training tasks notably influence pattern distribution across layers; (3) The hybrid pattern is crucial for semantic segmentation as it attends to both nearby and foreground elements; (4) The texture bias impedes model generalization in infrared tasks. Building on these insights, we propose UNIP, a UNified Infrared Pre-training framework, to enhance the pre-trained model performance. This framework uses the hybrid-attention distillation NMI-HAD as the pre-training target, a large-scale mixed dataset InfMix for pre-training, and a last-layer feature pyramid network LL-FPN for fine-tuning. Experimental results show that UNIP outperforms various pre-training methods by up to 13.5% in average mIoU on three infrared segmentation tasks, evaluated using fine-tuning and linear probing metrics. UNIP-S achieves performance on par with MAE-L while requiring only 1/10 of the computational cost. Furthermore, with fewer parameters, UNIP significantly surpasses state-of-the-art (SOTA) infrared or RGB segmentation methods and demonstrates the broad potential for application in other modalities, such as RGB and depth. Our code is available at https://github.com/casiatao/UNIP.

Graph self-supervised learning provides a powerful guarantee for learning high-quality representations in an unsupervised manner. Despite its early birth, the performance of generative graph self-supervised learning has long lagged behind that of up-and-coming contrastive learning, especially on node classification tasks. In this paper, we investigate potential issues in existing graph autoencoders and attribute their poor performance to three main aspects: complex decoder design, lack of desmoothing process in feature remap, and overemphasis on local topological proximity. To tackle these issues, we propose an effective and efficient graph autoencoder framework for unsupervised representation learning, which contains two key components: lightweight smoothness-aware feature reconstructor and global structural dependency catcher. After performing a desmoothing operation on encoded representations via a learnable high-pass filter, the feature decoder reconstructs the original features through a simple linear projection. The lightweight design liberates the decoder from self-supervised pretext tasks and puts the encoder more accountable for achieving optimization objectives, which promotes effective training of the encoder. Global structural dependency catcher utilizes graph diffusion to build a structural regularization to capture long-range topological dependency on a graph. The empirical studies demonstrate the effectiveness of our approach, which can surpass dominant contrastive learning methods.

Enlightened by the InfoMax principle, Graph Contrastive Learning (GCL) has achieved remarkable performance in processing large amounts of unlabeled graph data. Due to the impracticality of precisely calculating mutual information (MI), conventional contrastive learning methods turn to approximate its lower bound using parametric neural estimators, which inevitably introduces additional parameters and leads to increased computational complexity. Building upon a common Gaussian assumption on the distribution of node representations, a computationally tractable surrogate for the original MI can be rigorously derived, termed as Gaussian Mutual Information (GMI). Leveraging multi-view priors of GCL, we induce an efficient contrastive objective based on GMI with performance guarantees, eliminating the reliance on parameterized estimators and negative samples. The emergence of another decorrelation-based self-supervised learning branch parallels contrastive-based approaches. By positioning the proposed GMI-based objective as a pivot, we bridge the gap between these two research areas from two aspects of approximate form and consistent solution, which contributes to the advancement of a unified theoretical framework for self-supervised learning. Extensive comparison experiments, ablation studies, and visual analysis provide compelling evidence for the effectiveness and efficiency of our method while supporting our theoretical achievements.

The abundant features and structure information on graphs provide a potential guarantee for learning high-quality representations without supervision. Feature attribute represents the inherent properties of nodes, while structure attribute describes their neighborhood relationship. These two types of attributes can be regarded as different modal forms of the same instance and should be consistent in identifying a member. We propose to directly regard feature and structure attributes as two separate views to embed this consistency into contrastive learning method, realizing graph information interaction on feature and structure in a cross-modal contrastive framework. Under this framework, node representations are learned in an unsupervised manner by maximizing the agreement between feature representation and structure representation. In terms of negative samples, instead of randomly sampling points from empirical distribution, a simple yet effective multi-sample mixing strategy is proposed to synthesize true negative samples with greater probability, alleviating the tricky false negative issue. Extensive experiments on multiple types of graphs demonstrate the effectiveness of the proposed method.

Although recent advances have prompted the prosperity in graph contrastive learning, the researches on universal principles for model design and desirable properties of latent representations are still inadequate. From a statistical perspective, this paper proposes two principles for guidance and constructs a general self-supervised framework for negative-free graph contrastive learning. Reformulating data augmentation as a mixture process, the first one, termed consistency principle, lays stress on exploring and mapping cross-view common information to consistent and essence-revealing representations. For the purpose of instantiation, four statistical indicators are employed to estimate and maximize the correlation between representations from various views, whose accordant variation trend during training implies the extraction of common content. With awareness of the insufficiency of a solo consistency principle, suffering from degenerated and coupled solutions, a decorrelation principle is put forward to encourage diverse and informative representations. Accordingly, two specific strategies, performing in representation space and eigen spectral space, respectively, are propounded to decouple various representation channels. Under two principles, various combinations of concrete implementations derive a family of methods. The comparison experiments with current state-of-the-arts demonstrate the effectiveness and sufficiency of two principles for high-quality graph representations. Furthermore, visual studies reveal how certain principles affect learned representations.

Graph neural networks (GNNs) have been widely used to accomplish graph classification tasks such as predicting molecular properties and classifying the labels of proteins. Discovering the latent discriminative substructures (e.g., functional groups in molecules) is a vital task to enhance the classification performance. In this paper, this task is addressed as a problem of discriminative graph representation learning. Specifically, a novel node sampling strategy is developed to achieve this goal. To this end, graph-dependent sampling vectors are first learned by a mini-network to exploit various informative substructures on graphs and sample some representative nodes, which could be regarded as performing a distributed sampling on graphs. Then, the sampled nodes are organized together topologically as a subgraph with landing probabilities of random walks. Moreover, a self-adaptive pooling ratio of nodes is obtained via feature smoothness of graphs, eliminating the trouble of manual selection of subgraph size. As a result, these treatments are equivalent to performing the difficult step of down-pooling operation on non-grid graph data. Extensive experiments and ablation studies on multiple benchmark datasets demonstrate the effectiveness and superiority of our proposed approach. Additionally, interpretability studies illustrate the ability of our model to extract discriminative substructures.