Semantic scene completion simultaneously reconstructs the shapes of missing regions and predicts semantic labels for the entire 3D scene. Although point cloud-based methods are more efficient than voxel-based methods, existing point cloud-based approaches largely fail to fully leverage semantic information. To address this challenge, we propose a Prototype-Guided Transformer (ProtoFormer) that encodes semantic information into a set of semantic prototypes to guide the underlying Transformer for semantic scene completion. Specifically, we leverage semantic prototypes to enhance information from both geometric and semantic perspectives, and integrate the top-K attention mechanisms to guide scene completion and semantic awareness. Extensive qualitative and quantitative experimental results demonstrate that ProtoFormer outperforms state-of-the-art approaches with low complexity.

The primary goal of 3D point cloud completion is to reconstruct complete and high-resolution point clouds from incomplete and low-resolution inputs. While some recent approaches have achieved satisfactory completion performance by incorporating additional images, there remains room for improvement in fully exploiting and utilizing the rich geometric relation information contained in parts. To address this challenge, we propose a novel Semantic Guided Part Relation-aware Network (SGPRNet) for Point Cloud Completion. Its core innovation lies in establishing part semantic relations to guide the reconstruction of structurally consistent local geometries. Specifically, we utilize Large Multi-modal Models (LMMs) to automatically generate the specific text of 3D shape, which provides detailed geometric part relations descriptions. Building upon this, we design an Orthogonal Semantic Part Transfer (OSPT) module that learns transferable semantic relations between geometric parts. Subsequently, we develop a Semantic Geometric Relation-aware Transformer (SGRFormer) to progressively refine these semantic features, enhancing point cloud representation and guiding the generation of fine local structures. In addition, we establish a point-text pairs corpus, OmniObject3D-212/34 and Text-ViPC datasets based on existing OmniObject3D and ShapeNet-ViPC datasets, incorporating the specific text. Extensive experimental results demonstrate that our method outperforms existing state-of-the-art completion methods.

Graph Neural Networks (GNNs) demonstrate promising performance in data mining yet exhibit inherent vulnerabilities to adversarial attacks. Even imperceptible perturbations degrade model performance, seriously hindering the application of GNNs in reality. In recent years, adversarial defense methods based on model architecture have gained attention for their effectiveness. However, they exhibit limited effectiveness against emerging black-box influence maximization attacks (IMAs), which aim to maximize the spread of feature perturbations through a group of influential nodes. This may leave a potential risk in real-world applications. To address this issue, we propose a Graph Adversarial Defense method based on the Hilbert-Schmidt Independence Criterion (HSIC-GAD). Specifically, the proposed method leverages hidden representations to capture the dependence between preprocessed node features and label information. On this basis, we design a regularizer that simultaneously preserves the most relevant information for downstream tasks while filtering out adversarial perturbations from the input data. A simple theoretical analysis shows that the HSIC regularizer can reduce the sensitivity of the model to adversarial inputs. Additionally, it exhibits strong universality, consistently enhancing the adversarial robustness of diverse models. Extensive experiments on real-world datasets demonstrate that HSIC-GAD outperforms state-of-the-art defense methods against IMAs.

Detecting and identifying complex association patterns between two variables is a fundamental task. This requires association measures that satisfy both generality (the ability to capture a wide range of association structures) and equitability (the absence of bias toward specific association types). Designing such measures is challenging due to the distributional uncertainty, structural diversity, and mixture of association types found in large datasets. Granular computing offers a promising direction, as local neighborhood structures naturally encode multi-scale association information. Inspired by this insight, we introduce the maximal neighborhood coefficient (MNC), an association measure based on $k$k-NN granulation. MNC captures a broad range of associations without empirical bias while retaining local structural details often missed by existing measures. Extending this idea, we develop a family of maximal neighborhood nonparametric exploration (MNNE) statistics that supply richer auxiliary information for characterizing associations. Together, MNC and MNNE form a data-driven exploration toolkit that offers strong empirical performance and a new perspective on mining complex association patterns.

Graph Transformers (GTs) have shown great potential in graph representation learning over the past few years. In spite of their promising performance on small graphs, how to develop a scalable and powerful architecture still remains to be explored. While some of the existing works devote to designing linear attention mechanisms, the expressiveness is confined by their smooth attention score distributions. Another impact factor of expressiveness is the inductive bias. Unfortunately, GTs generally lack strong graph-specific inductive bias, which leads to failure in capturing long-range hierarchical structures or community structures. To address the issues above, we develop a Scalable and Comprehensive Graph Transformer (SCGT). Specifically, we leverage a Focused Graph Linear Attention (FGLA) to generate a sharp distribution of attention scores. In addition, we design a Comprehensive Positional Encoding (CPE) to capture comprehensive awareness of the original node-level features. Extensive empirical results show that SCGT achieves highly competitive performance with decent efficiency on 12 datasets. Theoretical analysis demonstrates the expressiveness of our proposed method.

Multi-label learning plays a crucial role in numerous real-world applications, where each instance may be associated with multiple semantic labels. However, in practical scenarios, label noise is widespread and poses a significant challenge, as it can simultaneously distort multiple labels due to ambiguous annotations, human fatigue, or automated labeling inaccuracies. While considerable progress has been made in developing noise-robust methods for single-label learning, addressing noise in multi-label settings remains substantially more challenging due to the intricate interplay among multiple potentially corrupted labels. This underscores the pressing need for effective label correction strategies tailored to noisy multi-label learning. To address this gap, we propose a latent contrastive embedding framework designed for noisy multi-label scenarios. The approach not only learns robust feature representations through supervised contrastive learning but also dynamically identifies clean labels via a small-loss guided sample selection strategy. Moreover, the embedding and label correction processes are jointly optimized, allowing the model to capture the semantic structure of both features and labels under noisy conditions. Specifically, we adopt a small-loss criterion to distinguish clean from noisy samples during the early training phase. In addition, a balanced loss is introduced to mitigate the effects of label imbalance and sample difficulty. Finally, comprehensive experiments conducted under various patterns and levels of label noise demonstrate the superior robustness and generalization ability of the proposed method in noisy multi-label classification tasks.

Rationalization, a data-centric framework, aims to build self-explanatory models to explain the prediction outcome by generating a subset of human-intelligible pieces of the input data. It involves a cooperative game model where a generator generates the most human-intelligible parts of the input (i.e., rationales), followed by a predictor that makes predictions based on these generated rationales. Conventional rationalization methods typically impose constraints via regularization terms to calibrate or penalize undesired generation. However, these methods are suffering from a problem called mode collapse, in which the predictor produces correct predictions yet the generator consistently outputs rationales with collapsed patterns. Moreover, existing studies are typically designed separately for specific collapsed patterns, lacking a unified consideration. In this paper, we systematically revisit cooperative rationalization from a novel game-theoretic perspective and identify the fundamental cause of this problem: the generator no longer tends to explore new strategies to uncover informative rationales, ultimately leading the system to converge to a suboptimal game equilibrium (correct predictionsv.scollapsed rationales). To solve this problem, we then propose a novel approach, Game-theoreticPolicyOptimization orientedRATionalization (PoRat), which progressively introduces policy interventions to address the game equilibrium in the cooperative game process, thereby guiding the model toward a more optimal solution state. We theoretically analyse the cause of such a suboptimal equilibrium and prove the feasibility of the proposed method. Furthermore, we validate our method on nine widely used real-world datasets and two synthetic settings, wherePoRatachieves up to 8.1% performance improvements over existing state-of-the-art methods. Code and data are available atanonymous.4open.science/r/Rationalization-PORAT-ECE9.

Adapting Vision Transformers (ViTs) for medical imaging is constrained by the scarcity of data and high-quality annotations, hindering effective training and robust generalization. Visual prompt learning offers a parameter-efficient solution for domain adaptation, but its success depends on accurate and task-relevant semantic guidance-a resource rarely available in real-world clinical practice despite its proven benefits. This motivates the need for mechanisms that can automatically extract reliable semantic cues from existing clinical data. To this end, we propose Graph-Enhanced Visual Prompting (GEVP), the first framework to incorporate cross-modal graph learning into prompt generation for medical imaging. GEVP models image patches and report tokens as graph nodes, captures their spatial and semantic relations via a graph neural network, and produces semantically rich prompts. These prompts are injected into a frozen ViT backbone, guiding attention to diagnostically relevant regions without heavy fine-tuning. A consistent downstream prediction mechanism leverages the pretrained prompt generator to handle both report-available and report-absent settings. Experiments on six public downstream datasets show GEVP surpasses strong prompt- and adapter-based baselines by up to +9.65% F1 on imbalanced tasks and delivers superior unseen disease classification.

Despite significant advancements in large visual-language models (LVLMs), hallucinations remain a major bottleneck in their practical applications. One key factor contributing to hallucinations is the over-reliance on language priors during the autoregressive text generation process. Visual Contrastive Decoding (VCD), a popular technique for mitigating hallucinations, perturbs the visual input and compares the perturbed output with the original. However, it often overlooks the gradual attenuation of visual information within the decoder, limiting the model's ability to generate text based on actual visual content. We propose a novel, training-free method—Visual-Enhanced Contrastive Decoding (VECD)—which addresses this issue by amplifying visual information within the decoder, thereby reducing hallucinations caused by excessive reliance on language priors. VECD dynamically selects later layers for visual injection, while retaining only essential visual tokens in early layers. This approach enhances the generation process by adaptively balancing visual and language priors. By comparing outputs with and without visual amplification, we derive a refined probability distribution for the next token. Moreover, we improve the beam search algorithm by introducing a visually guided token selection strategy, enabling the generation of text that aligns more closely with the image content. Our extensive experiments show that VECD significantly reduces hallucinations and improves the quality of generated text, demonstrating its effectiveness as a practical solution.

Graph Neural Networks are powerful tools for modeling graph-structured data but their interpretability remains a significant challenge. Existing model-agnostic GNN explainers aim to identify critical subgraphs or node features relevant to task predictions but often rely on GNN predictions for supervision, lacking ground-truth explanations. This limitation can introduce biases, causing explanations to fail in accurately reflecting the GNN's decision-making processes. To address this, we propose a novel explainer for GNNs with graph segmentation and contrastive learning. Our model introduces a graph segmentation learning module to divide the input graph into explanatory and redundant subgraphs. Next, we implement edge perturbation to augment these subgraphs, generating multiple positive and negative pairs for contrastive learning between explanatory and redundant subgraphs. Finally, we develop a contrastive learning module to guide the learning of explanatory and redundant subgraphs by pulling positive pairs with the same explanatory subgraphs closer while pushing negative pairs with different explanatory subgraphs far away. This approach allows for a clearer distinction of critical subgraphs, enhancing the fidelity of the explanations. We conducted extensive experiments on graph classification and node classification tasks, demonstrating the effectiveness of the proposed method.

The ubiquitous and unavoidable label noise brings great challenges to the generalization performance of learning methods.Label noise correction aims to detect and correct label noise in the data, which is one of the most potential methods to address this challenge.Current methods for label noise filtering that utilize primitive features primarily concentrate on identifying noise, which often limits their capacity to adaptively learn features crucial for specific tasks, thereby resulting in a higher rate of noise identification within the noise recognition process. On the other hand, deep neural networks, endowed with robust feature extraction capabilities, typically exhibit lower noise identification, as they are prone to fitting noise patterns during the recognition process, potentially undermining their overall efficacy. Moreover, Fuzzy Learning Machine (FLM) excels not only in feature extraction but also in noise tolerance, adeptly navigating data uncertainties. FLM enhances the accuracy of the labels by calculating the membership degrees of samples across categories and determining their fuzzy memberships. The introduction of a two-stage FLM-based framework, which employs a secondary learning mechanism for precise noise filtering and correction, has shown substantial improvements in noise correction across various large-scale noisy datasets, thereby significantly enhancing samples' quality and boosting the generalization capabilities of classifiers.

Graph contrastive learning (GCL) has become a hot topic in the field of graph representaion learning. In contrast to traditional supervised learning relying on a large number of labels, GCL exploits augmentation techniques to generate multiple views and positive/negative pairs, both of which greatly influence the performance. Unfortunately, commonly used random augmentations may disturb the underlying semantics of graphs. Moreover, traditional GNNs, a type of widely employed encoders in GCL, are inevitably confronted with over-smoothing and over-squashing problems. To address these issues, we propose GNN-Transformer Cooperative Architecture for Trustworthy Graph Contrastive Learning (GTCA), which inherits the advantages of both GNN and Transformer, incorporating graph topology to obtain comprehensive graph representations. Theoretical analysis verifies the trustworthiness of the proposed method. Extensive experiments on benchmark datasets demonstrate state-of-the-art empirical performance.

Dealing with the distribution shift is a significant challenge when building offline reinforcement learning (RL) models that can generalize from a static dataset to out-of-distribution (OOD) scenarios. Previous approaches have employed pessimism or conservatism strategies. More recently, data-driven work has taken a distributional perspective, treating offline data as a domain adaptation problem. However, these methods use heuristic techniques to simulate distribution shifts, resulting in a limited diversity of artificially created distribution gaps. In this paper, we propose a novel perspective: offline datasets inherently contain multiple latent distributions, with behavior data from diverse policies potentially following different distributions and data from the same policy across various time phases also exhibiting distribution variance. We introduce the Latent Distribution Representation Learning (LAD) framework, which aims to characterize the multiple latent distributions within offline data and reduce the distribution gaps between any pair of them. LAD consists of a min-max adversarial process: it first identifies the "worst-case" distributions to enlarge the diversity of distribution gaps and then reduces these gaps to learn invariant representations for generalization. We derive a generalization error bound to support LAD theoretically and verify its effectiveness through extensive experiments.

Traditional semi-supervised learning achieves significant success in closed-world scenarios. To better align with the openness of the real world, researchers propose open-world semi-supervised learning (OWSSL), which enables models to effectively recognize known and unknown classes even without labels for unknown classes. Recently, researchers have attempted to enhance the model performance in recognizing visually similar classes by integrating textual information. However, these attempts do not effectively align images with text, resulting in limited improvements in model performance. In response to this challenge, we propose a novel OWSSL method. By adopting a global-and-local textual prompt learning strategy to enhance image-text alignment effectiveness, and implementing a forward-and-backward strategy to reduce noise in image-text matching for unlabeled samples, we ultimately enhance the model’s ability to extract and recognize discriminative features across different classes. Experimental results on multiple fine-grained datasets demonstrate that our method achieves significant performance improvements compared to state-of-the-art methods.

Convolutional neural networks (CNNs) excel at learning spatial patterns through local receptive fields, while channel attention mechanisms enhance feature discrimination by emphasizing informative channels. Traditional approaches like SENet generate channel weights through global pooling and bottlenecked MLPs, but fundamentally ignore cross-channel dependencies—a critical limitation that discards contextual relationships between features. To address this, we propose the Cross-Channel Graph Attention mechanism (CCA), which explicitly models channel interactions via graph neural networks. Specifically, CCA represents channels as graph nodes, constructs sparse KNN graphs to capture inter-channel dependencies, and performs attention-based feature aggregation. Theoretically, we prove that CCA strictly expands the expressive power of conventional channel attention through full-rank attention matrices, while maintaining superior generalization via sparsity-induced Rademacher complexity bounds. Furthermore, we establish that CCA can universally approximate any continuous channel mixing function with controlled neighbor size (Theorem 3). Extensive experiments on image classification (CIFAR-10/100, mini-ImageNet) and super-resolution (Set5/14) validate that CCA-equipped networks achieve state-of-the-art performance—improving ResNet34 by 2.16% TOP-1 accuracy on CIFAR-10 and boosting VDSR’s PSNR by 0.43dB on ×3 scaling. Both theoretical and empirical analyses demonstrate that CCA achieves an optimal balance between feature interaction capacity and computational efficiency through its graph-structured attention paradigm. The code is available at https://github.com/openreview-pro/CCA

The growing demand for personalized learning underscores the importance of accurately predicting students’ future performance to support tailored education and optimize instructional strategies. Traditional approaches predominantly focus on temporal modeling using historical response records and learning trajectories. While effective, these methods often fall short in capturing the intricate interactions between students and learning content, as well as the subtle semantics of these interactions. To address these gaps, we present EduLLM, the first framework to leverage large language models in combination with hypergraph learning for student performance prediction. The framework incorporates FraS-HNN ($\underline{\mbox{Fra}}$melet-based $\underline{\mbox{S}}$igned $\underline{\mbox{H}}$ypergraph $\underline{\mbox{N}}$eural $\underline{\mbox{N}}$etworks), a novel spectral-based model for signed hypergraph learning, designed to model interactions between students and multiple-choice questions. In this setup, students and questions are represented as nodes, while response records are encoded as positive and negative signed hyperedges, effectively capturing both structural and semantic intricacies of personalized learning behaviors. FraS-HNN employs framelet-based low-pass and high-pass filters to extract multi-frequency features. EduLLM integrates fine-grained semantic features derived from LLMs, synergizing with signed hypergraph representations to enhance prediction accuracy. Extensive experiments conducted on multiple educational datasets demonstrate that EduLLM significantly outperforms state-of-the-art baselines, validating the novel integration of LLMs with FraS-HNN for signed hypergraph learning.

Classification is a cornerstone of machine learning research. Most of the existing classifiers assume that the concepts corresponding to classes can be precisely defined. This notion diverges from the widely accepted understanding in cognitive science, which posits that real-world concepts are often inherently ambiguous. To bridge this big gap, we propose a Human Cognition-Inspired Hierarchical Fuzzy Learning Machine (HC-HFLM), which leverages a novel hierarchical alignment loss to integrate rich class knowledge from human knowledge system into learning process. We further theoretically prove that minimizing this loss can align the hierarchical structure derived from data with those contained in class knowledge, resulting in clear semantics and high interpretability. Systematic experiments verify that the proposed method can achieve significant gains in interpretability and generalization performance.

Federated causal structure learning aims to infer causal relationships from data stored on individual clients, with privacy concerns. Most existing methods assume identical variable sets across clients and present federated strategies for aggregating local updates. However, in practice, clients often observe overlapping but non-identical variable sets, and non-overlapping variables may introduce spurious dependencies. Moreover, existing strategies typically reflect only the overall quality of local graphs, ignoring the varying importance of relationships within each graph. In this paper, we study federated causal structure learning with non-identical variable sets, aiming to design an effective strategy for aggregating “correct” and “good” (non-)causal relationships across distributed datasets. Specifically, we first develop theories for detecting spurious dependencies, examining whether the learned relationships are “correct” or not. Furthermore, we define stable relationships as those that are both “correct” and “good” across multiple graphs, and finally design a two-level priority selection strategy for aggregating local updates, obtaining a global causal graph over the integrated variables. Experimental results on synthetic, benchmark and real-world data demonstrate the effectiveness of our method.

Estimating Individual Treatment Effects (ITE) from observational data is challenging due to covariate shift and counterfactual absence. While existing methods attempt to balance distributions globally, they often lack fine-grained sample-level alignment, especially in scenarios with significant individual heterogeneity. To address these issues, we reconsider counterfactual as a proxy to emulate balanced randomization. Furthermore, we derive a theoretical bound that links the expected ITE estimation error to both factual prediction errors and representation distances between factuals and counterfactuals. Building on this theoretical foundation, we propose FCCL, a novel method designed to effectively capture the nuances of potential outcomes under different treatments by (i) generating diffeomorphic counterfactuals that adhere to the data manifold while maintaining high semantic similarity to their factual counterparts, and (ii) mitigating distribution shift via sample-level alignment grounded in our derived generalization-error bound, which considers factual-counterfactual similarity and category consistency. Extensive evaluations on benchmark datasets demonstrate that FCCL outperforms 13 state-of-the-art methods, particularly in capturing individual-level heterogeneity and handling sparse boundary samples.

Open-world semi-supervised learning (OWSSL) aims to recognize both known and unknown classes, but the labeled samples only cover the known classes. Existing OWSSL methods primarily represent classes as symbolic variables, which ignore the rich internal semantic information associated with the classes and thus hampers their ability to recognize unknown classes. Recent studies incorporate textual descriptions of classes to facilitate training, but these methods overlook the class semantic correlations, which constrains their effectiveness in recognizing unknown classes. To address these issues, we propose a novel OWSSL method. Our method fine-tunes only the image encoder during training while keeping the text encoder frozen, thereby preserving the rich semantic correlations learned during the pre-training phase. Furthermore, we employ a semantic margin to extract class semantic correlations from textual descriptions, which are then utilized in enhancing image representation discriminability. Experimental results across multiple datasets demonstrate that our method significantly outperforms representative OWSSL methods in the recognition of both known and unknown classes.

Multi-modal point cloud completion, which utilizes a complete image and a partial point cloud as input, is a crucial task in 3D computer vision. Previous methods commonly employ a cross-attention mechanism to fuse point clouds and images. However, these approaches often fail to fully leverage image information and overlook the intrinsic geometric details of point clouds that could complement the image modality. To address these challenges, we propose an interleaved attention enhanced Transformer (IAET) with three main components, i.e., token embedding, bidirectional token supplement, and coarse-to-fine decoding. IAET incorporates a novel interleaved attention mechanism to enable bidirectional information supplementation between the point cloud and image modalities. Additionally, to maximize the use of the supplemented image information, we introduce a view-guided upsampling module that leverages image tokens as queries to guide the generation of detailed point cloud structures. Extensive experiments demonstrate the effectiveness of IAET, highlighting its state-of-the-art performance on multi-modal point cloud completion benchmarks in various scenarios. The source code is freely accessible at https://github.com/doldolOuO/IAET.

The success of current graph contrastive learning methods largely relies on the choice of data augmentation and contrastive objectives. However, most existing methods tend to optimize these two components independently, neglecting their potential interplay, which leads to suboptimal quality of the learned embeddings. To address this issue, we propose Uncertainty-guided Graph Contrastive Learning (UGCL) from a unified perspective. The core of our method is the introduction of sample uncertainty, a critical metric that quantifies the degree of class ambiguity within individual samples. On this basis, we design a novel multi-scale data augmentation strategy and a weighted graph contrastive loss function, both of which significantly enhance the quality of embeddings. Theoretically, we demonstrate that UGCL can coordinate overall optimization objectives through uncertainty, and through experiments, we show that it improves the performance of tasks such as node classification, node clustering, and link prediction, thereby verifying the effectiveness of our method.

Multimodal emotion recognition aims to identify emotions by integrating multimodal features derived from spoken utterances. However, existing work often neglects the calibration of conversational entities, focusing mainly on extracting potential intra- or cross-modal information. This leads to the underutilization of utterance information that is essential for accurately characterizing emotion. Additionally, the lack of effective modeling of conversational patterns limits the ability to capture emotional pathways across contexts, modalities and speakers, impacting the overall emotional understanding. In this study, we propose the modality-calibrated hypergraph fusion network (MATCH), which leverages multimodal fusion and hypergraph learning techniques to address these challenges. In particular, we introduce an entity calibration strategy that refines the representations of conversational entities both at the modality and context levels, allowing for deeper insights into emotion-related cues. Furthermore, we present an emotion-aligned hypergraph fusion method that incorporates a line graph to explore conversational patterns, facilitating flexible knowledge transfer across modalities through hyperedge-level and graph-level alignments. Experiments demonstrate that MATCH outperforms state-of-the-art approaches on two benchmark datasets.

Explainable recommendation aims to offer high-quality explanations for recommendation results to help users in understanding and making decisions. Benefiting from the intuitive and easy-to-understand nature of natural language, existing work primarily concentrates on text-based explainable recommendation methods with the objective of generating explanatory text. However, most of these methods only focus on textual information while neglecting the multimodal information that can provide more comprehensive preferences and characteristics. To address this issue, we propose a novel Multimodal Enhanced Explainable Recommendation (MEER) model. Specifically, we integrate collaborative features and multimodal features into a multitask learning framework, jointly optimizing the tasks of rating prediction and explanation generation. Firstly, the pre-trained model CLIP is utilized to extract multimodal features. Subsequently, we adopt an attention mechanism to fuse the collaborative and multimodal features. To fully model the cross-modal feature correlations, we introduce a contrastive learning paradigm to constrain the fused features. Furthermore, we developed a personalized multimodal Transformer decoder, which is capable of comprehensively modeling both collaborative information and multimodal information. Experimental results on three real-world datasets demonstrate that our proposed method has achieved competitive performance in both recommendation accuracy and explanation quality.

Many confounding-resistant domain generalization methods for image classification have been developed based on causal interventions. However, their reliance on strong assumptions limits their effectiveness in handling unobserved confounders. Although recent work introduces instrumental variables (IVs) to overcome this limitation, the reliance on manually predefined instruments, particularly in the context of visual data, may result in severe bias or invalidity when IV conditions are violated. To address these issues, we propose a novel approach to automatically learning Visual Instrumental Variables for confounding-resistant Domain Generalization (VIV-DG). We observe that certain non-causal visual attributes in image data naturally satisfy the basic conditions required for valid IVs. Motivated by this insight, we propose the visual instrumental variable, a novel concept that extends classical IV theory to the visual domain. Furthermore, we develop an automatic visual instrumental variable learner that enforces IV conditions on learned representations, enabling the automatic learning of valid visual instrumental variables from image data. Ultimately, VIV-DG inherits the strengths of classical IVs to mitigate unobserved confounding and avoids the significant bias caused by violations of IV conditions in predefined IVs. Extensive experiments on multiple benchmarks verify that VIV-DG achieves superior generalization ability.

Existing early and late fusion frameworks in multimodal learning are confronted with the fundamental challenge of modality imbalance, wherein disparities in representational capacities induce inter-modal competition during training. Current research methodologies primarily rely on modality-level contribution assessments to measure gaps in representational capabilities and enhance poorly learned modalities, overlooking the dynamic variations of modality contributions across individual samples. To address this, we propose a Causal-aware Modality valuation approach for Balanced multimodal learning (CMoB). We define a benefit function based on Shannon's theory of informational uncertainty to evaluate the changes in the importance of samples across different stages of multimodal training. Inspired by human cognitive science, we propose a causal-aware modality contribution quantification method from a causal perspective to capture fine-grained changes in modality contribution degrees within samples. In the iterative training of multimodal learning, we develop targeted modal enhancement strategies that dynamically select and optimize modalities based on real-time evaluation of their contribution variations across training samples. Our method enhances the discriminative ability of key modalities and the learning capacity of weak modalities while achieving fine-grained balance in multimodal learning. Extensive experiments on benchmark multimodal datasets and multimodal frameworks demonstrate the superiority of our CMoB approach for balanced multimodal learning.

Offline reinforcement learning (RL) aims to learn optimal policies from static datasets while enhancing generalization to out-of-distribution (OOD) data. To mitigate overfitting to suboptimal behaviors in offline datasets, existing methods often relax constraints on policy and data or extract informative patterns through data-driven techniques. However, there has been limited exploration into structurally guiding the optimization process toward flatter regions of the solution space that offer better generalization. Motivated by this observation, we present \textit{FANS}, a generalization-oriented structured network framework that promotes flatter and robust policy learning by guiding the optimization trajectory through modular architectural design. FANS comprises four key components: (1) Residual Blocks, which facilitate compact and expressive representations; (2) Gaussian Activation, which promotes smoother gradients; (3) Layer Normalization, which mitigates overfitting; and (4) Ensemble Modeling, which reduces estimation variance. By integrating FANS into a standard actor-critic framework, we highlight that this remarkably simple architecture achieves superior performance across various tasks compared to many existing advanced methods. Moreover, we validate the effectiveness of FANS in mitigating overestimation and promoting generalization, demonstrating the promising potential of architectural design in advancing offline RL.

Hypergraphs offer a natural paradigm for modeling complex systems with multi-way interactions. Hypergraph neural networks (HGNNs) have demonstrated remarkable success in learning from such higher-order relational data. While such higher-order modeling enhances relational reasoning, the effectiveness of hypergraph learning remains bottlenecked by two persistent challenges: the scarcity of labeled data inherent to complex systems, and the vulnerability to structural noise in real-world interaction patterns. Traditional data augmentation methods, though successful in Euclidean and graph-structured domains, struggle to preserve the intricate balance between node features and hyperedge semantics, often disrupting the very group-wise interactions that define hypergraph value. To bridge this gap, we present HyperMixup, a hypergraph-aware augmentation framework that preserves higher-order interaction patterns through structure-guided feature mixing. Specifically, HyperMixup contains three critical components: 1) Structure-aware node pairing guided by joint feature-hyperedge similarity metrics, 2) Context-enhanced hierarchical mixing that preserves hyperedge semantics through dual-level feature fusion, and 3) Adaptive topology reconstruction mechanisms that maintain hypergraph consistency while enabling controlled diversity expansion. Theoretically, we establish that our method induces hypergraph-specific regularization effects through gradient alignment with hyperedge covariance structures, while providing robustness guarantees against combined node-hyperedge perturbations. Comprehensive experiments across diverse hypergraph learning tasks demonstrate consistent performance improvements over state-of-the-art baselines, with particular effectiveness in low-label regimes. The proposed framework advances hypergraph representation learning by unifying data augmentation with higher-order topological constraints, offering both practical utility and theoretical insights for relational machine learning.

Multi-criteria (MC) ratings as auxiliary supervisory signals can improve the prediction accuracy of recommender systems. The existing MC methods learn the representations of users and items in Euclidean space to estimate the interaction probabilities. However, this modeling paradigm ignores two important aspects. Firstly, when embedding power-law distribution data and personalized MC preferences in Euclidean space, the model may produce suboptimal solutions due to the distortion of the hierarchical structure. Secondly, the inevitable noise in MC ratings may hinder the recommendation quality of the model. To address the above issues, we propose a novel framework called Hyperbolic Multi-Criteria Recommendation (HMCR), which aims to mine users' MC behavioral features on hyperbolic manifolds and mitigate the noise interference through knowledge transfer among the criteria. Specifically, we map the representations on each criterion view to a hyperbolic space with adjustable curvature based on the Lorentz model, which is used to capture the hierarchical structure of collective user behavior. The MC preferences of individual users are fused by calculating the hyperbolic attention among each criterion and the overall rating. Moreover, we design a self-supervised contrastive loss to suppress the negative impact of noise interactions on the model. The experimental results on four real-world datasets show that the HMCR significantly outperforms the existing baselines.

Dynamic graph representation learning aims to capture the evolution of graph structures and obtain accurate node embeddings, a crucial task in graph machine learning. The Hawkes point process, a mathematical framework effective for modeling the influence of historical events on future occurrences, has been validated as a powerful tool for capturing the dynamics of graph evolution in dynamic graph representation learning. However, existing dynamic graph representation learning methods based on the Hawkes point process primarily model excitation at the individual node level, failing to adequately account for structural influences during graph evolution. This limitation restricts their ability to comprehensively capture network evolution patterns. To address this limitation, we propose a Hawkes Point Process-enhanced Dynamic Graph Neural Network (HP-DGNN) model. This model leverages the Hawkes point process to model both individual node histories and structural histories, capturing their respective influences on future node interactions. By integrating individual and structural influences in computing Hawkes conditional intensity, the model comprehensively captures the impacts of both layers on future node interactions. We evaluate our proposed model on two downstream tasks of dynamic graph representation learning: dynamic link prediction and future node degree prediction. Compared to 12 state-of-the-art methods, our model consistently demonstrates superior performance, underscoring its effectiveness in capturing the complexities of graph evolution.

Graph Neural Network (GNN) is a popular semi-supervised graph representation learning method, whose performance strongly relies on the quality and quantity of labeled nodes. Given the insufficiency of labeled nodes in many real applications, many multi-channel GNNs have been developed to extract self-supervised information by leveraging consistency and complementarity among augmented graphs from different channels. However, these methods often struggle to balance conflicting self-supervised constraints, enhancing certain types of information at the expense of others. To tackle this problem, we propose a Multi-channel Disentangled Graph Neural Network (MD-GraphNet), which effectively classifies self-supervised constraints by learning disentangled representations. Specifically, our model enforces consistency constraints for shared representations, graph reconstruction constraints for complementary (or private) representations, and aligning constraints for fused representations. Our model overcomes the confusion and loss problems of different types of self-supervised signals. Experimental results on benchmark datasets demonstrate the effectiveness of MD-GraphNet for semi-supervised node classification.

Existing clustering ensemble methods typically fuse all base clusterings in one shot under unsupervised settings, making it difficult to distinguish the quality of individual base clusterings and to exploit latent prior knowledge; consequently, their adaptability to data distributions and overall performance are limited. To address these issues, this paper proposes the Self-Constrained Clustering Ensemble (SCCE) algorithm. SCCE treats the pseudolabels automatically generated from current clustering results as selfsupervised signals and performs metric learning to obtain a linear transformation that enlarges interclass distances while compressing intraclass distances. The base clusterings are then reclustered in the new metric space to enhance separability and consistency. Afterward, ensemble updating is iteratively applied, forming a self-driven closed loop that continuously improves model performance. Theoretical analysis shows that the model converges efficiently via alternating optimization, with computational complexity on the same order as mainstream methods. Experiments on public datasets demonstrate that the proposed algorithm significantly outperforms representative clustering ensemble approaches, validating its effectiveness and robustness in scenarios lacking external supervision.

Shape from focus (SFF) is a technique used to estimate the depth of a scene from a sequence of multifocus images. Existing SFF methods can be categorized into two groups: traditional methods and deep learning-based methods. Traditional methods typically employ a focus measure (FM) operator to assess the sharpness of individual pixels in a single-frame image, often overlooking the associations within the image sequence. Deep learning methods generally rely on labeled datasets, which are often challenging to obtain in real-world scenarios. Based on these observations, we propose a novel sequence association-based (SAS) framework aimed at enhancing the generalizability of SFF methods. In the SAS framework, an image sequence is treated as complete three-dimensional (3D) data throughout the processes of multiview decomposition, selective fusion and multiscale feature aggregation. Furthermore, the framework includes a tighter multiview learning generalization error bound to guide the development of the selective fusion method. This method leverages isomorphisms among multiple views to effectively mitigate the adverse effects of outlier noise on the reconstruction of various scenes. Comprehensive experiments on seven synthetic datasets and two real scenes with unknown labels demonstrate the effectiveness and generalizability of the SAS framework compared to state-of-the-art SFF methods.

Adapting Vision Transformers (ViTs) to medical image analysis is challenging due to the scarcity of annotated data and the significant domain shift from natural to medical images. Traditional fine-tuning approaches, while effective, require storing separate model parameters for each task, leading to high computational costs. Existing prompt tuning methods reduce this overhead by introducing task-specific prompt tokens, but they often fail to fully leverage label semantics, resulting in suboptimal performance for medical tasks. To address these limitations, we propose a label-semantic-based prompt tuning method (LPT), which transforms the visual prompt learning problem into a text-image alignment task. Unlike traditional prompt methods that only focus on visual prompts, LPT incorporates label semantics through a cross-attention-based module to better align image features with the target labels. This approach not only captures rich semantic information from the labels but also enhances the model’s ability to extract fine-grained image details relevant to specific medical conditions. By leveraging label-text alignment during training, LPT improves both label utilization and model adaptability, enabling more accurate predictions. Extensive experiments on eight diverse medical datasets demonstrate that LPT significantly improves diagnostic accuracy and generalization, outperforming both traditional fine-tuning and current prompt-based methods, especially in data-limited scenarios.

Contrastive Learning (CL) is an effective self-supervised learning method. It performs instance-level contrastiveness based on the image representations, which enables the model to extract abstract information from images. However, when training data is insufficient, abstract information fails to distinguish samples from different classes. This problem is more severe in the pre-training of Vision Transformer (ViT). In general, detailed information is crucial for enhancing the discrimination of representations. Patch representations, which focus on the details of images, are often overlooked in existing methods that train ViT through CL, resulting in the confusion of similar samples. To address this problem, we propose a Contrastive Learning model with Enhancing Detailed Information (CL-EDI) for pre-training ViT. Our model consists of dual ViT contrastive modules. The first module is similar to MoCo V3, which can learn abstract information about images. The role of the second ViT contrastive module is to enhance detailed information in data representations by aggregating patch representations of images. Extensive experiments demonstrate the necessity of learning detailed information. Across several datasets, our model surpasses existing approaches in image classification, transfer learning and object detection tasks.

Partial multilabel learning is a pivotal area in machine learning that tackles scenarios where training instances are annotated with a set of candidate labels, only a subset of which is relevant. Existing approaches typically rely on global-level feature learning or noise disambiguation; however, they often struggle to effectively capture the multigranularity relationships inherent in feature and label spaces, and tend to overlook critical intrafeature information essential for accurate label discrimination. To address these limitations, we propose a novel partial multilabel learning framework based on a dynamic coarse-to-fine granularity feature aggregation strategy, which hierarchically extracts feature representations across multiple levels of granularity and dynamically emphasizes label-relevant feature components. Specifically, the dynamic fine-granularity graph captures label-specific local information by modeling the fuzzy aggregations among fine-granularity feature components, while the dynamic coarse-granularity graph learns adaptive label representations by identifying feature-aware correlations of labels and suppressing noise. By jointly leveraging these two complementary granularity levels, the model effectively integrates multilevel semantic relationships and enhances the overall discriminative capacity of the learned features. Extensive experiments conducted on benchmark datasets under varying noise conditions demonstrate that the proposed method consistently outperforms state-of-the-art approaches in partial multilabel classification.

Establishing local semantic correspondences between medical images and their corresponding reports is crucial for effective medical vision-language pre-training. However, existing methods encounter two major challenges: (1) lesion regions in radiological images are often small, blurry, or lack clear boundaries, complicating accurate localization; and (2) medical reports typically contain redundant or non-diagnostic words, hindering precise semantic alignment. To overcome these issues, we propose MedAligner, a specialized local alignment network for medical vision-language pre-training. MedAligner employs dual encoders to extract both global and local representations and uses global contrastive learning to maintain coarse semantic consistency. To enhance local alignment, we introduce a Word-Region Alignment, which generates a learnable word-pixel similarity matrix that is sparsified to identify salient lesion regions accurately. Additionally, our Diagnostic Term Filtering dynamically samples high-importance diagnostic terms from reports, aligning them with identified lesion areas via a local contrastive loss. Importantly, we adopt a progressive training strategy that gradually refines both the input text and semantic alignment. This is achieved by reconstructing concise diagnostic reports and progressively updating word-pixel similarity, generating increasingly accurate image-text pairs. Extensive experiments demonstrate that MedAligner significantly surpasses existing approaches on tasks such as phrase grounding, image-text retrieval, and zero-shot classification, setting new benchmarks in medical vision-language pre-training.

Clustering is a fundamental technique widely used for exploring the inherent data structure. Many studies indicate that an appropriate feature representation can effectively improve clustering performance. However, the existing feature representation methods are based on correlation to select or extract features, which makes it hard to deal with spurious correlations. The spurious correlations mislead the correlation-based methods to consider features that have no causal relationship as being correlative, which limits the clustering performance and feature interpretability. To tackle this issue, inspired by causal learning, we propose a new joint optimization Clustering Method with Causal Feature Embedding (CM-CaFE) , which utilizes the causality of features to learn more discriminative representation for clustering. Specifically, to eliminate spurious correlations among features, we first employ any state-of-the-art Markov blanket learning method to learn an undirected causal graph. Next, we extract the maximal fully connected causal subgraphs from the learned undirected causal graph and propose an approach to merge them to generate the causal matrix. Based on the causal matrix, we present an objective function that consists of a clustering loss term and a causal matrix fitting term to learn a causal transformation matrix. The causal transformation matrix is utilized to map the original data into a new space for clustering. Finally, we comprehensively compare the proposed method with some state-of-the-art clustering approaches on several datasets to demonstrate the effectiveness and interpretability of the proposed method.

Feature weighting aims to assign different weights to features based on their importance in machine learning tasks. In clustering tasks, the existing methods learn feature importance based on the clustering results derived from the collaborative contribution of all features, which overlooks the independent effect of each feature. In fact, there are underlying causal relationships between features and the clustering results, and the features with high causal effects are always more crucial for clustering. Therefore, we propose an enhancedFeatureWeighting method viaCausalEffect forClustering, calculating the causal effect of each feature on the clustering results for obtaining the independent contribution of each feature. Specifically, we start by identifying the causal relationships among the features and utilizing the causal relationships to generate a reasonable treatment group. Next, we compare the changes in the data distribution between the treatment and control groups to determine the causal effect of each feature. Finally, the causal effects of features are used for enhancing the clustering-driven weight learning. Moreover, we present a theory of relative order consistency in causal effect. Experimental results demonstrate that utilizing causal effect in weight learning facilitates efficient convergence and achieves superior accuracy compared to state-of-the-art clustering algorithms.

Asynchronous pipeline model parallelism with a “1F1B” (one forward, one backward) schedule generates little bubble overhead and always provides quite a high throughput. However, the “1F1B” schedule inevitably leads to weight inconsistency and weight staleness issues due to the cross-training of different mini-batches across GPUs. To simultaneously address these two problems, in this paper, we propose an optimizer-dependent weight prediction strategy (a.k.a PipeOptim) for asynchronous pipeline training. The key insight of our proposal is that we employ a weight prediction strategy in the forward pass to approximately ensure that each mini-batch uses consistent and staleness-free weights to compute the forward pass of the “1F1B” schedule. To be concrete, we first construct the weight prediction scheme based on the update rule of the used optimizer when training the deep neural network models. Then throughout the “1F1B” pipeline training, each mini-batch is mandated to execute weight prediction, subsequently employing the predicted weights to perform the forward pass. As a result, PipeOptim 1) inherits the advantage of the “1F1B” schedule and generates high throughput, and 2) can ensure effective parameter learning regardless of the type of the used optimizer. We conducted extensive experimental evaluations using nine different deep-learning models to verify the effectiveness of our proposal. The experiment results demonstrate that PipeOptim outperforms the other five popular pipeline approaches including GPipe, PipeDream, PipeDream-2BW, SpecTrain, and XPipe.

As a promising strategy to achieve generalizable graph learning tasks, graph invariant learning emphasizes identifying invariant subgraphs for stable predictions on biased unknown distribution by selecting the important edges/nodes based on their contributions to the predictive tasks (i.e., subgraph predictivity). However, the existing approaches solely relying on subgraph predictivity face a challenge: the learned invariant subgraph often contains numerous spurious nodes and shows poor connectivity, undermining the generalization power of Graph Neural Networks (GNNs). To tackle this issue, we propose a summary graph-induced Invariant Learning (SIL) model that innovatively adopts a summary graph to leverage both the subgraph connectivity and predictivity for learning strong connected and accurate invariant subgraphs. Specifically, SIL first learns a summary graph containing multiple strongly connected supernodes while maintaining structure consistency with the original graph. Second, the learned summary graph is disentangled into an invariant supernode and spurious counterparts to eliminate the interference of highly predictive edges and nodes. Finally, SIL identifies a potential invariant subgraph from the invariant supernode to accomplish generalization tasks. Additionally, we provide a theoretical analysis of the summary graph learning mechanism, guaranteeing that the learned summary graph is consistent with the original graph. Experimental results validate the effectiveness of the SIL model.

Attributed Question Answering (AQA) aims to provide both a trustworthy answer and a reliable attribution report for a given question. Retrieval is a widely adopted approach, including two general paradigms: Retrieval-Then-Read (RTR) and post-hoc retrieval. Recently, Large Language Models (LLMs) have shown remarkable proficiency, prompting growing interest in AQA among researchers. However, RTR-based AQA often suffers from irrelevant knowledge and rapidly changing information, even when LLMs are adopted, while post-hoc retrievalbased AQA struggles with comprehending long-form answers with complex logic, and precisely identifying the content needing revision and preserving the original intent. To tackle these problems, this paper proposes an Atomic fact decompositionbased Retrieval and Editing (ARE) framework, which decomposes the generated long-form answers into molecular clauses and atomic facts by the instruction-tuned LLMs. Notably, the instruction-tuned LLMs are fine-tuned using a well-constructed dataset, generated from large scale Knowledge Graphs (KGs). This process involves extracting one-hop neighbors from a given set of entities and transforming the result into coherent long-form text. Subsequently, ARE leverages a search engine to retrieve evidences related to atomic facts, inputting these evidences into an LLM-based verifier to determine whether the facts require expansion for re-retrieval or editing. Furthermore, the edited facts are backtracked into the original answer, with evidence aggregated based on the relationship between molecular clauses and atomic facts. Extensive evaluations demonstrate the superior performance of our proposed method over the state-of-the-arts on several datasets, with an additionally proposed new metricAttrpfor evaluating the precision of evidence attribution.

Graph Contrastive Learning (GCL) plays a crucial role in multimedia applications due to its effectiveness in analyzing graph-structured data. Existing GCL methods focus on maximizing the agreement of node representations across different augmentations, which leads to the neglect of unique and complementary information in each augmentation. In this paper, we propose a fusion-based GCL model (FB-GCL) that learns fused representations to effectively capture complementary information from both the graph structure and node attributes. Our model consists of two modules: a graph fusion encoder and a graph contrastive module. The graph fusion encoder adaptively fuses the representations learned from the topology graph and the attribute graph. The graph contrastive module extracts supervision signals from the raw graph by leveraging both the pairwise relationships within the graph structure and the multi-label information from the attributes. Extensive experiments on seven benchmark datasets demonstrate that FB-GCL enhances performance in node classification and link prediction tasks. This improvement is especially valuable for multimedia data analysis, as integrating graph structure and attribute information is crucial for effectively understanding and processing complex datasets.

The increasing interest in learning from paired medical images and textual reports highlights the need for methods that can achieve multi-grained alignment between these two modalities. However, most existing approaches overlook finegrained semantic alignment, which can constrain the quality of the generated representations. To tackle this problem, we propose the Multi-Grained Vision-and-Language Alignment (MGVLA) model, which effectively leverages multi-grained correspondences between medical images and texts at different levels, including disease, instance, and token levels. For disease-level alignment, our approach adopts the concept of contrastive learning and uses medical terminologies detected from textual reports as soft labels to guide the alignment process. At the instance level, we propose a strategy for sampling hard negatives, where images and texts with the same disease type but differing in details such as disease locations and severity are considered as hard negatives. This strategy helps our approach to better distinguish between positive and negative image-text pairs, ultimately enhancing the quality of our learned representations. For token-level alignment, we employ a masking and recovery technique to achieve finegrained semantic alignment between patches and sub-words. This approach effectively aligns the different levels of granularity between the image and language modalities. To assess the efficacy of our MGVLA model, we conduct comprehensive experiments on the image-text retrieval and phrase grounding tasks.

In this article, a new unsupervised contrastive clustering (CC) model is introduced, namely, image CC with self-learning pairwise constraints (ICC-SPC). This model is designed to integrate pairwise constraints into the CC process, enhancing the latent representation learning and improving clustering results for image data. The incorporation of pairwise constraints helps reduce the impact of false negatives and false positives in contrastive learning, while maintaining robust cluster discrimination. However, obtaining prior pairwise constraints from unlabeled data directly is quite challenging in unsupervised scenarios. To address this issue, ICC-SPC designs a pairwise constraints learning module. This module autonomously learns pairwise constraints among data samples by leveraging consensus information between latent representation and pseudo-labels, which are generated by the clustering algorithm. Consequently, there is no requirement for labeled images, offering a practical resolution to the challenge posed by the lack of sufficient supervised information in unsupervised clustering tasks. ICC-SPC's effectiveness is validated through evaluations on multiple benchmark datasets. This contribution is significant, as we present a novel framework for unsupervised clustering by integrating contrastive learning with self-learning pairwise constraints.

The bagging method has received much application and attention in recent years due to its good performance and simple framework. It has facilitated the advanced random forest method and accuracy-diversity ensemble theory. Bagging is an ensemble method based on simple random sampling (SRS) method with replacement. However, SRS is the most foundation sampling method in the field of statistics, where exists some other advanced sampling methods for probability density estimation. In imbalanced ensemble learning, down-sampling, over-sampling, and SMOTE methods have been proposed for generating base training set. However, these methods aim at changing the underlying distribution of data rather than simulating it better. The ranked set sampling (RSS) method uses auxiliary information to get more effective samples. The purpose of this article is to propose a bagging ensemble method based on RSS, which uses the ordering of objects related to the class to obtain more effective training sets. To explain its performance, we give a generalization bound of ensemble from the perspective of posterior probability estimation and Fisher information. On the basis of RSS sample having a higher Fisher information than SRS sample, the presented bound theoretically explains the better performance of RSS-Bagging. The experiments on 12 benchmark datasets demonstrate that RSS-Bagging statistically performs better than SRS-Bagging when the base classifiers are multinomial logistic regression (MLR) and support vector machine (SVM).

Recommender systems are designed to assist users in discovering interesting items and bringing profits to online platforms. The existing works primarily explore the correlation between historical feedback and model predictions through the data-driven paradigm based on a single user-item rating matrix (i.e., overall rating). However, this single-criterion methods ignore the users’ multi-criteria (MC) behavioral characteristics. For example, a hotel system allows users to rate from multiple dimensions, such as environment and location (i.e., MC ratings). Moreover, selection bias is pervasive in user behavior data. Traditional data-driven methods may induce spurious association and amplified biases. To address the above challenges, we propose a debiasing framework called Multi-Criteria Causal Recommendation (MCCR), which encapsulates users’ diverse MC preferences and employs causal inference to construct novel training and inference strategies. Specifically, we first represent the causal relationships among variables in MC scenarios through the structural causal model. Then, we mitigate the negative impact of selection bias through the back-door adjustment. Next, a graph representation learning framework suitable for MC ratings is developed, which is used to extract higher-order information and infer the heterogeneity of users’ preferences with different criteria. Experimental results on six real datasets demonstrate that the MCCR significantly outperforms the existing baselines.

Event Argument Extraction (EAE) aims to extract arguments for specified events from a text.Previous research has mainly focused on addressing long-distance dependencies of arguments, modeling co-occurrence relationships between roles and events, but overlooking potential inductive biases: (i) semantic differences among arguments of the same type and (ii) large margin separation between arguments of the different types.Inspired by prototype networks, we introduce a new model named HMPEAE, which takes the two inductive biases above as targets to locate prototypes and guide the model to learn argument representations based on these prototypes.Specifically, we set multiple prototypes to represent each role to capture intra-class differences.Simultaneously, we use hypersphere as the output space for prototypes, defining large margin separation between prototypes to encourage the model to learn significant differences between different types of arguments effectively.We solve the "argument-prototype" assignment as an optimal transport problem to optimize the argument representation and minimize the absolute distance between arguments and prototypes to achieve compactness within sub-clusters.Experimental results on the RAMS and WikiEvents datasets show that HMPEAE achieves state-of-the-art performances.

The Transformer architecture has recently gained considerable attention in the field of graph representation learning, as it naturally overcomes several limitations of Graph Neural Networks (GNNs) with customized attention mechanisms or positional and structural encodings. Despite making some progress, existing works tend to overlook external information of graphs, specifically the correlation between graphs. Intuitively, graphs with similar structures should have similar representations. Therefore, we propose Graph External Attention (GEA) --- a novel attention mechanism that leverages multiple external node/edge key-value units to capture inter-graph correlations implicitly. On this basis, we design an effective architecture called Graph External Attention Enhanced Transformer (GEAET), which integrates local structure and global interaction information for more comprehensive graph representations. Extensive experiments on benchmark datasets demonstrate that GEAET achieves state-of-the-art empirical performance. The source code is available for reproducibility at: https://github.com/icm1018/GEAET.

Offline Reinforcement Learning (RL) commonly suffers from the out-of-distribution (OOD) overestimation issue due to the distribution shift. Prior work gradually shifts their focus from suppressing OOD overestimation to avoiding overly conservative learning from suboptimal behavior policies to improve generalization. However, most approaches explicitly delimit boundaries for OOD actions based on the support in the dataset, which can potentially impede the data near these boundaries from acquiring realistic estimates. This paper investigates how to loosen the rigid demarcation of OOD boundaries, adaptively extracting knowledge from empirical data to implicitly improve the model's generalization to nearby unseen data. We introduce an adversarial data splitting (ADS) framework that enforces the model to generalize the distribution shifts simulated from the train/validation subsets splitting of the dataset. Specifically, ADS is modeled as a min-max optimization problem inspired by meta-learning and solved by iterating over the following two steps. First, we train the model on the train-subset to minimize its loss on the validation-subset. Then, we adversarially generate the "hardest" train/validation subsets with the maximum distribution shift, making the model incapable of generalization at that splitting. We derive a generalization error bound for theoretically understanding ADS and verify the effectiveness with extensive experiments. Code is available at https://github.com/DkING-lv6/ADS.

Zero-shot node classification is a vital task in the field of graph data processing, aiming to identify nodes of classes unseen during the training process. Prediction bias is one of the primary challenges in zero-shot node classification, referring to the model's propensity to misclassify nodes of unseen classes as seen classes. However, most methods introduce external knowledge to mitigate the bias, inadequately leveraging the inherent cluster information within the unlabeled nodes. To address this issue, we employ spectral analysis coupled with learnable class prototypes to discover the implicit cluster structures within the graph, providing a more comprehensive understanding of classes. In this paper, we propose a spectral approach for zero-shot node classification (SpeAr). Specifically, we establish an approximate relationship between minimizing the spectral contrastive loss and performing spectral decomposition on the graph, thereby enabling effective node characterization through loss minimization. Subsequently, the class prototypes are iteratively refined based on the learned node representations, initialized with the semantic vectors. Finally, extensive experiments verify the effectiveness of the SpeAr, which can further alleviate the bias problem.

A challenging problem in seeking to bring multi-agent reinforcement learning (MARL) techniques into real-world applications, such as autonomous driving and drone swarms, is how to control multiple agents safely and cooperatively to accomplish tasks. Most existing safe MARL methods learn the centralized value function by introducing a global state to guide safety cooperation. However, the global coupling arising from agents’ safety constraints and the exponential growth of the state-action space size limit their applicability in instant communication or computing resource-constrained systems and larger multi-agent systems. In this paper, we develop a novel scalable and theoretically-justified multi-agent constrained policy optimization method. This method utilizes the rigorous bounds of the trust region method and the bounds of the truncated advantage function to provide a new local policy optimization objective for each agent. Also, we prove that the safety constraints and the joint policy improvement can be met when each agent adopts a sequential update scheme to optimize a $\kappa$-hop policy. Then, we propose a practical algorithm called Scalable MAPPO-Lagrangian (Scal-MAPPO-L). The proposed method’s effectiveness is verified on a collection of benchmark tasks, and the results support our theory that decentralized training with local interactions can still improve reward performance and satisfy safe constraints.

In point cloud, some regions typically exist nodes from multiple categories, i.e., these regions have both homophilic and heterophilic nodes. However, most existing methods ignore the heterophily of edges during the aggregation of the neighborhood node features, which inevitably mixes unnecessary information of heterophilic nodes and leads to blurred boundaries of segmentation. To address this problem, we model the point cloud as a homophilic-heterophilic graph and propose a graph regulation network (GRN) to produce finer segmentation boundaries. The proposed method can adaptively adjust the propagation mechanism with the degree of neighborhood homophily. Moreover, we build a prototype feature extraction module, which is utilised to mine the homophily features of nodes from the global prototype space. Theoretically, we prove that our convolution operation can constrain the similarity of representations between nodes based on their degree of homophily. Extensive experiments on fully and weakly supervised point cloud semantic segmentation tasks demonstrate that our method achieves satisfactory performance. Especially in the case of weak supervision, that is, each sample has only 1%-10% labeled points, the proposed method has a significant improvement in segmentation performance.

Commonsense question answering (CQA) generally means that the machine uses its mastered commonsense to answer questions without relevant background material, which is a challenging task in natural language processing. Existing methods focus on retrieving relevant subgraphs from knowledge graphs based on key entities and designing complex graph neural networks to perform reasoning over the subgraphs. However, they have the following problems: i) the nested entities in key entities lead to the introduction of irrelevant knowledge; ii) the QA context is not well integrated with the subgraphs; and iii) insufficient context knowledge hinders subgraph nodes understanding. In this paper, we present a heterogeneous-graph reasoning with context paraphrase method (HCP), which introduces the paraphrase knowledge from the dictionary into key entity recognition and subgraphs construction, and effectively fuses QA context and subgraphs during the encoding phase of the pre-trained language model (PTLM). Specifically, HCP filters the nested entities through the dictionary's vocabulary and constructs the Heterogeneous Path-Paraphrase (HPP) graph by connecting the paraphrase descriptions11The paraphrase descriptions are English explanations of words or phrases in WordNet and Wiktionary.with the key entity nodes in the subgraphs. Then, by constructing the visible matrices in the PTLM encoding phase, we fuse the QA context representation into the HPP graph. Finally, to get the answer, we perform reasoning on the HPP graph by Mask Self-Attention. Experimental results on CommonsenseQA and OpenBookQA show that fusing QA context with HPP graph in the encoding stage and enhancing the HPP graph representation by using context paraphrase can improve the machine's commonsense reasoning ability.

Causal chain graphs model the dependency structure between individuals when the assumption of individual independence in causal inference is violated. However, causal chain graphs are often unknown in practice and require learning from data. Existing learning algorithms have certain limitations. Specifically, learning local information requires multiple subset searches, building the skeleton requires additional conditional independence testing, and directing the edges requires obtaining local information from the skeleton again. To remedy these problems, we propose a novel algorithm for learning causal chain graph structure. The algorithm alternately learns the adjacencies and spouses of each variable as local information and doubly prunes them to obtain more accurate local information, which reduces subset searches, improves its accuracy, and facilitates subsequent learning. It then directly constructs the chain graphs skeleton using the learned adjacencies without conditional independence testing. Finally, it directs the edges of complexes using the learned adjacencies and spouses to learn chain graphs without reacquiring local information, further improving its efficiency. We conduct theoretical analysis to prove the correctness of our algorithm and compare it with the state-of-the-art algorithms on synthetic and real-world datasets. The experimental results demonstrate our algorithm is more reliable than its rivals.

Improving the generalization performance of classifiers from data pre-processing perspective has recently received considerable attention in the machine learning community. Although many methods have been proposed in the past decades, most of them lack theoretical foundations and cannot guarantee better generalization performance of classifiers on processed datasets. To overcome this flaw, in this paper, we propose a method, which is supported by Bayesian decision theory and percolation theory, to improve generalization performance by removing Bayesian confusing samples (abbr. BCS). Specifically, for a training set, we define the samples that misclassified by the Bayesian optimal classifier as BCS and prove that a classifier trained on the training set after removing BCS can obtain better generalization performance. To find out BCS, we indicate that BCS can be identified according to the size of global homogeneous cluster, a set of samples with the same labels, based on percolation theory. Based on these analysis, we propose a method to construct global homogeneous clusters and remove BCS from the training set. Extensive experiments show that the proposed method is effective for a number of classical and state-of-the-art classifiers.

Inferring causal structures from experimentation is a challenging task in many fields. Most causal structure learning algorithms with unknown interventions are proposed to discover causal relationships over an identical variable set. However, often due to privacy, ethical, financial, and practical concerns, the variable sets observed by multiple sources or domains are not entirely identical. While a few algorithms are proposed to handle the partially overlapping variable sets, they focus on the case of known intervention targets. Therefore, to be close to the real-world environment, we consider discovering causal relationships over overlapping variable sets under the unknown intervention setting and exploring a scenario where a problem is studied across multiple domains. Here, we propose an algorithm for discovering the causal relationships over the integrated set of variables from unknown interventions, mainly handling the entangled inconsistencies caused by the incomplete observation of variables and unknown intervention targets. Specifically, we first distinguish two types of inconsistencies and then deal with respectively them by presenting some lemmas. Finally, we construct a fusion rule to combine learned structures of multiple domains, obtaining the final structures over the integrated set of variables. Theoretical analysis and experimental results on synthetic, benchmark, and real-world datasets have verified the effectiveness of the proposed algorithm.

Causal structure learning (CSL) from observational data is a crucial objective in various machine learning applications. Recent advances in CSL have focused on local-to-global learning, which offers improved efficiency and accuracy. The local-to-global CSL algorithms first learn the local skeleton of each variable in a dataset, then construct the global skeleton by combining these local skeletons, and finally orient edges to infer causality. However, data quality issues such as noise and small samples often result in the presence of problematicasymmetric edgesduring global skeleton construction, hindering the creation of a high-quality global skeleton. To address this challenge, we propose a novel local-to-global CSL algorithm with a progressive enhancement strategy and make the following novel contributions: 1) To construct an accurate global skeleton, we design a novel strategy to iteratively correctasymmetric edgesand progressively improve the accuracy of the global skeleton. 2) Based on the learned accurate global skeleton, we design an integrated global skeleton orientation strategy to infer the correct directions of edges for obtaining an accurate and reliable causal structure. Extensive experiments demonstrate that our method achieves better performance than the existing CSL methods.

Graph Neural Networks (GNNs) have attracted significant research interest in various graph data modeling tasks. To advance trustworthy, reliable, and safe Artificial Intelligence (AI) systems for practical applications, adversarial robustness learning on GNNs has drawn widespread attention among researchers. Numerous attack methods, including white-box attacks, gray-box attacks, and black-box attacks, have been proposed, but black-box attacks are widely considered to be the most challenging and practical in real-world applications. In this paper, we focus on the challenging and realistic black-box attack scenario on GNNs, where the attacker has no information about the structure and parameters of the target model. We first theoretically demonstrate that the loss changes of the GNNs are related to the node voting matrix, which is subject to the graph topology information and is independent to the structures of GNNs. Then, we propose a novel black-box attack strategy for GNNs based on the theoretical results, i.e., node voting influence-based GNNs black-box adversarial attack, named VoteAttack. Specifically, the VoteAttack algorithm iteratively chooses a group of significant nodes based on mutual voting among nodes (the node voting matrix) and considers the voting weights among nodes. Furthermore, the VoteAttack algorithm modifies the attributes of the selected nodes to create a perturbed graph and ultimately utilizes the perturbed graph to attack GNNs. Experimental results on popular GNNs and graph datasets indicate that the proposed attack strategy outperforms baseline strategies.

Electronic health records (EHRs) contain vast medical information like diagnosis, medication, and procedures, enabling personalized drug recommendations and treatment adjustments. However, current drug recommendation methods only model patients' health conditions from EHR data, neglecting the rich relationships within the data. This paper seeks to utilize a heterogeneous information network (HIN) to represent EHR and develop a graph representation learning method for medication recommendation. However, three critical issues need to be investigated: (1) co-occurrence of diagnosis and drug for the same patient does not imply their relevance; (2) patients' directly associated information may not be sufficient to reflect their health conditions; and (3) the cold start problem exists when patients have no historical EHRs. To tackle these challenges, we develop a bi-channel heterogeneous local structural encoder to decouple and extract the diverse information in HIN. Additionally, a global information capture and fusion module, aggregating meta-paths to form a global representation, is introduced to fill the information gaps in records. A longitudinal model using rich structural information available in EHR data is proposed for drug recommendations to new patients. Experimental results on real-world EHR data demonstrate significant improvements over existing approaches.

Consensus clustering is to find a high quality and robust partition that is in agreement with multiple existing base clusterings. However, its computational cost is often very expensive and the quality of the final clustering is easily affected by uncertain consensus relations between clusters. In order to solve these problems, we develop a new -type algorithm, called -relations-based consensus clustering with double entropy-norm regularizers (KRCC-DE). In this algorithm, we build an optimization model to learn a consensus-relation matrix between final and base clusters and employ double entropy-norm regularizers to control the distribution of these consensus relations, which can reduce the impact of the uncertain consensus relations. The proposed algorithm uses an iterative strategy with strict updating formulas to get the optimal solution. Since its computation complexity is linear with the number of objects, base clusters, or final clusters, it can take low computational costs to effectively solve the consensus clustering problem. In experimental analysis, we compared the proposed algorithm with other -type-based and global-search consensus clustering algorithms on benchmark datasets. The experimental results illustrate that the proposed algorithm can balance the quality of the final clustering and its computational cost well.

Few-shot knowledge graph completion (FKGC), which aims to infer new triples for a relation using only a few reference triples of the relation, has attracted much attention in recent years. Most existing FKGC methods learn a transferable embedding space, where entity pairs belonging to the same relations are close to each other. In real-world knowledge graphs (KGs), however, some relations may involve multiple semantics, and their entity pairs are not always close due to having different meanings. Hence, the existing FKGC methods may yield suboptimal performance when handling multiple semantic relations in the few-shot scenario. To solve this problem, we propose a new method named adaptive prototype interaction network (APINet) for FKGC. Our model consists of two major components: 1) an interaction attention encoder (InterAE) to capture the underlying relational semantics of entity pairs by modeling the interactive information between head and tail entities and 2) an adaptive prototype net (APNet) to generate relation prototypes adaptive to different query triples by extracting query-relevant reference pairs and reducing the data inconsistency between support and query sets. Experimental results on two public datasets demonstrate that APINet outperforms several state-of-the-art FKGC methods. The ablation study demonstrates the rationality and effectiveness of each component of APINet.

Graph neural networks (GNNs) have made great progress in graph-based semi-supervised learning (GSSL). However, most existing GNNs are confronted with the oversmoothing issue that limits their expressive ability. A key factor that leads to this problem is the excessive aggregation of information from other classes when updating the node representation. To alleviate this limitation, we propose an effective method called GUIded Dropout over Edges (GUIDE) for training deep GNNs. The core of the method is to reduce the influence of nodes from other classes by removing a certain number of inter-class edges. In GUIDE, we drop edges according to the edge strength, which is defined as the time an edge acts as a bridge along the shortest path between node pairs. We find that the stronger the edge strength, the more likely it is to be an inter-class edge. In this way, GUIDE can drop more inter-class edges and keep more intra-class edges. Therefore, nodes in the same community or class are more similar, whereas different classes are more separated in the embedded space. In addition, we perform some theoretical analysis of the proposed method, which explains why it is effective in alleviating the oversmoothing problem. To validate its rationality and effectiveness, we conduct experiments on six public benchmarks with different GNNs backbones. Experimental results demonstrate that GUIDE consistently outperforms state-of-the-art methods in both shallow and deep GNNs.

Recently, knowledge graphs (KGs) have won noteworthy success in commonsense question answering.Existing methods retrieve relevant subgraphs in the KGs through key entities and reason about the answer with language models (LMs) and graph neural networks.However, they ignore (i) optimizing the knowledge representation and structure of subgraphs and (ii) deeply fusing heterogeneous QA context with subgraphs.In this paper, we propose a dynamic heterogeneous-graph reasoning method with LMs and knowledge representation learning (DHLK), which constructs a heterogeneous knowledge graph (HKG) based on multiple knowledge sources and optimizes the structure and knowledge representation of the HKG using a two-stage pruning strategy and knowledge representation learning (KRL).It then performs joint reasoning by LMs and Relation Mask Self-Attention (RMSA).Specifically, DHLK filters key entities based on the dictionary vocabulary to achieve the first-stage pruning while incorporating the paraphrases in the dictionary into the subgraph to construct the HKG.Then, DHLK encodes and fuses the QA context and HKG using LM, and dynamically removes irrelevant KG entities based on the attention weights of LM for the second-stage pruning.Finally, DHLK introduces KRL to optimize the knowledge representation and perform answer reasoning on the HKG by RMSA.We evaluate DHLK at CommonsenseQA and OpenBookQA, and show its improvement on existing LM and LM+KG methods.

Compositional zero-shot learning (CZSL) aims to distinguish images from unseen compositional classes, which consist of state and object concepts that individually appear in some seen compositional images. The key challenge of CZSL is how to effectively mitigate the contextuality issue for achieving a desirable compositional transfer from seen classes to unseen ones. In CZSL, the visual appearances of the same state are inconsistent when combined with different objects. To address the above dilemma, we propose a swap-reconstruction autoencoder (SRA) to capture the intrinsic context of the ambiguous states. Specifically, SRA learns a consistent embedding space for multi-modal data. A swap-reconstruction mechanism is designed to disentangle the visual embedding of states and objects. The loss including a superclass-oriented state swap-reconstruction loss and object swap-reconstruction loss model the contextual relationship between states and objects. Extensive experiments demonstrate that SRA outperforms current state-of-the-art methods on the three benchmark datasets.

The generalization performance of machine learning methods depends heavily on the quality of data representation. However, existing researches rarely consider representation learning from the perspective of generalization error. In this paper, we prove that generalization error of representation learning function can be estimated effectively by solving two convex optimization problems. Based on it, we propose a general representation learning framework. And then, we apply the proposed framework to two most commonly used nonlinear mapping methods, i.e., kernel based method and deep neural network (DNN), and thus design a kernel selection method and a DNN boosting framework, correspondingly. Finally, extensive experiments verify the effectiveness of the proposed methods.

We investigate the expressive power of depth-2 bandlimited random neural networks. A random net is a neural network where the hidden layer parameters are frozen with random assignment, and only the output layer parameters are trained by loss minimization. Using random weights for a hidden layer is an effective method to avoid non-convex optimization in standard gradient descent learning. It has also been adopted in recent deep learning theories. Despite the well-known fact that a neural network is a universal approximator, in this study, we mathematically show that when hidden parameters are distributed in a bounded domain, the network may not achieve zero approximation error. In particular, we derive a new nontrivial approximation error lower bound. The proof utilizes the technique of ridgelet analysis, a harmonic analysis method designed for neural networks. This method is inspired by fundamental principles in classical signal processing, specifically the idea that signals with limited bandwidth may not always be able to perfectly reconstruct the original signal. We corroborate our theoretical results with various simulation studies, and generally, two main take-home messages are offered: (i) Not any distribution for selecting random weights is feasible to build a universal approximator; (ii) A suitable assignment of random weights exists but to some degree is associated with the complexity of the target function.

Reinforcement learning (RL) has made significant progress in areas such as Atari games and robotic control, where the agents have perfect sensing capabilities. However, in many real-world sequential decision-making tasks, the observation data could be noisy or incomplete due to the intrinsic low quality of the sensors or unexpected malfunctions; that is, the agent’s perceptions are rarely perfect. The current POMDP RL methods, such as particle-based and Gaussian-based, can only provide a probability estimate of hidden states rather than certain belief regions, which may lead to inefficient and even wrong decision-making. This paper proposes a novel algorithm called Set-membership Belief state-based Reinforcement Learning (SBRL), which consists of two parts: a Set-membership Belief state learning Model (SBM) for learning bounded belief state sets and an RL controller for making decisions based on SBM. We prove that our belief estimation method can provide a series of belief state sets that always contain the true states under the unknown-but-bounded (UBB) noise. The effectiveness of the proposed method is verified on a collection of benchmark tasks, and the results show that our method outperforms the state-of-the-art methods.

Although deep learning has been extensively studied and achieved remarkable performance on single image super-resolution (SISR), existing convolutional neural networks (CNN) mainly focus on broader and deeper architecture design, ignoring the detailed information of the image itself and the potential relationship between the features. Recently, several attempts have been made to address the SISR with graph representation learning. However, existing GNN-based methods learning to deal with the SISR problem are limited to the information processing of the entire image or the relationship processing between different feature images of the same layer, ignoring the interdependence between the extracted features of different layers, which is not conducive to extracting deeper hierarchical features. In this paper, we propose an interlayer feature representation based graph neural network for image super-resolution (LSGNN), which consists of a layer feature graph representation learning module and a channel spatial attention module. The layer feature graph representation learning module mainly captures the interdependence between the features of different layers, which can learn more fine-grained image detail features. In addition, we also unified a channel attention module and a spatial attention module into our model, which takes into account the channel dimension information and spatial scale information, to improve the expressive ability, and achieve high quality image details. Extensive experiments and ablation studies demonstrate the superiority of the proposed model.

As a leading graph clustering technique, spectral clustering is one of the most widely used clustering methods to capture complex clusters in data. Some additional prior information can help it to further reduce the difference between its clustering results and users' expectations. However, it is hard to get the prior information under unsupervised scene to guide the clustering process. To solve this problem, we propose a self-constrained spectral clustering algorithm. In this algorithm, we extend the objective function of spectral clustering by adding pairwise and label self-constrained terms to it. We provide the theoretical analysis to show the roles of the self-constrained terms and the extensibility of the proposed algorithm. Based on the new objective function, we build an optimization model for self-constrained spectral clustering so that we can simultaneously learn the clustering results and constraints. Furthermore, we propose an iterative method to solve the new optimization problem. Compared to other existing versions of spectral clustering algorithms, the new algorithm can discover a high-quality cluster structure of a data set without prior information. Extensive experiments on benchmark data sets illustrate the effectiveness of the proposed algorithm.

For a classification task, we usually select an appropriate classifier via model selection. How to evaluate whether the chosen classifier is optimal? One can answer this question via Bayes error rate (BER). Unfortunately, estimating BER is a fundamental conundrum. Most existing BER estimators focus on giving the upper and lower bounds of the BER. However, evaluating whether the selected classifier is optimal based on these bounds is hard. In this article, we aim to learn the exact BER instead of bounds on BER. The core of our method is to transform the BER calculation problem into a noise recognition problem. Specifically, we define a type of noise called Bayes noise and prove that the proportion of Bayes noisy samples in a data set is statistically consistent with the BER of the data set. To recognize the Bayes noisy samples, we present a method consisting of two parts: selecting reliable samples based on percolation theory and then employing a label propagation algorithm to recognize the Bayes noisy samples based on the selected reliable samples. The superiority of the proposed method compared to the existing BER estimators is verified on extensive synthetic, benchmark, and image data sets.

Graph convolutional neural networks can effectively process geometric data and thus have been successfully used in point cloud data representation. However, existing graph-based methods usually adopt the K-nearest neighbor (KNN) algorithm to construct graphs, which may not be optimal for point cloud analysis tasks, owning to the solution of KNN is independent of network training. In this paper, we propose a novel graph structure learning convolutional neural network (GSLCN) for multiple point cloud analysis tasks. The fundamental concept is to propose a general graph structure learning architecture (GSL) that builds long-range and short-range dependency graphs. To learn optimal graphs that best serve to extract local features and investigate global contextual information, respectively, we integrated the GSL with the designed graph convolution operator under a unified framework. Furthermore, we design the graph structure losses with some prior knowledge to guide graph learning during network training. The main benefit is that given labels and prior knowledge are taken into account in GSLCN, providing useful supervised information to build graphs and thus facilitating the graph convolution operation for the point cloud. Experimental results on challenging benchmarks demonstrate that the proposed framework achieves excellent performance for point cloud classification, part segmentation, and semantic segmentation.

The pure accuracy measure is used to eliminate random consistency from the accuracy measure. Biases to both majority and minority classes in the pure accuracy are lower than that in the accuracy measure. In this paper, we demonstrate that compared with the accuracy measure and F-measure, the pure accuracy measure is class distribution insensitive and discriminative for good classifiers. The advantages make the pure accuracy measure suitable for traditional classification. Further, we mainly focus on two points: exploring a tighter generalization bound on pure accuracy based learning paradigm and designing a learning algorithm based on the pure accuracy measure. Particularly, with the self-bounding property, we build an algorithm-independent generalization bound on the pure accuracy measure, which is tighter than the existing bound of an order O(1/√N) (N is the number of instances). The proposed bound is free from making a smoothness or convex assumption on the hypothesis functions. In addition, we design a learning algorithm optimizing the pure accuracy measure and use it in the selective ensemble learning setting. The experiments on sixteen benchmark data sets and four image data sets demonstrate that the proposed method statistically performs better than the other eight representative benchmark algorithms.

Causal structure learning has been extensively studied and widely used in machine learning and various applications. To achieve an ideal performance, existing causal structure learning algorithms often need to centralize a large amount of data from multiple data sources. However, in the privacy-preserving setting, it is impossible to centralize data from all sources and put them together as a single dataset. To preserve data privacy, federated learning as a new learning paradigm has attached much attention in machine learning in recent years. In this paper, we study a privacy-aware causal structure learning problem in the federated setting and propose a novel federated PC (FedPC) algorithm with two new strategies for preserving data privacy without centralizing data. Specifically, we first propose a novel layer-wise aggregation strategy for a seamless adaptation of the PC algorithm into the federated learning paradigm for federated skeleton learning, then we design an effective strategy for learning consistent separation sets for federated edge orientation. The extensive experiments validate that FedPC is effective for causal structure learning in federated learning setting.

The majority of the classical dimensionality reduction methods can be unified into a graph-embedding-based framework. A fixed graph constructed in a high-dimensional space has been extensively employed in the graph-embedding-based dimensionality reduction methods. However, a fixed graph often cannot characterize the structure of high-dimensional data owing to the curse of dimensionality. To solve this problem, we combine graph construction and dimensionality reduction into a coherent framework. Thus, the constructed graph can be updated dynamically in dimensionality reduction. In the existing methods based on the coherent framework, graphs are usually constructed by a type of neighborhood relationship and single clustering result. This study proposes an unsupervised dimensionality reduction method guided by fusing multiple clustering results. In the proposed method, multiple clustering results are first obtained by the k-means algorithm, and then a graph is constructed using a weighted co-association matrix of fusing the clustering results to capture data distribution information. Based on the graph, we present an objective function of combining graph construction and dimensionality reduction to implement mutual guidance between them. Numerical experiments on real data sets illustrate that the proposed method achieves significant improvement over some representative and state-of-the-art unsupervised dimensionality reduction methods.

Graph matching endeavors to find corresponding nodes across two or more graphs, which plays a fundamental role in many vision and pattern matching tasks. However, existing graph matching algorithms often meet abnormal graphs with missing node features and suffer from numerous cluttered outliers in practical applications. To address these, we propose a novel deep graph matching method called Random Deep Graph Matching (RDGM). Different from the deterministic affinity inference in existing deep graph matching methods, RDGM performs message passing in a random manner during model training through randomly masking some available node features in the source or target graph, so that the affinity inference between nodes is insensitive to specific neighborhoods. In addition, a hierarchical attention graph neural network framework is devised in the node embedding process of RDGM, which can obtain more sufficient high-order structural information to reduce the impact of latent noise on affinity learning. Extensive experiments suggest that the proposed RDGM outperforms state-of-the-art graph matching methods, and demonstrates strong robustness and generalization performance.

We consider the problem of distinguishing direct causes from direct effects of a target variable of interest from multiple manipulated datasets with unknown manipulated variables and nonidentical data distributions. Recent studies have shown that datasets attained from manipulated experiments (i.e., manipulated data) contain richer causal information than observational data for causal structure learning. Thus, in this article, we propose a new algorithm, which makes full use of the interventional properties of a causal model to discover the direct causes and direct effects of a target variable from multiple datasets with different manipulations. It is more suited to real-world cases and is also a challenge to be addressed in this article. First, we apply the backward framework to learn parents and children (PC) of a given target from multiple manipulated datasets. Second, we orient some edges connected to the target in advance through the assumption that the target variable is not manipulated and then orient the remaining undirected edges by finding invariant V-structures from multiple datasets. Third, we analyze the correctness of the proposed algorithm. To the best of our knowledge, the proposed algorithm is the first that can identify the local causal structure of a given target from multiple manipulated datasets with unknown manipulated variables. Experimental results on standard Bayesian networks validate the effectiveness of our algorithm.

How to get a good value estimation is one of the key problems in reinforcement learning (RL). Current off-policy methods, such as Maxmin Q-learning, TD3 and TADD, suffer from the underestimation problem when solving the overestimation problem. In this paper, we propose the Quasi-Median Operation, a novel way to mitigate the underestimation bias by selecting the quasi-median from multiple state-action values. Based on the quasi-median operation, we propose Quasi-Median Q-learning (QMQ) for the discrete action tasks and Quasi-Median Delayed Deep Deterministic Policy Gradient (QMD3) for the continuous action tasks. Theoretically, the underestimation bias of our method is improved while the estimation variance is significantly reduced compared to Maxmin Q-learning, TD3 and TADD. We conduct extensive experiments on the discrete and continuous action tasks, and results show that our method outperforms the state-of-the-art methods.

In this paper, we consider the problem of lacking theoretical foundation and low execution efficiency of the instance selection methods based on the k-nearest neighbour rule when processing large-scale data. We point out that the core idea of these methods can be explained from the perspective of Bayesian decision theory, that is, to find which instances are reducible, irreducible, and deleterious. Then, based on the percolation theory, we establish the relationship between these three types of instances and local homogeneous cluster (i.e., a set of instances with the same labels). Finally, we propose a method based on an accelerated k-means algorithm to construct local homogeneous clusters and remove the superfluous instances. The performance of our method is studied on extensive synthetic and benchmark data sets. Our proposed method can handle large-scale data more effectively than the state-of-the-art instance selection methods. All code and data results are available at https://github.com/CQQXY161120/Instance-Selection.

We consider the problem of reducing the false discovery rate in multiple high-dimensional interventional datasets under unknown targets. Traditional algorithms merged directly multiple causal graphs learned, which ignores the contradictions of different datasets, leading to lots of inconsistent directions of edges. For reducing the contradictory information, we propose a new algorithm, which first learns an interventional Markov equivalence class (I-MEC) before merging multiple graphs. It utilizes the full power of the constraints available in interventional data and combines ideas from local learning, intervention, and search-and-score techniques in a principled and effective way in different intervention experiments. Specifically, local learning on multiple datasets is used to build a causal skeleton. Perfect intervention destroys some possible triangles, leading to the identification of more possible V-structures. And then a theoretically correct I-MEC is learned. Search and scoring techniques based on the learned I-MEC further identify the remaining unoriented edges. Both theoretical analysis and experiments on benchmark Bayesian networks with the number of variables from 20 to 724 validate that the effectiveness of our algorithm in reducing the false discovery rate in high-dimensional interventional data.

Zero-shot node classification is a very important challenge for classical semi-supervised node classification algorithms, such as Graph Convolutional Network (GCN) which has been widely applied to node classification. In order to predict the unlabeled nodes from unseen classes, zero-shot node classification needs to transfer knowledge from seen classes to unseen classes. It is crucial to consider the relations between the classes in zero-shot node classification. However, the GCN only considers the relations between the nodes, not the relations between the classes. Therefore, the GCN can not handle the zero-shot node classification effectively. This paper proposes a Dual Bidirectional Graph Convolutional Networks (DBiGCN) that consists of dual BiGCNs from the perspective of the nodes and the classes, respectively. The BiGCN can integrate the relations between the nodes and between the classes simultaneously in an united network. In addition, to make the dual BiGCNs work collaboratively, a label consistency loss is introduced, which can achieve mutual guidance and mutual improvement between the dual BiGCNs. Finally, the experimental results on real-world graph data sets verify the effectiveness of the proposed method.

Classification is one of the most important problems in machine learning and the nature of it is concept cognition. So far, dozens of different classifiers have been designed. Although their working mechanisms vary widely, few of them fully consider concept cognition. In this paper, a new learning machine, fuzzy learning machine (FLM), is proposed from the perspective of concept cognition. Inspired by cognitive science, its working mechanism is of strong interpretability. At the same time, FLM roots in set theory and fuzzy set theory, so FLM has a solid mathematical foundation. The systematic experimental results on a large number of data sets show that FLM can achieve excellent performance, even with the simple implementation.

Link prediction has become a significant research problem in deep learning, and the graph-based autoencoder model is one of the most important methods to solve it. The existing graph-based autoencoder models only learn a single set of distributions, which cannot accurately represent the mixed distribution in real graph data. Meanwhile, existing learning models have been greatly restricted when the graph data has insufficient attribute information and inaccurate topology information. In this paper, we propose a novel graph embedding framework, termed multi-scale variational graph autoencoder (MSVGAE), which learns multiple sets of low-dimensional vectors of different dimensions through the graph encoder to represent the mixed probability distribution of the original graph data, and performs multiple sampling in each dimension. Furthermore, a self-supervised learning strategy (i.e., graph feature reconstruction auxiliary learning) is introduced to fully use the graph attribute information to help the graph structure learning. Experiment studies on real-world graphs demonstrate that the proposed model achieves state-of-the-art performance compared with other baseline methods in link prediction tasks. Besides, the robustness analysis shows that the proposed MSVGAE method has obvious advantages in the processes of graph data with insufficient attribute information and inaccurate topology information.

Multi-modal classification (MMC) aims to integrate the complementary information from different modalities to improve classification performance. Existing MMC methods can be grouped into two categories: traditional methods and deep learning-based methods. The traditional methods often implement fusion in a low-level original space. Besides, they mostly focus on the inter-modal fusion and neglect the intra-modal fusion. Thus, the representation capacity of fused features induced by them is insufficient. The deep learning-based methods implement the fusion in a high-level feature space where the associations among features are considered, while the whole process is implicit and the fused space lacks interpretability. Based on these observations, we propose a novel interpretative association-based fusion method for MMC, named AF. In AF, both the association information and the high-order information extracted from feature space are simultaneously encoded into a new feature space to help to train an MMC model in an explicit manner. Moreover, AF is a general fusion framework, and most existing MMC methods can be embedded into it to improve their performance. Finally, the effectiveness and the generality of AF are validated on 22 datasets, four typically traditional MMC methods adopting best modality, early, late and model fusion strategies and a deep learning-based MMC method.

Graph classification aims to predict the class labels of graphs and has a wide range of applications in many real-world domains. However, most of existing graph neural networks for graph classification tasks use 90$\%$of labeled graphs for training and the remaining 10$\%$for testing, which obviously struggle in solving the problem of the scarcity of labeled graphs in real-world graph classification scenarios. And it is arduous to label a large number of graph examples for training because of the difficulty and resource consumption in the tagging process. Motivated by this, we propose a novel active and semi-supervised graph neural network (ASGNN) framework, which endeavors to complete graph classification tasks with a small number of labeled graph examples and available unlabeled graph examples. In our framework, active learning selects high-uncertain and representative graph examples from the test set and add them to the training set after annotation. Semi-supervised learning is utilized to select the high-confidence unlabeled graph examples containing structural information from the test set, and add them to the training set after pseudo labeling. To improve the generalization performance of the graph classification model, multiple GNNs are trained collaboratively for promoting the expressiveness of each other and increasing the reliability of graph classification results. Overall, the ASGNN framework takes fully use of unlabeled graph examples to reinforce graph classification effectively, and can be applied to any existing supervised graph neural networks for graph classification. Experimental results on benchmark graph datasets demonstrate that the proposed framework yields competitive performance on graph classification tasks with only a small number of labeled graph examples.

The existence of output noise will bring difficulties to supervised learning. Noise filtering, aiming to detect and remove polluted samples, is one of the main ways to deal with the noise on outputs. However, most of the filters are heuristic and could not explain the filtering influence on the generalization error (GE) bound. The hyper-parameters in various filters are specified manually or empirically, and they are usually unable to adapt to the data environment. The filter with an improper hyper-parameter may overclean, leading to a weak generalization ability. This paper proposes a unified framework of optimal sample selection (OSS) for the output noise filtering from the perspective of error bound. The covering distance filter (CDF) under the framework is presented to deal with noisy outputs in regression and ordinal classification problems. Firstly, two necessary and sufficient conditions for a fixed goodness of fit in regression are deduced from the perspective of GE bound. They provide the unified theoretical framework for determining the filtering effectiveness and optimizing the size of removed samples. The optimal sample size has the adaptability to the environmental changes in the sample size, the noise ratio, and noise variance. It offers a choice of tuning the hyper-parameter and could prevent filters from overcleansing. Meanwhile, the OSS framework can be integrated with any noise estimator and produces a new filter. Then the covering interval is proposed to separate low-noise and high-noise samples, and the effectiveness is proved in regression. The covering distance is introduced as an unbiased estimator of high noises. Further, the CDF algorithm is designed by integrating the cover distance with the OSS framework. Finally, it is verified that the CDF not only recognizes noise labels correctly but also brings down the prediction errors on real apparent age data set. Experimental results on benchmark regression and ordinal classification data sets demonstrate that the CDF outperforms the state-of-the-art filters in terms of prediction ability, noise recognition, and efficiency.

Semi-supervised clustering is one of important research topics in cluster analysis, which uses pre-given knowledge as constraints to improve the clustering performance. While clustering a data set, people often get prior constraints from different information sources, which may have different representations and contents, to guide clustering process. However, most of existing semi-supervised clustering algorithms are based on single-source constraints and rarely consider to integrate multi-source constraints to enhance the clustering quality. To solve the problem, we analyze the relations among different types of constraints and propose an uniform representation for them. Based it, we propose a new semi-supervised clustering algorithm to find out a clustering that has good cluster structure and high consensus of all the sources of constraints. In the algorithm, we construct an optimization objective model and its solution method to achieve the aim. This algorithm can integrate multi-source constraints well to reduce the effect of incorrect constraints from single sources and find out a high-quality clustering. By the experimental studies on several benchmark data sets, we illustrate the effectiveness of the proposed algorithm, compared to other semi-supervised clustering algorithms.

Due to the complex structure of the real-world data, nonlinearly separable clustering is one of popular and widely studied clustering problems. Currently, various types of algorithms, such as kernel k-means, spectral clustering and density clustering, have been developed to solve this problem. However, it is difficult for them to balance the efficiency and effectiveness of clustering, which limits their real applications. To get rid of the deficiency, we propose a three-level optimization model for nonlinearly separable clustering which divides the clustering problem into three sub-problems: a linearly separable clustering on the object set, a nonlinearly separable clustering on the cluster set and an ensemble clustering on the partition set. An iterative algorithm is proposed to solve the optimization problem. The proposed algorithm can use low computational cost to effectively recognize nonlinearly separable clusters. The performance of this algorithm has been studied on synthetical and real data sets. Comparisons with other nonlinearly separable clustering algorithms illustrate the efficiency and effectiveness of the proposed algorithm.

Clustering by jointly exploiting information from multiple views can yield better performance than clustering on one single view. Some existing multi-view clustering methods aim at learning a weight for each view to determine its contribution to the final solution. However, the view-weighted scheme can only indicate the overall importance of a view, which fails to recognize the importance of each inner cluster of a view. A view with higher weight cannot guarantee all clusters in this view have higher importance than them in other views. In this paper, we propose a cluster-weighted kernel k-means method for multi-view clustering. Each inner cluster of each view is assigned a weight, which is learned based on the intra-cluster similarity of the cluster compared with all its corresponding clusters in different views, to make the cluster with higher intra-cluster similarity have a higher weight among the corresponding clusters. The cluster labels are learned simultaneously with the cluster weights in an alternative updating way, by minimizing the weighted sum-of-squared errors of the kernel k-means. Compared with the view-weighted scheme, the cluster-weighted scheme enhances the interpretability for the clustering results. Experimental results on both synthetic and real data sets demonstrate the effectiveness of the proposed method.

In this paper, we provide an explicit theoretical connection between Sparse subspace clustering (SSC) and spectral clustering (SC) from the perspective of learning a data similarity matrix. We show that spectral clustering with Gaussian kernel can be viewed as sparse subspace clustering with entropy-norm (SSC+E). Compared to SSC, SSC+E can obtain an analytical, symmetrical, nonnegative and nonlinearly-representational similarity matrix. Besides, SSC+E makes use of Gaussian kernel to compute the sparse similarity matrix of objects, which can avoid the complex computation of the sparse optimization program of SSC. Finally, we provide the experimental analysis to compare the efficiency and effectiveness of sparse subspace clustering and spectral clustering on ten benchmark data sets. The theoretical and experimental analysis can well help users for the selection of high-dimensional data clustering algorithms.

Summary Multidimensional Bayesian network classifier (MBC) has become a popular classification model because of their intuitive graphical representation ability among class variables. But learning MBC and performing multidimensional classification based on the MBC can be very computationally demanding. For the tractability of performing multidimensional classification, a class‐bridge‐decomposable (CB‐decomposable) MBC model is proposed and it alleviates the computation complexity. But there are few works to efficiently and systematically learn the CB‐decomposable MBC model. Thus, we focus on addressing a naive learning algorithm of CB‐decomposable MBCs. Briefly, we learn the CB‐decomposable MBC model by dividing it into three components: class subgraph, bridge subgraph, and feature subgraph. First, we analyze why the class subgraph can be learned based on general Bayesian network learning methods. Second, we give how to learn bridge subgraph based on information gain ratio. Third, to make the CB‐decomposable MBC model effective and simple, we also study the learning and updating strategies of feature subgraph. Further, we propose the naive learning algorithm of the CB‐decomposable MBC. Finally, by comparing with other methods on several benchmark datasets, experimental results illustrate that our naive learning algorithm not only has higher accuracies, lower learning, and classification times but also has simple and intuitive representation ability.

Ordinal classification is an important classification task, in which there exists a monotonic constraint between features and the decision class. In this article, we aim to develop a method of fusing ordinal decision trees with fuzzy rough-set-based attribute reduction. Most of the existing attribute reduction methods for ordinal decision tables are based on the dominance rough set theory or significance measures. However, the crisp dominance relation is difficult in making full use of the information of attribute values; and the reducts based on significance measures are poor in interpretability and may contain unnecessary attributes. In this article, we first define a discernibility matrix with fuzzy dominance rough set. With this discernibility matrix, multiple reducts can be found, which provide multiple complementary feature subspaces with original information. Then, diverse ordinal trees can be established from these feature subspaces, and finally, the trees are fused by majority voting. The experimental results show that the proposed fusion method performs significantly better than other fusion methods using dominance rough set or significance measures.

Attribute subset selection is an important issue in data mining and information processing. However, most automatic methodologies consider only the relevance factor between samples while ignoring the diversity factor. This may not allow the utilization value of hidden information to be exploited. For this reason, we propose a hybrid model named intuitionistic fuzzy (IF) rough set to overcome this limitation. The model combines the technical advantages of rough set and IF set and can effectively consider the above-mentioned statistical factors. First, fuzzy information granules based on IF relations are defined and used to characterize the hierarchical structures of the lower and upper approximations of IF rough set within the framework of granular computing. Then, the computation of IF rough approximations and knowledge reduction in IF information systems are investigated. Third, based on the approximations of IF rough set, significance measures are developed to evaluate the approximation quality and classification ability of IF relations. Furthermore, a forward heuristic algorithm for finding one optimal reduct of IF information systems is developed using these measures. Finally, numerical experiments are conducted on public datasets to examine the effectiveness and efficiency of the proposed algorithm in terms of the number of selected attributes, computational time, and classification accuracy.

Cluster ensemble is a very important tool that aggregates several base clusterings to generate a single output clustering with improved robustness and stability. However, the quality of the final clustering is often affected by uncertainties on the generation and integration of base clusterings. In this paper, we develop an information-theoretical framework which makes an effort to obtain a final clustering with high consensus on both the original data set and the base clustering set by minimizing the two uncertainties of cluster ensemble. In this framework, we provide a weighted consensus measure based on information entropy to evaluate the quality of a clustering, the similarity between clusters and the similarity between objects. Based on the measure, we propose three weighted cluster ensemble algorithms with different ensemble strategies in the framework, including the weighted feature consensus algorithm, the weighted relabeling consensus algorithm and the weighted pairwise-similarity consensus algorithm. In the experimental analysis, we compare the proposed algorithms with other existing clustering ensemble algorithms on several data sets. The comparison results illustrate the proposed algorithms are very effective and robust.

Fuzzy cluster ensemble is an important research component of ensemble learning, which is used to aggregate several fuzzy base clusterings to generate a single output clustering with improved robustness and quality. However, since clustering is unsupervised, where “accuracy” does not have a clear meaning, it is difficult for existing ensemble methods to integrate multiple fuzzy k-means clusterings to find arbitrarily shaped clusters. To overcome the deficiency, we propose a new ensemble clusterer (algorithm) of multiple fuzzy k-means clusterings based on a local hypothesis. In the new algorithm, we study the extraction of local-credible memberships from a base clustering, the production of multiple base clusterings with different local-credible spaces, and the construction of cluster relation based on indirect overlap of local-credible spaces. The proposed ensemble clusterer not only inherits the scalability of fuzzy k-means but also overcomes the inability to find arbitrarily shaped clusters. We compare the proposed algorithm with other cluster ensemble algorithms on several synthetical and real datasets. The experimental results illustrate the effectiveness and efficiency of the proposed algorithm.

In data mining, objects are often represented by a set of features, where each feature of an object has only one value. However, in reality, some features can take on multiple values, for instance, a person with several job titles, hobbies, and email addresses. These features can be referred to as set-valued features and are often treated with dummy features when using existing data mining algorithms to analyze data with set-valued features. In this paper, we propose an SV- $k$ -modes algorithm that clusters categorical data with set-valued features. In this algorithm, a distance function is defined between two objects with set-valued features, and a set-valued mode representation of cluster centers is proposed. We develop a heuristic method to update cluster centers in the iterative clustering process and an initialization algorithm to select the initial cluster centers. The convergence and complexity of the SV- $k$ -modes algorithm are analyzed. Experiments are conducted on both synthetic data and real data from five different applications. The experimental results have shown that the SV- $k$ -modes algorithm performs better when clustering real data than do three other categorical clustering algorithms and that the algorithm is scalable to large data.

In large-scale data classification tasks, it is becoming more and more challenging in finding a true class from a huge amount of candidate categories. Fortunately, a hierarchical structure usually exists in these massive categories. The task of utilizing this structure for effective classification is called hierarchical classification. It usually follows a top-down fashion which predicts a sample from the root node with a coarse-grained category to a leaf node with a fine-grained category. However, misclassification is inevitable if the information is insufficient or large uncertainty exists in the prediction process. In this scenario, we can design a stopping strategy to stop the sample at an internal node with a coarser category, instead of predicting a wrong leaf node. Several studies address the problem by improving performance in terms of hierarchical accuracy and informative prediction. However, all of these researches ignore an important issue: when predicting a sample at the current node, the error is inclined to occur if large uncertainty exists in the next lower level children nodes. In this paper, we integrate this uncertainty into a risk problem: when predicting a sample at a decision node, it will take precipitance risk in predicting the sample to a children node in the next lower level on one hand, and take conservative risk in stopping at the current node on the other. We address the risk problem by designing a Local Bayes Risk Minimization (LBRM) framework, which divides the prediction process into recursively deciding to stop or to go down at each decision node by balancing these two risks in a top-down fashion. Rather than setting a global loss function in the traditional Bayes risk framework, we replace it with different uncertainty in the two risks for each decision node. The uncertainty on the precipitance risk and the conservative risk are measured by information entropy on children nodes and information gain from the current node to children nodes, respectively. We propose a Weighted Tree Induced Error (WTIE) to obtain the predictions of minimum risk with different emphasis on the two risks. Experimental results on various datasets show the effectiveness of the proposed LBRM algorithm.

There is always a lack of a cluster validity function and optimization strategy to find out clusters and catch the evolution trend of cluster structures on a categorical data stream. Therefore, this paper presents an optimization model for clustering categorical data streams. In the model, a cluster validity function is proposed as the objective function to evaluate the effectiveness of the clustering model while each new input data subset is flowing. It simultaneously considers the certainty of the clustering model and the continuity with the last clustering model in the clustering process. An iterative optimization algorithm is proposed to solve an optimal solution of the objective function with some constraints. Furthermore, we strictly derive a detection index for drifting concepts from the optimization model. We propose a detection method that integrates the detection index and the optimization model to catch the evolution trend of cluster structures on a categorical data stream. The new method can effectively avoid ignoring the effect of the clustering validity on the detection result. Finally, using the experimental studies on several real data sets, we illustrate the effectiveness of the proposed algorithm in clustering categorical data streams, compared with existing data-streams clustering algorithms.

Cold-start link prediction is a term for information starved link prediction where little or no topological information is present to guide the determination of whether links to a node will form. Due to the lack of topological information, traditional topology-based link prediction methods cannot be applied to solve the cold-start link prediction problem. Therefore, an effective approach is presented through establishing connections between non-topological and topological information. In the approach, topological information is first extracted by a latent-feature representation model, then a logistic model is proposed to establish the connections between topological and non-topological information, and finally the linking possibility between cold-start users and existing users is calculated. Experiments with three types of real-world social networks Weibo, Facebook, and Twitter show that the proposed approach is more effective in solving the cold-start link prediction problem and establishing connections between topological and non-topological information.

Learning from categorical data plays a fundamental role in such areas as pattern recognition, machine learning, data mining, and knowledge discovery. To effectively discover the group structure inherent in a set of categorical objects, many categorical clustering algorithms have been developed in the literature, among which k -modes-type algorithms are very representative because of their good performance. Nevertheless, there is still much room for improving their clustering performance in comparison with the clustering algorithms for the numeric data. This may arise from the fact that the categorical data lack a clear space structure as that of the numeric data. To address this issue, we propose, in this paper, a novel data-representation scheme for the categorical data, which maps a set of categorical objects into a Euclidean space. Based on the data-representation scheme, a general framework for space structure based categorical clustering algorithms (SBC) is designed. This framework together with the applications of two kinds of dissimilarities leads two versions of the SBC-type algorithms. To verify the performance of the SBC-type algorithms, we employ as references four representative algorithms of the k -modes-type algorithms. Experiments show that the proposed SBC-type algorithms significantly outperform the k -modes-type algorithms.

Multigranulation rough set theory is a relatively new mathematical tool for solving complex problems in the multigranulation or distributed circumstances which are characterized by vagueness and uncertainty. In this paper, we first introduce the multigranulation approximation space. According to the idea of fusing uncertain, imprecise information, we then present three uncertainty measures: fusing information entropy, fusing rough entropy, and fusing knowledge granulation in the multigranulation approximation space. Furthermore, several essential properties (equivalence, maximum, minimum) are examined and the relationship between the fusion information entropy and the fusion rough entropy is also established. Finally, we prove these three measures are monotonously increasing as the partitions become finer. These results will be helpful for understanding the essence of uncertainty measures in multigranulation rough space and enriching multigranulation rough set theory.

Decision makers often express their evaluations on decision problems with multi-granular linguistic terms. This fact leads to the unification of the multi-granular linguistic terms into a single linguistic set in the literature. However, this unification process increases the complexity of computation and the subjectivity in the determination of transformation functions. To overcome this deficiency, this paper aims to develop a normalized numerical scaling method for determining the semantics of multi-granular linguistic terms in the same domain. We first introduce a class of numerical scaling functions to generate several balanced or unbalanced linguistic sets. Since these scaled linguistic sets have different domains, we then develop a normalized numerical scaling method to form them into the unique interval [0,1]. As a result of this development, two classes of normalized scaling functions are derived from the priori scale information and applications of piecewise linear interpolation and piecewise arc interpolation. Finally, an example is given to illustrate how the method works.

A fuzzy granular structure refers to a mathematical structure of the collection of fuzzy information granules granulated from a dataset, while a fuzzy information granularity is used to measure its uncertainty. However, the existing forms of fuzzy information granularity have two limitations. One is that when the fuzzy information granularity of one fuzzy granular structure equals that of the other, one can say that these two fuzzy granular structures possess the same uncertainty, but these two fuzzy granular structures may be not equivalent to each other. The other limitation is that existing axiomatic approaches to fuzzy information granularity are still not complete, under which when the partial order relation among fuzzy granular structures cannot be found, their coarseness/fineness relationships will not be revealed. To address these issues, a so-called fuzzy granular structure distance is proposed in this study, which can well discriminate the difference between any two fuzzy granular structures. Besides this advantage, the fuzzy granular structure distance has another important benefit: It can be used to establish a generalized axiomatic constraint for fuzzy information granularity. By using the axiomatic constraint, the coarseness/fineness of any two fuzzy granular structures can be distinguished. In addition, through taking the fuzzy granular structure distances of a fuzzy granular structure to the finest one and the coarsest one into account, we also can build a bridge between fuzzy information granularity and fuzzy information entropy. The applicable analysis on 12 real-world datasets shows that the fuzzy granular structure distance and the generalized fuzzy information granularity have much better performance than existing methods.

Ordinal classification with a monotonicity constraint is a kind of classification tasks, in which the objects with better attribute values should not be assigned to a worse decision class. Several learning algorithms have been proposed to handle this kind of tasks in recent years. The rank entropy-based monotonic decision tree is very representative thanks to its better robustness and generalization. Ensemble learning is an effective strategy to significantly improve the generalization ability of machine learning systems. The objective of this work is to develop a method of fusing monotonic decision trees. In order to achieve this goal, we take two factors into account: attribute reduction and fusing principle. Through introducing variable dominance rough sets, we firstly propose an attribute reduction approach with rank-preservation for learning base classifiers, which can effectively avoid overfitting and improve classification performance. Then, we establish a fusing principe based on maximal probability through combining the base classifiers, which is used to further improve generalization ability of the learning system. The experimental analysis shows that the proposed fusing method can significantly improve classification performance of the learning system constructed by monotonic decision trees.

Choquet integral, as an adequate aggregation operator, extends the weighted mean operator by considering interactions among attributes. Choquet integral has been widely used in many real multi-attribute decision making. Weights (fuzzy measures) of attribute sets directly affect the decision results in multi-attribute decision making. In this paper, we aim to propose an objective method based on granular computing for determining the weights of the attribute sets. To address this issue, we first analyze the implied preorder relations under four evaluation forms and construct the corresponding preorder granular structures. Then, we define fuzzy measure of an attribute set by the similarity degree between a special preorder pairs. Finally, we employ two numerical examples for illustrating the feasibility and effectiveness of the proposed method. It is deserved to point out that the weight of each attribute subset can be learned from a given data set by the proposed method, not but be given subjectively by the decision maker. This idea provides a new perspective for multi-attribute decision making.

Many real data increase dynamically in size. This phenomenon occurs in several fields including economics, population studies, and medical research. As an effective and efficient mechanism to deal with such data, incremental technique has been proposed in the literature and attracted much attention, which stimulates the result in this paper. When a group of objects are added to a decision table, we first introduce incremental mechanisms for three representative information entropies and then develop a group incremental rough feature selection algorithm based on information entropy. When multiple objects are added to a decision table, the algorithm aims to find the new feature subset in a much shorter time. Experiments have been carried out on eight UCI data sets and the experimental results show that the algorithm is effective and efficient.

As a leading partitional clustering technique, k-modes is one of the most computationally efficient clustering methods for categorical data. In the k-modes, a cluster is represented by a "mode," which is composed of the attribute value that occurs most frequently in each attribute domain of the cluster, whereas, in real applications, using only one attribute value in each attribute to represent a cluster may not be adequate as it could in turn affect the accuracy of data analysis. To get rid of this deficiency, several modified clustering algorithms were developed by assigning appropriate weights to several attribute values in each attribute. Although these modified algorithms are quite effective, their convergence proofs are lacking. In this paper, we analyze their convergence property and prove that they cannot guarantee to converge under their optimization frameworks unless they degrade to the original k-modes type algorithms. Furthermore, we propose two different modified algorithms with weighted cluster prototypes to overcome the shortcomings of these existing algorithms. We rigorously derive updating formulas for the proposed algorithms and prove the convergence of the proposed algorithms. The experimental studies show that the proposed algorithms are effective and efficient for large categorical datasets.

Abstract The notion of information systems provides a convenient tool for knowledge representation of objects in terms of their attribute values, while partial ordering is usually used to research the rough monotonicity of an uncertainty measure in information systems. In this paper, we first reveal the limitations of existing partial orderings to describe information granulations in information systems with several illustrative examples. Then, a generalized partial ordering with a set‐size nature is proposed to overcome their shortcoming and some of its important properties are derived. Finally, we prove that several existing information granulations all satisfy the granulation monotonicity induced by the proposed partial ordering. The presented partial ordering appears to be well suited to characterize the nature of information granulations in an information system. These results will be very helpful for studying granular computing and uncertainty in information systems.

$K$-means is a partitional clustering technique that is well-known and widely used for its low computational cost. The representative algorithms include the hard$k$-means and the fuzzy$k$-means. However, the performance of these algorithms tends to be affected by skewed data distributions, i.e., imbalanced data. They often produce clusters of relatively uniform sizes, even if input data have varied cluster sizes, which is called the “uniform effect.” In this paper, we analyze the causes of this effect and illustrate that it probably occurs more in the fuzzy$k$-means clustering process than the hard$k$-means clustering process. As the fuzzy index$m$increases, the “uniform effect” becomes evident. To prevent the effect of the “uniform effect,” we propose a multicenter clustering algorithm in which multicenters are used to represent each cluster, instead of one single center. The proposed algorithm consists of the three subalgorithms: the fast global fuzzy$k$-means, Best M-Plot, and grouping multicenter algorithms. They will be, respectively, used to address the three important problems: 1) How are the reliable cluster centers from a dataset obtained? 2) How are the number of clusters which these obtained cluster centers represent determined? 3) How is it judged as to which cluster centers represent the same clusters? The experimental studies on both synthetic and real datasets illustrate the effectiveness of the proposed clustering algorithm in clustering balanced and imbalanced data.

Clustering analysis is an outstanding contribution to data mining and knowledge discovery from large-scale data, which has become an important research direction. Many excellent researches have been developed, whereas there is a blind point which is not addressed. How to organize data with measurement errors? What is needed for this purpose is the concept of error number- an error number which is defined as a(x)±αa(x), where a(x) is the measurement value of an object x under an attribute a, and αa(x)is the value of its measurement error. In this paper, we will explore tentatively data clustering with measurement errors through employing the framework of k-means clustering algorithm, which can be regarded as an introductory work.

Zadeh’s seminal work in theory of fuzzy-information granulation in human reasoning is inspired by the ways in which humans granulate information and reason with it. This has led to an interesting research topic: granular computing (GrC). Although many excellent research contributions have been made, there remains an important issue to be addressed: What is the essence of measuring a fuzzy-information granularity of a fuzzy-granular structure? What is needed to answer this question is an axiomatic constraint with a partial-order relation that is defined in terms of the size of each fuzzy-information granule from a fuzzy-binary granular structure. This viewpoint is demonstrated for fuzzy-binary granular structure, which is called the binary GrC model by Lin. We study this viewpoint from from five aspects in this study, which are fuzzy BINARY-granular-structure operators, partial-order relations, measures for fuzzy-information granularity, an axiomatic approach to fuzzy-information granularity, and fuzzy-information entropies.

A fundamental assumption often made in unsupervised learning is that the problem is static, i.e., the description of the classes does not change with time. However, many practical clustering tasks involve changing environments. It is hence recognized that the methods and techniques to analyze the evolving trends for changing environments are of increasing interest and importance. Although the problem of clustering numerical time-evolving data is well-explored, the problem of clustering categorical time-evolving data remains as a challenging issue. In this paper, we propose a generalized clustering framework for categorical time-evolving data, which is composed of three algorithms: a drifting-concept detecting algorithm that detects the difference between the current sliding window and the last sliding window, a data-labeling algorithm that decides the most-appropriate cluster label for each object of the current sliding window based on the clustering results of the last sliding window, and a cluster-relationship-analysis algorithm that analyzes the relationship between clustering results at different time stamps. The time-complexity analysis indicates that these proposed algorithms are effective for large datasets. Experiments on a real dataset show that the proposed framework not only accurately detects the drifting concepts but also attains clustering results of better quality. Furthermore, compared with the other framework, the proposed one needs fewer parameters, which is favorable for specific applications.

The original rough-set model is primarily concerned with the approximations of sets described by a single equivalence relation on a given universe. With granular computing point of view, the classical rough-set theory is based on a single granulation. This correspondence paper first extends the rough-set model based on a tolerance relation to an incomplete rough-set model based on multigranulations, where set approximations are defined through using multiple tolerance relations on the universe. Then, several elementary measures are proposed for this rough-set framework, and a concept of approximation reduct is introduced to characterize the smallest attribute subset that preserves the lower approximation and upper approximation of all decision classes in this rough-set model. Finally, several key algorithms are designed for finding an approximation reduct.

Based on an intuitionistic knowledge content nature of information gain, the concept of combination entropy CE(A) in incomplete information systems is first introduced, and some of its important properties are given. Then, the conditional combination entropy CE(Q | P) and the mutual information CE(P;Q) are defined. Unlike all existing measures for the uncertainty in incomplete information systems, the relationships among these three concepts can be established, which are formally expressed as CE(Q | P) = CE(P ⋃ Q)-CE(P) and CE(P;Q) = CE(P)-CE(P | Q). Furthermore, a variant CE(CA) of the combination entropy with maximal consistent block nature is introduced to measure the uncertainty of an incomplete information system in the view of maximal consistent block technique. Its monotonicity is the same as that of the combination entropy. Finally, the combination granulation CG(A) and its variant CG(CA) with maximal consistent block nature are defined to measure discernibility ability of an incomplete information system, and the relationship between the combination entropy and the combination granulation is established as well. These results will be very helpful for understanding the essence of knowledge content and uncertainty measurement in incomplete information systems. Note that the combination entropy also can be further extended to measure the uncertainty in non-equivalence-based information systems.

Based on the intuitionistic knowledge content nature of information gain, the concepts of combination entropy and combination granulation are introduced in rough set theory. The conditional combination entropy and the mutual information are defined and their several useful properties are derived. Furthermore, the relationship between the combination entropy and the combination granulation is established, which can be expressed as CE(R) + CG(R) = 1. All properties of the above concepts are all special instances of those of the concepts in incomplete information systems. These results have a wide variety of applications, such as measuring knowledge content, measuring the significance of an attribute, constructing decision trees and building a heuristic function in a heuristic reduct algorithm in rough set theory.

Rough set theory is a relatively new mathematical tool for computer applications in circumstances characterized by vagueness and uncertainty. In this paper, we address uncertainty of rough sets for incomplete information systems. An axiom definition of knowledge granulation for incomplete information systems is obtained, under which a measure of uncertainty of a rough set is proposed. This measure has some nice properties such as equivalence, maximum and minimum. Furthermore, we prove that the uncertainty measure is effective and suitable for measuring roughness and accuracy of rough sets for incomplete information systems.

Rough set theory is a relatively new mathematical tool for use in computer applications in circumstances which are characterized by vagueness and uncertainty. In this paper, we introduce the concepts of information entropy, rough entropy and knowledge granulation in rough set theory, and establish the relationships among those concepts. These results will be very helpful for understanding the essence of concept approximation and establishing granular computing in rough set theory.