Recently, Heterogeneous Graph Contrastive Learning (HGCL) has received significant attention due to its impressive capability to represent heterogeneous graphs without detailed annotations. However, the inherent fragility of heterogeneous graph structures makes HGCL vulnerable to perturbation attacks. Most existing defense works for heterogeneous graphs primarily focus on supervised scenarios, which protect all nodes equally via structural pruning. This defensive mechanism can result in insufficient structure information for HGCL, thus degrading performance in self-supervised scenarios without labels. In this paper, we argue that some nodes are more susceptible to attacks, and the influence of the perturbation attack will accumulate across layers during representation aggregation. To tackle these problems, we propose a novel Heterogeneous Graph Contrastive Learning with Hierarchical Vulnerability Protection (HVP-HGCL), which identifies the most vulnerable nodes to perturbation attack and protects them across different aggregation layers to improve the robustness of HGCL. Specifically, we first design the Vulnerability Detection (VD) based on the HGCL framework to determine which nodes are more sensitive to attack in self-supervised scenarios. Subsequently, we propose a simple but efficient Hierarchical Protection (HP) to safeguard those vulnerable nodes from attack noise during different layers. Combining the above two modules, HVP-HGCL can not only improve the robustness of HGCL but also ensure sufficient structural information for effective contrastive learning. Extensive experiments demonstrate that HVP-HGCL improves robustness against adversarial attacks and achieves competitive performance on downstream tasks.

In federated recommendation systems, model poisoning attacks aim to manipulate the gradient information of multiple target items sent back from local clients to the central server, with the goal of abnormally increasing their exposure across the system. Existing multi-target attack approaches directly manipulate multiple target items and apply a uniform attack strategy to all target items, which may lead to suboptimal promotion effectiveness. To address this issue, we introduce ProitMTA, a novel multi-target model poisoning attack framework that introduces proxy items and provides tailored attack strategies for target items. ProitMTA employs a three-stage process that balances the promotion of multiple target items while preserving recommendation quality. First,proxy item generationuses a Gaussian Mixture Model to create proxy items that represent diverse attack strategies. Second,proxy attack constructiondesigns customized gradient manipulation strategies for each proxy item. Finally,proxy-based target item attacktransfers these strategies to actual target items, enhancing their promotion while minimizing the negative impact on system performance. Through comprehensive experiments on multiple base federated recommendation frameworks and diverse real-world datasets, we demonstrate that ProitMTA outperforms existing attack methods, achieving higher success rates in target item promotion with minimal system-wide performance degradation. Our research highlights the vulnerability of federated recommendation systems when facing multi-target poisoning attacks and underscores the importance of researching effective defense mechanisms We have released our code athttps://github.com/zdy769243418/ProitMTA.

Multi-task Learning (MTL) involves training multiple tasks within a single model to improve overall performance by leveraging shared knowledge. However, this joint training can result in performance degradation due to task conflicts, typically manifesting as conflicts in task gradients. Existing solutions primarily focus on modeling task gradient relationships, which overlook the differences in how the same data sample influences different tasks. These differences are the source of intricate task gradient relationships and could further lead to varying degrees of impact from conflicts on tasks. To tackle these challenges, we propose DTJS, a novel adaptive Data and Task Joint Scheduling approach for MTL, which uniquely considers the influence of data within each task and the distinct task perception of gradient conflicts from an innovative scheduling perspective. Specifically, we design intra-task scheduling to quantify the difficulty level of the data based on its influence within each task, facilitating easy-to-hard data scheduling. Concurrently, inter-task scheduling is proposed to capture the diverse relationship among joint learning tasks via assessing the severity of conflicts between tasks and adaptively considering their effects on individual tasks through learnable task conflict perception. Furthermore, DTJS utilizes a bi-level optimization strategy that alternately updates model parameters and the learnable task conflict perception, taking into account their interdependence. Scheduled model gradients are used to optimize the MTL model, while implicit gradients refine the learnable task conflict perception. Extensive experimental results not only demonstrate that DTJS improves the performance of the MTL model over SOTA methods across various scenarios but also explain how DTJS schedules both data and tasks to bring performance improvements. The code is available at https://github.com/ZeyuLiu0706IDTJS.

Anchor-based multi-view graph clustering has recently gained popularity as an effective approach for clustering data with multiple views. However, existing methods have limitations in terms of handling inconsistent information and noise across views, resulting in an unreliable consensus representation. Additionally, post-processing is needed to obtain final results after anchor graph construction, which negatively affects clustering performance. In this paper, we propose a Robust and Consistent Anchor Graph Learning method (RCAGL) for multi-view clustering to address these challenges. RCAGL constructs a consistent anchor graph that captures inter-view commonality and filters out view-specific noise by learning a consistent part and a view-specific part simultaneously. A k-connectivity constraint is imposed on the consistent anchor graph, leading to a clear graph structure and direct generation of cluster labels without additional post-processing. Experimental results on several benchmark datasets demonstrate the superiority of RCAGL in terms of clustering accuracy, scalability to large-scale data, and robustness to view-specific noise, outperforming advanced multi-view clustering methods. Our code is publicly available at https://github.com/Tracesource/RCAGL.

Large Language Models (LLMs) offer new opportunities for the next Point-Of-Interest (POI) prediction task, leveraging their capabilities in semantic understanding of POI trajectories. However, previous LLM-based methods, which are superficially adapted to next POI prediction, largely overlook critical challenges associated with applying LLMs to this task. Specifically, LLMs encounter two critical challenges: (1) a lack of intrinsic understanding of numeric spatiotemporal data, which hinders accurate modeling of users' spatiotemporal distributions and preferences; and (2) an excessively large and unconstrained candidate POI space, which often results in random or irrelevant predictions. To address these issues, we propose a Collaborative Multi-Agent Framework for Next POI Prediction, named CoMaPOI. Through the close interaction of three specialized agents (Profiler, Forecaster, and Predictor), CoMaPOI collaboratively addresses the two critical challenges. The Profiler agent is responsible for converting numeric data into language descriptions, enhancing semantic understanding. The Forecaster agent focuses on dynamically constraining and refining the candidate POI space. The Predictor agent integrates this information to generate high-precision predictions. Extensive experiments on three benchmark datasets (NYC, TKY, and CA) demonstrate that CoMaPOI achieves state-of-the-art performance, improving all metrics by 5% to 10% compared to SOTA baselines. This work pioneers the investigation of challenges associated with applying LLMs to complex spatiotemporal tasks by leveraging tailored collaborative agents. Our source code is available at: https://github.com/Chips98/CoMaPOI.

Federated Class Incremental Learning (FCIL) has emerged as a new paradigm due to its applicability in real-world scenarios. In FCIL, clients continuously generate new data with unseen class labels and do not share local data due to privacy restrictions, and each client’s class distribution evolves dynamically and independently. However, existing work still faces two significant challenges. Firstly, current methods lack a better balance between maintaining sound anti-forgetting effects over old data (stability) and ensuring good adaptability for new tasks (plasticity). Secondly, some FCIL methods overlook that the incremental data will also have a non-identical label distribution, leading to poor performance. This paper proposes CGoFed, which includes relax-constrained gradient update and cross-task gradient regularization modules. The relax-constrained gradient update prevents forgetting the knowledge about old data while quickly adapting to the new data by constraining the gradient update direction to a gradient space that minimizes interference with historical tasks. The cross-task gradient regularization also finds applicable historical models from other clients and trains a personalized global model to address the non-identical label distribution problem. The results demonstrate that the CGoFed performs well in alleviating catastrophic forgetting and improves model performance by 8% -23% compared with the SOTA comparison method.

Although a variety of models have been proposed for urban spatio-temporal forecasting, most existing forecasting models are developed manually for specific tasks. By investigating the correlation between multi-order derivative and spatio-temporal data, we propose a generic yet simple plug-in structure, namedTaylorS, to improve the performance and generalization of existing forecasting models. The TaylorS converts the non-linear regression problem into a multi-order non-linear approximation problem by plugging a Taylor expansion into the forecasting task. To achieve this, we design a two-step training framework, including a training step and an adjusting step. During training, we train a given forecasting model as a base model to be equipped with prior knowledge. During adjusting, we fine-tune the base model while plugging an adjustment model into the base model. The adjustment model, as a multi-order expansion, takes the multi-order derivative of data to evaluate data uncertainty for further forecasting approximation and adjustment. Extensive experimental results demonstrate that the proposed TaylorS framework can consistently improve the performance of existing state-of-the-art methods and generalize these methods to different forecasting tasks.

Multi-view graph clustering (MVGC) explores pairwise correlations of entire instances and comprehensively aggregates diverse source information with optimal graph structure. One major issue of practical MVGC is the high time and space complexities prohibiting being applied on large-scale applications. As a promising solution of addressing large-scale problems, anchor-based strategy identifies small portion and key landmarks to serve as replacements for the entire dataset. Despite of its efficiency, anchors chosen across views may be semantically unaligned contrasting to naturally-aligned full sample setting, which may lead to the latter inappropriate graph fusion. Limited attention has been focused on the mentioned Multi-View Anchor-Unaligned Problem (MV-AUP) in the existing literature. In this paper, we first revisit existing multi-view anchor graph clustering frameworks and present the MV-AUP phenomenon. Then, we propose a novelMulti-viewCorrespondingAnchorGraphAlignmentFusion framework (MV-CAGAF), which elegantly solves MV-AUP with structural representation matching in multi-dimensional spaces. Further, we theoretically prove our proposed structural matching approach can be regarded as minimizing the EMD distance of the two relative anchor distributions. Based on this, we design the innovative multi-view anchor graph fusion paradigm with correspondence alignment, which inherits the linear sample complexity for scalable cross-view clustering. Our proposed MV-CAGAF achieves significant improvements with the help of the novel fusion framework on comprehensive benchmark datasets. Most importantly, the experimental results on both of the simulated and real-world datasets significantly prove the importance of cross-view alignment for large-scale multi-view clustering.

Multi-task learning (MTL) has succeeded in various industrial applications by utilizing common knowledge among joint training tasks to enhance the generalization of MTL models, resulting in improved performance across all training tasks simultaneously. Unfortunately, training all tasks simultaneously often causes performance degradation compared to single-task models since different tasks might conflict with each other. Despite existing MTL methods that aim to mitigate task conflicts by manipulating task gradients at each iteration, they ignore the potential influence of noisy data from different batches on task gradients. Consequently, the current iteration's task gradient may not accurately reflect the task itself, leading to inadequate alleviation of the dilemma of task conflicts. Moreover, existing works seldom explore the potential source of task conflicts and merely pose an assumption. In this paper, we conduct an in-depth empirical investigation into the potential sources of performance degradation of MTL and find that task gradient conflict is one of the primary reasons for the performance degradation of tasks. Then, to address the task conflicts problem, we propose a novel gradient manipulation approach, namely MoCoGrad, which manipulates task gradients by leveraging the momentum information of the task to calibrate the gradients of conflicting tasks. In addition, we derive theoretical guarantees for the con-vergence of our proposed MoCoGrad and theoretically analyze the convergence rate of MoCoGrad. Finally, to evaluate the effectiveness of MoCoGrad, extensive experiments are conducted on six real-world datasets from different domains. Our approach yields the best performance across all tasks in all six MTL benchmarks, demonstrating the effectiveness and superiority of our method.

Graph contrastive learning (GCL) has gained increasing interest as a solution for graph representation learning. In GCL, graph augmentation is essential to generate contrastive samples used for contrastive learning. Recently, most existing methods employ learnable graph view generators to augment graphs based on the node probability distribution adaptively. However, these methods cannot ensure that semantic-related nodes are preserved during graph augmentation, leading to performance degradation. To tackle this issue, we propose a novel approach called Semantic-aware Graph Contrastive Learning (SGCL), which can generate high-quality contrastive samples by only augmenting semantic-unrelated nodes so as to facilitate the performance of GCL on downstream tasks. Specifically, we first design a Lipschitz constant generator to compute the Lipschitz constants that measure the semantic relevance of each node. Then, we propose the Lipschitz graph augmentation to augment graphs while only dropping these semantic-unrelated nodes with small Lipschitz constants. Furthermore, we propose semanticaware contrastive learning to obtain more refined representations by contrasting the graph-level representation of anchor graphs and high-quality generated samples. Experimental results on unsupervised learning and transfer learning demonstrate the effectiveness of SGCL compared to state-of-the-art methods.

Accurate forecasting of traffic flow plays a crucial role in building smart cities in the new era. Previous work has achieved success in learning inherent spatial and temporal patterns of traffic flow. However, existing works investigated the multiple periodicities (e.g., hourly, daily, and weekly) of traffic via entanglement learning, which has not yet dealt with distribution shift and interaction shift problems in traffic flow. In this paper, we propose a novel disentanglement learning network, called MUSE-Net, to tackle the limitations of entanglement learning by simultaneously factorizing the exclusiveness and interaction of multi-periodic patterns in traffic flow. Grounded in the theory of mutual information, we first learn and dis-entangle exclusive and interactive representations of traffics from multi-periodic patterns. Then, we utilize semantic-pushing and semantic-pulling regularizations to encourage the learned representations to be independent and informative. Moreover, we derive a lower bound estimator to tractably optimize the disentanglement problem with multiple variables and propose a joint training model for traffic forecasting. Extensive experimental results on several real-world traffic datasets demonstrate the effectiveness of the proposed framework. The code is available at: https://github.com/JianyangQin/MUSE-Net.

Information spreads quickly through social media platforms, especially fake news with negative or even malicious intentions. In recent years, psychological studies have found that explicit reminders of fake news would diminish its consequence. Therefore, it is crucial to identify their authenticity at an early stage to avoid serious consequences. However, existing methods for fake news detection either utilize auxiliary information including users’ profiles and related events propagation networks or require sufficient and high-quality training data, which is not suitable for early fake news detection in real. An increasing number of social media news not only involves natural language content but also visual content such as images and videos, which give us a new view of fake news detection at an early stage by multi-modal data. In this paper, we propose a Multi-modal Prompt Learning framework (MPL) based on the multi-modal pre-trained model CLIP for early detection of fake news. A learnable prompt module is developed to adaptively and efficiently generate prompt representations to boost the semantic context. MPL can be implemented in supervised or few-shot settings. Extensive experiments show that the proposed MPL obtains substantial performance and efficiency improvement for the early-stage fake news detection task. The results demonstrate that MPL performs considerably well compared to both the state-ofthe-art supervised multi-modal models and the latest promptbased few-shot multi-modal models. Especially, the high recall of fake news and the high precision of real news that MPL achieved compared to other baselines verify that it will better approach one of the motivations that providing early notification of “maybe real” or “maybe fake” with the release of the news.

Cross-domain Sequential Recommendation (CSR) which leverages user sequence data from multiple domains has received extensive attention in recent years. However, the existing CSR methods require sharing origin user data across domains, which violates the General Data Protection Regulation (GDPR). Thus, it is necessary to combine federated learning (FL) and CSR to fully utilize knowledge from different domains while preserving data privacy. Nonetheless, the sequence feature heterogeneity across different domains significantly impacts the overall performance of FL. In this paper, we propose FedDCSR, a novel federated cross-domain sequential recommendation framework via disentangled representation learning. Specifically, to address the sequence feature heterogeneity across domains, we introduce an approach called inter-intra domain sequence representation disentanglement (SRD) to disentangle the user sequence features into domain-shared and domain-exclusive features. In addition, we design an intra domain contrastive infomax (CIM) strategy to learn richer domain-exclusive features of users by performing data augmentation on user sequences. Extensive experiments on three real-world scenarios demonstrate that FedDCSR achieves significant improvements over existing baselines1.

Anchor-based multi-view graph clustering has recently gained popularity as an effective approach for clustering data with multiple views. However, existing methods have limitations in terms of handling inconsistent information and noise across views, resulting in an unreliable consensus representation. Additionally, post-processing is needed to obtain final results after anchor graph construction, which negatively affects clustering performance. In this paper, we propose a Robust and Consistent Anchor Graph Learning method (RCAGL) for multi-view clustering to address these challenges. RCAGL constructs a consistent anchor graph that captures inter-view commonality and filters out view-specific noise by learning a consistent part and a view-specific part simultaneously. A$k$-connectivity constraint is imposed on the consistent anchor graph, leading to a clear graph structure and direct generation of cluster labels without additional post-processing. Experimental results on several benchmark datasets demonstrate the superiority of RCAGL in terms of clustering accuracy, scalability to large-scale data, and robustness to view-specific noise, outperforming advanced multi-view clustering methods. Our code is publicly available athttps://github.com/Tracesource/RCAGL.

Masked graph autoencoder (MGAE) has emerged as a promising self-supervised graph pre-training (SGP) paradigm due to its simplicity and effectiveness. However, existing efforts perform the mask-then-reconstruct operation in the raw data space as is done in computer vision (CV) and natural language processing (NLP) areas, while neglecting the important non-Euclidean property of graph data. As a result, the highly unstable local structures largely increase the uncertainty in inferring masked data and decrease the reliability of the exploited self-supervision signals, leading to inferior representations for downstream evaluations. To address this issue, we propose a novel SGP method termed Robust mAsked gRaph autoEncoder (RARE) to improve the certainty in inferring masked data and the reliability of the self-supervision mechanism by further masking and reconstructing node samples in the high-order latent feature space. Through both theoretical and empirical analyses, we have discovered that performing a joint mask-then-reconstruct strategy in both latent feature and raw data spaces could yield improved stability and performance. To this end, we elaborately design a masked latent feature completion scheme, which predicts latent features of masked nodes under the guidance of high-order sample correlations that are hard to be observed from the raw data perspective. Specifically, we first adopt a latent feature predictor to predict the masked latent features from the visible ones. Next, we encode the raw data of masked samples with a momentum graph encoder and subsequently employ the resulting representations to improve the predicted results through latent feature matching. Extensive experiments on seventeen datasets have demonstrated the effectiveness and robustness of RARE against state-of-the-art (SOTA) competitors across three downstream tasks. Our source code is available athttps://github.com/WxTu/RARE.

Time series anomaly detection is instrumental in maintaining system availability in various domains. Current work in this research line mainly focuses on learning data normality deeply and comprehensively by devising advanced neural network structures and new reconstruction/prediction learning objectives. However, their one-class learning process can be misled by latent anomalies in training data (i.e., anomaly contamination) under the unsupervised paradigm. Their learning process also lacks knowledge about the anomalies. Consequently, they often learn a biased, inaccurate normality boundary. To tackle these problems, this paper proposes calibrated one-class classification for anomaly detection, realizing contamination-tolerant, anomaly-informed learning of data normality via uncertainty modeling-based calibration and native anomaly-based calibration. Specifically, our approach adaptively penalizes uncertain predictions to restrain irregular samples in anomaly contamination during optimization, while simultaneously encouraging confident predictions on regular samples to ensure effective normality learning. This largely alleviates the negative impact of anomaly contamination. Our approach also creates native anomaly examples via perturbation to simulate time series abnormal behaviors. Through discriminating these dummy anomalies, our one-class learning is further calibrated to form a more precise normality boundary. Extensive experiments on ten real-world datasets show that our model achieves substantial improvement over sixteen state-of-the-art contenders.

Multi-view learning becomes increasingly attractive and promising because multimodal or multi-view data are commonly encountered in real-world applications. In this study, we develop a novel multi-view Takagi–Sugeno–Kang (TSK) fuzzy system framework to handle classification problems for such data. We propose an anchor and graph subspace clustering strategy to discover and represent the actual latent data distribution for each view separately. In this way, the discriminate anchors (landmarks) are learned to capture the main structure of the multi-view data. This strategy also provides a computationally efficient clustering algorithm with respect to the number of instances. These resulting anchors are formed as the prototypes of information granules (IGs) for fuzzy modeling. Then we construct an information-granule-based multi-view TSK fuzzy classification model inherited from the natural interpretability of fuzzy rule-based systems. Concretely, the relationship between the multi-view input and label output spaces is depicted by IGs-oriented fuzzy rules. The experimental studies involve various commonly used benchmark datasets, which indicate that our proposed method achieves comparable or better performance compared to the state-of-the-art algorithms.

Fake news detection attracts many researchers’ attention due to the negative impacts on the society. Most existing fake news detection approaches mainly focus on semantic analysis of news’ contents. However, the detection performance will dramatically decrease when the content of news is short. In this paper, we propose a novelfake news detection multi-task learning (FDML)model based on the following observations: 1) some certain topics have higher percentages of fake news; and 2) some certain news authors have higher intentions to publish fake news. FDML model investigates the impact of topic labels for the fake news and introduce contextual information of news at the same time to boost the detection performance on the short fake news. Specifically, the FDML model consists of representation learning and multi-task learning parts to train the fake news detection task and the news topic classification task, simultaneously. As far as we know, this is the first fake news detection work that integrates the above two tasks. The experiment results show that the FDML model outperforms state-of-the-art methods on real-world fake news dataset.

Multiple kernel clustering (MKC) algorithm aims to group data into different categories by optimally integrating information from a group of pre-specified kernels. Though demonstrating superiorities in various applications, we observe that existing MKC algorithms usuallydo not sufficiently consider the local density around individual data samplesandexcessively limit the representation capacity of the learned optimal kernel, leading to unsatisfying performance. In this paper, we propose an algorithm, called optimal neighborhood MKC with adaptive local kernels (ON-ALK), to address the two issues. In specific, we construct adaptive local kernels to sufficiently consider the local density around individual data samples, where different numbers of neighbors are discriminatingly selected on each sample. Further, the proposed ON-ALK algorithm boosts the representation of the learned optimal kernel via relaxing it into the neighborhood area of weighted combination of the pre-specified kernels. To solve the resultant optimization problem, a three-step iterative algorithm is designed and theoretically proven to be convergent. After that, we also study the generalization bound of the proposed algorithm. Extensive experiments have been conducted to evaluate the clustering performance. As indicated, the algorithm significantly outperforms state-of-the-art methods in recent literatures on six challenging benchmark datasets, verifying its advantages and effectiveness.

Astounding results from transformer models with Vision-and Language Pretraining (VLP) on joint vision-and-language downstream tasks have intrigued the multi-modal community. On the one hand, these models are usually so huge that make us more difficult to fine-tune and serve real-time online applications. On the other hand, the compression of the original transformer block will ignore the difference in information between modalities, which leads to the sharp decline of retrieval accuracy.

Unsupervised domain adaptation (UDA) attempts to transfer specific knowledge from one domain with labeled data to another domain without labels. Recently, maximum squares loss has been proposed to tackle UDA problem but it does not consider the prediction diversity which has proven beneficial to UDA. In this paper, we propose a novel normalized squares maximization (NSM) loss in which the maximum squares is normalized by the sum of squares of class sizes. The normalization term enforces the class sizes of predictions to be balanced to explicitly increase the diversity. Theoretical analysis shows that the optimal solution to NSM is one-hot vectors with balanced class sizes, i.e., NSM encourages both discriminate and diverse predictions. We further propose a robust variant of NSM, RNSM, by replacing the square loss with L2,1-norm to reduce the influence of outliers and noises. Experiments of cross-domain image classification on two benchmark datasets illustrate the effectiveness of both NSM and RNSM. RNSM achieves promising performance compared to state-of-the-art methods. The code is available at https://github.com/wj-zhang/NSM.

Non-negative matrix factorization (NMF) decomposes any non-negative matrix into the product of two low dimensional non-negative matrices. Since NMF learns effective parts-based representation, it has been widely applied in computer vision and data mining. However, traditional NMF has the risk learning rank-deficient basis on high-dimensional dataset with few examples especially when some examples are heavily corrupted by outliers. In this paper, we propose a Logdet divergence based sparse NMF method (LDS-NMF) to deal with the rank-deficiency problem. In particular, LDS-NMF reduces the risk of rank deficiency by minimizing the Logdet divergence between the product of basis matrix with its transpose and the identity matrix, meanwhile penalizing the density of the coefficients. Since the objective function of LDS-NMF is nonconvex, it is difficult to optimize. In this paper, we develop a multiplicative update rule to optimize LDS-NMF in the frame of block coordinate descent, and theoretically prove its convergence. Experimental results on popular datasets show that LDS-NMF can learn more stable representations than those learned by representative NMF methods.