Heterogeneous graphs are increasingly used to represent complex real-world scenarios with diverse entities and interactions by meta structures. Recently, the search of meta structures is combined with graph neural architecture search to automatically extract the semantic knowledge for various tasks in heterogeneous graphs. However, prior research primarily focuses on identifying meta structures that are universally applicable across all nodes in a graph, neglecting the variations in meta structure selection that arise from the unique features and topology of individual nodes. To address this challenge, we introduce a Node-Centric approach to search Meta Structures in heterogeneous graphs (NC-MS for short). NC-MS implements a node level method that discover meaningful meta structures tailored to each node, capturing subtle differences in meta structure choices between nodes and providing nuanced identification. Additionally, NC-MS utilizes an efficient and differentiable network to enhance operational efficiency. Empirical studies across three real-world datasets validate the superiority of NC-MS, demonstrating its ability to outperform existing models in heterogeneous graph neural networks.

Graph Anomaly Detection (GAD) identifies deviant patterns including anomalous nodes, edges, and subgraphs in graph data, with significant applications in social networks, cybersecurity, and financial risk control. While spectral methods have proven effective for homogeneous graph anomaly detection, their application to heterogeneous graphs remains challenging due to structural complexity and semantic richness. Existing heterogeneous graph anomaly detection methods either rely on manually designed meta-paths or decompose the graph into homogeneous subgraphs, leading to limited flexibility or loss of structural integrity. To address these limitations, we propose the Graph Wavelet Transformer (GWT), a novel spectral-based approach that integrates global graph properties and spectral analysis without requiring meta-path information. GWT employs a three-stage process: heterogeneous-to-homogeneous graph conversion, global dependency modeling via graph transformers, and spectral-aware feature enhancement focusing on frequency band components. Extensive experiments on multiple benchmarks demonstrate that GWT significantly outperforms ten baseline methods, providing a new paradigm for heterogeneous graph anomaly detection that preserves structural completeness while achieving computational efficiency.

Anomaly detection in continuous-time event sequences is a crucial task in safety-critical applications. While existing methods primarily focus on developing a superior test statistic, they fail to provide guarantees regarding the false positive rate (FPR), which undermines their reliability in practical deployments. In this paper, we propose CADES (Conformal Anomaly Detection in Event Sequences), a novel test procedure based on conformal inference for the studied task with finite-sample FPR control. Specifically, by using the time-rescaling theorem, we design two powerful non-conformity scores tailored to event sequences, which exhibit complementary sensitivities to different abnormal patterns. CADES combines these scores with Bonferroni correction to leverage their respective strengths and addresses non-identifiability issues of existing methods. Theoretically, we prove the validity of CADES and further provide strong guarantees on calibration-conditional FPR control. Experimental results on synthetic and real-world datasets, covering various types of anomalies, demonstrate that CADES outperforms state-of-the-art methods while maintaining FPR control.

Graph Transformers (GTs) have emerged as powerful tools for handling graph-structured data through global attention mechanisms. While GTs can effectively capture long-range dependencies, they introduce difficulties in optimization due to their complex, non-differentiable operators, which cannot be directly handled by standard gradient-based optimizers (such as Adam or AdamW). To investigate the above issues, this work adopts the line of Zeroth-Order Optimization (ZOO) technique. However, direct integration of ZOO incurs considerable challenges due to the sharp loss landscape and steep gradients within the GT parameter space. Under the above observations, we propose a Sharpness-aware Zeroth-order Optimizer (SZO) that combines Sharpness-Aware Minimization (SAM) technique facilitating convergence within a flatter neighborhood, and leverages parallel computing for efficient gradient estimation. Theoretically, we provide a comprehensive analysis of the optimizer from both convergence and generalization perspectives. Empirically, we conduct extensive experiments on various classical GTs across a wide range of benchmark datasets, which underscore the superior performance of SZO over the state-of-the-art optimizers.

Accurately modeling and forecasting complex systems governed by partial differential equations (PDEs) is crucial in various scientific and engineering domains. However, traditional numerical methods struggle in real-world scenarios due to incomplete or unknown physical laws. Meanwhile, machine learning approaches often fail to generalize effectively when faced with scarce observational data and the challenge of capturing local and global features. To this end, we propose the Physics-encoded Spectral Attention Network (PeSANet), which integrates local and global information to forecast complex systems with limited data and incomplete physical priors. The model consists of two key components: a physics-encoded block that uses hard constraints to approximate local differential operators from limited data, and a spectral-enhanced block that captures long-range global dependencies in the frequency domain. Specifically, we introduce a novel spectral attention mechanism to model inter-spectrum relationships and learn long-range spatial features. Experimental results demonstrate that PeSANet outperforms existing methods across all metrics, particularly in long-term forecasting accuracy, providing a promising solution for simulating complex systems with limited data and incomplete physics.

Cross-domain recommendation (CDR) has gained significant attention for its ability to address data sparsity issue. However, most existing CDR methods focus primarily on improving recommendation accuracy while largely overlooking fairness considerations, which can lead to biased outcomes and unfair treatment of different user groups. To solve this critical problem, we investigate whether fairness can be transferred from the source domain to the target domain. Our analysis suggests that fairness can be effectively transferred if the fairness of the source domain is ensured and the distributions of the source and target domains are well aligned. Based on this, we propose the FairCDR, a novel framework that can achieve the knowledge transfer of fairness and user preferences simultaneously. FairCDR owns two phases: single-domain fairness guarantee and inter-domain distribution alignment. In the first phase, we employ an adversarial learning-based recommender (ALR) to disentangle user preferences from sensitive attributes in the source domain. In the second phase, we introduce a new mutual learning-based diffusion model (MLDiff), which engages in mutual learning with ALR to progressively align the distributions of the source and target domains. This improves ALR's adaptability to distribution shifts, ultimately ensuring fairness and recommendation performance in the target domain. Extensive experiments on multiple real-world cross-domain datasets demonstrate that FairCDR surpasses existing strong baselines in both fairness and recommendation quality.

Multimodal recommendation, which utilizes rich multimodal information to learn user preferences, has attracted significant attention. Most works focus on designing powerful encoders for extracting multimodal features, and simply aggregate the learned features together to make prediction. Consequently, they have a limited capacity to learn the inter-modality knowledge including the modality-shared and modality-unique knowledge. In fact, learning the modality-shared knowledge enables us to align cross-modality data for fusing heterogeneous modality features. Learning the modality-unique knowledge is equally important when recommendation tasks only involve a small amount of shared features and the necessary information is contained within specific modality. In this article, we propose Contrastive Modality-Disentangled Learning (CMDL) to overcome this critical limitation. CMDL exactly captures the inter-modality knowledge by achieving modality disentanglement. Specifically, CMDL first disentangles the initial representation into the modality-invariant and modality-specific representations. Afterwards, CMDL introduces a novel manner of contrastive learning to approximate the MI upper bounds for achieving disentanglement regularization. Building upon the proposed regularization, CMDL encourages the modality-invariant and modality-specific representations to capture the modality-shared and modality-unique knowledge respectively and to be statistically independent to each other. Empirically, extensive experiments are conducted on benchmark datasets, demonstrating the superior performance of CMDL compared with strong multimodal recommenders.

Knowledge graph embedding (KGE) models achieved state-of-the-art results on many knowledge graph tasks including link prediction and information retrieval.Despite the superior performance of KGE models in practice, we discover a deficiency in the expressiveness of some popular existing KGE models called Z-paradox.Motivated by the existence of Z-paradox, we propose a new KGE model called MQuinE that does not suffer from Zparadox while preserves strong expressiveness to model various relation patterns including symmetric/asymmetric, inverse, 1-N/N-1/N-N, and composition relations with theoretical justification.Experiments on real-world knowledge bases indicate that Z-paradox indeed degrades the performance of existing KGE models, and can cause more than 20% accuracy drop on some challenging test samples.Our experiments further demonstrate that MQuinE can mitigate the negative impact of Z-paradox and outperform existing KGE models by a visible margin on link prediction tasks.

E-commerce platforms heavily rely on the wide range of products they offer. Detecting communities within these products is essential for efficient risk management and personalized recommendations. In this paper, we introduce a novel framework that utilizes unsupervised pre-trained social network computing methods to detect communities in e-commerce products. Our key technical contribution lies in capturing domain-specific knowledge from academic social networks during the pre-training phase, which is then transferred to the fine-tuning process of the unsupervised e-commerce product graph model. The framework begins with both unsupervised and supervised pre-training modules, conducting masked graph modeling and node classification tasks on academic social networks. Following this, the pre-trained graph autoencoder is fine-tuned on new datasets. In the final stage, graph clustering on node embeddings is performed using an ensemble method of fast K-means algorithms. We evaluate the proposed framework on large-scale real-world datasets from Amazon’s e-commerce platform. Results demonstrate that our method significantly outperforms other classical benchmark methods, leading the second-best by over 10% in adjusted Rand index, and discovering more than 500 subgraphs in the large-scale e-commerce network.

Recently link weight prediction has attracted an increasing research interest due to its merits in quantifying the strength between nodes within a graph. Nonetheless, current link weight prediction methods focus solely on graph topology, disregarding node feature information embedded in graphs. In real-world applications, we often collect heterogeneous graph data where multiple types of nodes linked by multiple types of edges are available for analysis, and it is essential and challenging to quantify the proximity of different types of nodes. To solve this challenge, we present a new model for Heterogeneous Graph Link Weight Prediction (HLWP for short). In HLWP, message passing in heterogeneous graph neural networks is described as a meta structure, which can be effectively designed by Differentiable Neural Architecture Search (DARTS) algorithms. Thus, HLWP can enhance the message passing in heterogeneous graphs by DARTS. In addition, HLWP employs a perturbation-based algorithm to enhance stability and precision. Through empirical experiments conducted on three real-world datasets, we demonstrate that HLWP achieves accurate predictions of link weights. Our results highlight the superiority of HLWP over existing methods for link weight prediction and baseline GNN models in terms of accurately predicting link weights within heterogeneous graphs.

Methods based on graph neural networks for solving combinatorial optimization (CO) problems have exhibited promising results in tackling a range of NP-hard problems, eliminating the necessity for reliance on manually created domain knowledge. Existing models including reinforcement learning (RL) framework assume that combinatorial instances in the training set contribute equally during training. Nevertheless, there is considerable variation in the quality of training instances, and the performance of models may suffer from the inclusion of low-quality training instances. This paper expands the current scope of neural solvers for CO problems through the incorporation of curriculum learning (CL). To alleviate the adverse impact of low-quality training instances, we propose CL4CO which utilizes CL strategy, a selective training method, to train models based on the rank of instances' quality in neural Combinatorial Optimization framework. Also, we introduce several candidate topology-aware metrics based on heterophily ratio and evaluation of clustering for the training scheduler. Furthermore, it is noteworthy to emphasize that it has potential to enhance the generalization capacity of RL-based baselines and we give a experimental validation. This enhancement plugin from the fact that CL empowers the acquired RL-based solver to effectively leverage commonly shared features within the same class of CO. Empirically, we conduct a case study on MaxCut, a classical discrete Oil-vector CO, to verify our findings and our results demonstrate that CL4CO is efficient and superiority with good generalization ability.

We present a novel perspective on temporal point processes (TPPs) by reformulating their intensity processes as solutions to stochastic differential equations (SDEs). In particular, we first prove the equivalent SDE formulations of several classical TPPs, including Poisson processes, Hawkes processes, and self-correcting processes. Based on these proofs, we introduce a unified TPP framework called Neural Jump-Diffusion Temporal Point Process (NJDTPP), whose intensity process is governed by a neural jump-diffusion SDE (NJDSDE) where the drift, diffusion, and jump coefficient functions are parameterized by neural networks. Compared to previous works, NJDTPP exhibits model flexibility in capturing intensity dynamics without relying on any specific functional form, and provides theoretical guarantees regarding the existence and uniqueness of the solution to the proposed NJDSDE. Experiments on both synthetic and real-world datasets demonstrate that NJDTPP is capable of capturing the dynamics of intensity processes in different scenarios and significantly outperforms the state-of-the-art TPP models in prediction tasks.

The concept of the sharing economy has gained broad recognition, and within this context, Sharing E-Bike Battery (SEB) have emerged as a focal point of societal interest. Despite the popularity, a notable discrepancy remains between user expectations regarding the remaining battery range of SEBs and the reality, leading to a pronounced inclination among users to find an available SEB during emergency situations. In response to this challenge, the integration of Artificial Intelligence of Things (AIoT) and battery-swap services has surfaced as a viable solution. In this paper, we propose a novel structural Transformer-based model, referred to as the SEB-Transformer, designed specifically for predicting the battery range of SEBs. The scenario is conceptualized as a dynamic heterogeneous graph that encapsulates the interactions between users and bicycles, providing a comprehensive framework for analysis. Furthermore, we incorporate the graph structure into the SEB-Transformer to facilitate the estimation of the remaining e-bike battery range, in conjunction with mean structural similarity, enhancing the prediction accuracy. By employing the predictions made by our model, we are able to dynamically adjust the optimal cycling routes for users in real-time, while also considering the strategic locations of charging stations, thereby optimizing the user experience. Empirically our results on real-world datasets demonstrate the superiority of our model against nine competitive baselines. These innovations, powered by AIoT, not only bridge the gap between user expectations and the physical limitations of battery range but also significantly improve the operational efficiency and sustainability of SEB services. Through these advancements, the shared electric bicycle ecosystem is evolving, making strides towards a more reliable, user-friendly, and sustainable mode of transportation.

In the context of privacy-aware social web services, secure causal reasoning on graphs is an essential task, for example, identifying the key influential nodes/users through which information, ideas, trends spread across the network. Despite the significant advancements achieved by neural network-based methods in these domains, existing research often assumes complete observable of graph data, an assumption that conflicts with privacy concerns in social web services. This assumption may not hold in numerous real-world social analysis applications due to privacy restrictions. In such scenarios, some graph links might remain unobserved until the model is trained, posing a critical challenge for privacy preservation. To address this issue, we introduce a novel model named Causal Effect on Coarsened Graphs (Cecoar), which is specifically designed for privacy-aware social web services. This model employs a new type of coarsened graph that accurately represents partially observed nodes and edges, aligning with the privacy constraints of social web services. We have developed three crucial modules to enable causal estimation on coarsened graphs: (1) a graph sampling module, (2) a confounder encoder, and (3) a graph-based encoder with causal prediction layers. Furthermore, we present a comprehensive theoretical analysis of the proposed model, demonstrating its performance and generalization capabilities. We evaluate our model on two benchmark datasets for causal estimation, and the results confirm its effectiveness in handling graph data containing private information. The code is in https://github.com/liu-yang-maker/Cecoar-ICWS24.

Decision-focused learning (DFL) combines both machine learning and combinatorial optimization so as to enhance the quality of decision-making. In general, DFL adds an optimization layer after the neural network and solves a focused combinatorial optimization problem. The optimization layer is usually based on KKT conditions or surrogate functions. However, the optimizer associated with the DFL is inflexible and requires a large amount of expert knowledge. Furthermore, it has been shown that the optimizer exhibits a lack of robustness and differentiability when confronted with complex tasks, especially in the graph domain. To solve this problem, we study a more generic situation by taking the optimization layer as a black-box operator on graphs. Then, we design a mixed zeroth-order optimization to differentiate the layer. Specifically, we combine DFL with graph neural networks and present a decision-focused graph neural network named ZO4Graph based on a mixed zeroth-order optimizer. Empirically, we conduct extensive experiments compared with two-stage models on community detection. Numerical results demonstrate that our proposed framework outperforms its peers.

Anomaly detection in multi-type event sequences is a crucial and challenging problem with important applications in various domains, including cybersecurity, finance and healthcare. Temporal point process has emerged as a powerful technique for modeling event sequences and has gained considerable attention in the field of anomaly detection. However, existing temporal point process approaches are either inapplicable to multi-type event sequence data or incur the loss of valuable information in subsequences associated with specific event types. To this end, we propose a novel Multiple Hypothesis Testing based Anomaly Detection method (MultiAD) to detect anomalous multi-type event sequences. The basic idea of MultiAD is to capture the underlying distribution of normal sequences using a neural multivariate point process, based on which the original hypothesis testing problem can be converted into a multiple hypothesis testing using the multivariate time rescaling theorem. By conducting multiple hypothesis tests on the time-rescaled subsequences, MultiAD makes full use of the valuable information contained within individual subsequences. Moreover, we claim that the existing test statistic ignores the sequential information of inter-event time intervals and propose new statistics to address this shortcoming. Finally, we employ the kernel method to obtain a smooth estimator of the distribution of the proposed statistics under the null hypothesis. This ensures a more accurate and reliable computation of the p-value, providing robust statistical inference. Extensive experimental results demonstrate that MultiAD significantly outperforms the state-of-the-art methods on both synthetic and real-world data.

Markov logic network (MLN) is a powerful statistical modeling framework for probabilistic logic reasoning. Despite the elegancy and effectiveness of MLN, the inference of MLN is known to suffer from an efficiency issue. Even the state-of-the-art MLN engines can not scale to medium-size real-world knowledge bases in the open-world setting, i.e., all unobserved facts in the knowledge base need predictions. In this work, by focusing on a certain class of first-order logic rules that are sufficiently expressive, we develop a highly efficient MLN inference engine called MLN4KB that can leverage the sparsity of knowledge bases. MLN4KB enjoys quite strong theoretical properties; its space and time complexities can be exponentially smaller than existing MLN engines. Experiments on both synthetic and real-world knowledge bases demonstrate the effectiveness of the proposed method. MLN4KB is orders of magnitudes faster (more than 103 times faster on some datasets) than existing MLN engines in the open-world setting. Without any approximation tricks, MLN4KB can scale to real-world knowledge bases including WN-18 and YAGO3-10 and achieve decent prediction accuracy without bells and whistles.

Graph neural networks (GNNs) are powerful models to handle graph data and can achieve state-of-the-art in many critical tasks including node classification and link prediction. However, existing graph neural networks still face both challenges of over-smoothing and over-squashing based on previous literature. To this end, we propose a new Curvature-based topology-aware Dropout sampling technique named CurvDrop, in which we integrate the Discrete Ricci Curvature into graph neural networks to enable more expressive graph models. Also, this work can improve graph neural networks by quantifying connections in graphs and using structural information such as community structures in graphs. As a result, our method can tackle the both challenges of over-smoothing and over-squashing with theoretical justification. Also, numerous experiments on public datasets show the effectiveness and robustness of our proposed method. The code and data are released in https://github.com/liu-yang-maker/Curvature-based-Dropout.

WiFi fingerprint-based localization is regarded as one of the most promising techniques for indoor localization. However, this raises serious privacy concerns. Current approaches to mitigate the privacy concerns rely on the encryption with large calculation consumption. In this paper, we propose a data obfuscation mechanism based on the generalized version of differential privacy. We extend the standard definition to the indoor WiFi fingerprint data for spatial counting where the inputs belong to multiple dimensions of numerical data in a limited range. With a given privacy budget, the proposed method generalizes the original dataset, and then specializes it using differential privacy. As the designed novel scheme expand the range for specialization, the data set released by the proposed algorithm can yield better mining results. Furthermore, experimental results give out comparisons between nonuniform and uniform ε selection scheme, and find uniform ε selection scheme can fully use the privacy budget in our situation.