Real-world graphs or networks are usually heterogeneous, involving multiple types of nodes and relationships. Heterogeneous graph neural networks (HGNNs) can effectively handle these diverse nodes and edges, capturing heterogeneous information within the graph, thus exhibiting outstanding performance. However, most methods of HGNNs usually involve complex structural designs, leading to problems such as high memory usage, long inference time, and extensive consumption of computing resources. These limitations pose certain challenges for the practical application of HGNNs, especially for resource-constrained devices. To mitigate this issue, we propose the Spiking Heterogeneous Graph Attention Networks (SpikingHAN), which incorporates the brain-inspired and energy-saving properties of Spiking Neural Networks (SNNs) into heterogeneous graph learning to reduce the computing cost without compromising the performance. Specifically, SpikingHAN aggregates metapath-based neighbor information using a single-layer graph convolution with shared parameters. It then employs a semantic-level attention mechanism to capture the importance of different meta-paths and performs semantic aggregation. Finally, it encodes the heterogeneous information into a spike sequence through SNNs, simulating bioinformatic processing to derive a binarized 1-bit representation of the heterogeneous graph. Comprehensive experimental results from three real-world heterogeneous graph datasets show that SpikingHAN delivers competitive node classification performance. It achieves this with fewer parameters, quicker inference, reduced memory usage, and lower energy consumption.

Graph Transformers (GTs), which integrate message passing and self-attention mechanisms simultaneously, have achieved promising empirical results in graph prediction tasks. However, the design of scalable and topology-aware node tokenization has lagged behind other modalities. This gap becomes critical as the quadratic complexity of full attention renders them impractical on large-scale graphs. Recently, Spiking Neural Networks (SNNs), as brain-inspired models, provided an energy-saving scheme to convert input intensity into discrete spike-based representations through event-driven spiking neurons. Inspired by these characteristics, we propose a linear-time Graph Transformer with Spiking Node Tokenization (GT-SNT) for node classification. By integrating multi-step feature propagation with SNNs, spiking node tokenization generates compact, locality-aware spike count embeddings as node tokens to avoid predefined codebooks and their utilization issues. The codebook guided self-attention leverages these tokens to perform node-to-token attention for linear-time global context aggregation. In experiments, we compare GT-SNT with other state-of-the-art baselines on node classification datasets ranging from small to large. Experimental results show that GT-SNT achieves comparable performances on most datasets and reaches up to 130× faster inference speed compared to other GTs.

Individual fairness (IF) in graph neural networks (GNNs), which emphasizes the need for similar individuals should receive similar outcomes from GNNs, has been a critical issue. Despite its importance, research in this area has been largely unexplored in terms of (1) a clear understanding of what induces individual unfairness in GNNs and (2) a comprehensive consideration of identifying similar individuals. To bridge these gaps, we conduct a preliminary analysis to explore the underlying reason for individual unfairness and observe correlations between IF andsimilarity consistency, a concept introduced to evaluate the discrepancy in identifying similar individuals based on graph structure versus node features. Inspired by our observations, we introduce two metrics to assess individual similarity from two distinct perspectives: topology fusion and feature fusion. Building upon these metrics, we proposeSimilarity-awareGNNs forIndividualFairness, namedSaGIF. The key insight behind SaGIF is the integration of individual similarities by independently learning similarity representations, leading to an improvement of IF in GNNs. Our experiments on several real-world datasets validate the effectiveness of our proposed metrics and SaGIF. Specifically, SaGIF consistently outperforms state-of-the-art IF methods while maintaining utility performance.

The in-context learning (ICL) capability of large language models (LLMs) enables them to perform challenging tasks using provided demonstrations. However, ICL is highly sensitive to the ordering of demonstrations, leading to instability in predictions. This paper shows that this vulnerability can be exploited to design a natural attack—difficult for model providers to detect—that achieves nearly 80% success rate on LLaMA-3 by simply permuting the demonstrations. Existing mitigation methods primarily rely on post-processing and fail to enhance the model's inherent robustness to input permutations, raising concerns about safety and reliability of LLMs. To address this issue, we propose Permutation-resilient learning (PEARL), a novel framework based on distributionally robust optimization (DRO), which optimizes model performance against the worst-case input permutation. Specifically, PEARL consists of a permutation-proposal network (P-Net) and the LLM. The P-Net generates the most challenging permutations by treating it as an optimal transport problem, which is solved using an entropy-constrained Sinkhorn algorithm. Through minimax optimization, the P-Net and the LLM iteratively optimize against each other, progressively improving the LLM's robustness. Experiments on synthetic pre-training and real-world instruction tuning tasks demonstrate that PEARL effectively mitigates permutation attacks and enhances performance. Notably, despite being trained on fewer shots and shorter contexts, PEARL achieves performance gains of up to 40% when scaled to many-shot and long-context scenarios, highlighting its efficiency and generalization capabilities.

Harmful fine-tuning (HFT), performed directly on open-source LLMs or through Fine-tuning-as-a-Service, breaks safety alignment and poses significant threats. Existing methods aim to mitigate HFT risks by learning robust representation on alignment data or making harmful data unlearnable, but they treat each data sample equally, leaving data vulnerability patterns understudied. In this work, we reveal that certain subsets of alignment data are consistently more prone to forgetting during HFT across different fine-tuning tasks and exhibit lower robustness compared to other subsets. Inspired by these findings, we propose Vulnerability-Aware Alignment (VAA), which calculates data vulnerability, partitions data into "vulnerable" and "invulnerable" groups, and encourages balanced learning using a group distributionally robust optimization (Group DRO) framework. Specifically, VAA learns an adversarial sampler that samples examples from the currently underperforming group and then applies group-dependent adversarial perturbations to the data during training, aiming to encourage a balanced learning process across groups. Experiments across four fine-tuning tasks demonstrate that VAA significantly reduces harmful scores while preserving downstream task performance, outperforming state-of-the-art baselines.

Inspired by the impressive capabilities of GPT-4o, there is growing interest in enabling speech language models (SLMs) to engage in natural, fluid spoken interactions with humans. Recent advancements have led to the development of several SLMs that demonstrate promising results in this area. However, current approaches have yet to fully exploit dual-channel speech data, which inherently captures the structure and dynamics of human conversation. In this work, we systematically explore the use of dual-channel speech data in the context of modern large language models, and introduce a novel generative modeling paradigm—Next-Token-Pair Prediction (NTPP)—to enable speaker-independent dual-channel spoken dialogue learning using decoder-only architectures for the first time. We evaluate our approach on standard benchmarks, and empirical results show that our proposed method, NTPP, significantly improves the conversational abilities of SLMs in terms of turn-taking prediction, response coherence, and naturalness. Moreover, compared to existing methods, NTPP achieves substantially lower inference latency, highlighting its practical efficiency for real-time applications. Demo and code can be found at https://audio-3059.pages.dev.

Large language models (LLMs) are widely adopted to generate synthetic datasets for various natural language processing (NLP) tasks, such as text classification and summarization. However, accurately measuring the diversity of these synthetic datasets—an aspect crucial for robust model performance—remains a significant challenge. In this paper, we introduce DCScore, a novel method for measuring synthetic dataset diversity from a classification perspective. Specifically, DCScore formulates diversity evaluation as a sample classification task, leveraging mutual relationships among samples. We further provide theoretical verification of the diversity-related axioms satisfied by DCScore, highlighting its role as a principled diversity evaluation method. Experimental results on synthetic datasets reveal that DCScore enjoys a stronger correlation with multiple diversity pseudo-truths of evaluated datasets, underscoring its effectiveness. Moreover, both empirical and theoretical evidence demonstrate that DCScore substantially reduces computational costs compared to existing methods. Code is available at: https://github.com/bluewhalelab/dcscore.

Real-world graphs are typically complex, exhibiting heterogeneity in the global structure, as well as strong heterophily within local neighborhoods. While a growing body of literature has revealed the limitations of graph neural networks (GNNs) in handling homogeneous graphs with heterophily, little work has been conducted on investigating the heterophily properties in the context of heterogeneous graphs. To bridge this research gap, we identify the heterophily in heterogeneous graphs using metapaths and propose two practical metrics to quantitatively describe the levels of heterophily. Our empirical investigations on real-world heterogeneous graphs have revealed that heterogeneous graph neural networks (HGNNs), which inherit many mechanisms from GNNs designed for homogeneous graphs, struggle to generalize to heterogeneous graphs with heterophily or low levels of homophily. To address the challenge, we present Hetero$^{2}$2Net, a heterophily-aware HGNN that incorporates masked metapath prediction and masked label prediction tasks to effectively and flexibly handle both homophilic and heterophilic heterogeneous graphs. We evaluate the performance of Hetero$^{2}$2Net on five real-world heterogeneous graph benchmarks with varying levels of heterophily. Experimental results demonstrate that Hetero$^{2}$2Net outperforms strong baselines in the semi-supervised node classification task. In particular, Hetero$^{2}$2Net scales to an industrial-scale commercial graph with 13 M nodes and 157 M edges, demonstrating its effectiveness in handling large and complex heterogeneous graphs.

Liang Chen, Yatao Bian, Yang Deng, Deng Cai, Shuaiyi Li, Peilin Zhao, Kam-Fai Wong. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

As the typical retraining paradigm is unacceptably time-and resource-consuming, researchers are turning to model editing to find an effective way that supports both consecutive and batch scenarios to edit the model behavior directly.Despite all these practical expectations, existing model editing methods fail to realize all of them.Furthermore, the memory demands for such sequential model editing approaches tend to be prohibitive, frequently necessitating an external memory that grows incrementally over time.To cope with these challenges, we propose CoachHooK, a model editing method that simultaneously supports sequential and batch editing.CoachHooK is memory-friendly as it only needs a small amount of it to store several hook layers whose size remains unchanged over time.Experimental results demonstrate the superiority of our method over other batch-supportive model editing methods under both single-round and consecutive batch editing scenarios.Extensive analyses of CoachHooK have been conducted to verify the stability of our method over a number of consecutive steps.

While contrastive self-supervised learning has become the de-facto learning paradigm for graph neural networks, the pursuit of higher task accuracy requires a larger hidden dimensionality to learn informative and discriminative full-precision representations, raising concerns about computation, memory footprint, and energy consumption burden (largely overlooked) for real-world applications. This work explores a promising direction for graph contrastive learning (GCL) with spiking neural networks (SNNs), which leverage sparse and binary characteristics to learn more biologically plausible and compact representations. We propose SpikeGCL, a novel GCL framework to learn binarized 1-bit representations for graphs, making balanced trade-offs between efficiency and performance. We provide theoretical guarantees to demonstrate that SpikeGCL has comparable expressiveness with its full-precision counterparts. Experimental results demonstrate that, with nearly 32x representation storage compression, SpikeGCL is either comparable to or outperforms many fancy state-of-the-art supervised and self-supervised methods across several graph benchmarks.

Low-rank adaptation (LoRA) has recently gained much interest in fine-tuning foundation models. It effectively reduces the number of trainable parameters by incorporating low-rank matrices $A$ and $B$ to represent the weight change, i.e., $\Delta W=BA$. Despite LoRA’s progress, it faces storage challenges when handling extensive customization adaptations or larger base models. In this work, we aim to further compress trainable parameters by enjoying the powerful expressiveness of the Fourier transform. Specifically, we introduce FourierFT, which treats $\Delta W$ as a matrix in the spatial domain and learns only a small fraction of its spectral coefficients. With the trained spectral coefficients, we implement the inverse discrete Fourier transform to recover $\Delta W$. Empirically, our FourierFT method shows comparable or better performance with fewer parameters than LoRA on various tasks, including natural language understanding, natural language generation, instruction tuning, and image classification. For example, when performing instruction tuning on the LLaMA2-7B model, FourierFT surpasses LoRA with only 0.064M trainable parameters, compared to LoRA’s 33.5M. Our code is released at this link.

Benefit from the widespread implement of data collection technologies, a large number of data services are emerged in various industries like finance and energy. The complex internal connectivity within these services can be naturally modeled as the graph-structured data, which facilitates services’ understanding based on graph mining. However, the heavy label reliance limits the development of traditional graph learning methods on data services. Graph contrastive learning (GCL), as a popular graph representation learning approach without relying on manual labels, has recently achieved a non-negligible effect. Thus, for alleviating above problems, we propose a simple yet effective method named FastGCL containing a better contrastive scheme which can be tailored to the characteristics of graph neural networks (e.g., neighborhood aggregation). Specifically, by constructing weighted-aggregated and non-aggregated neighborhood information as positive and negative samples respectively, FastGCL identifies the potential semantic information of data without disturbing the graph topology and node attributes, resulting in faster training and convergence speeds. Extensive experiments have been conducted on node classification and graph classification tasks, showing that FastGCL has competitive classification performance and significant training speedup compared to existing state-of-the-art methods.

Graph clustering, a fundamental and challenging task in graph mining, aims to classify nodes in a graph into several disjoint clusters. In recent years, graph contrastive learning (GCL) has emerged as a dominant line of research in graph clustering and advances the new state-of-the-art. However, GCL-based methods heavily rely on graph augmentations and contrastive schemes, which may potentially introduce challenges such as semantic drift and scalability issues. Another promising line of research involves the adoption of modularity maximization, a popular and effective measure for community detection, as the guiding principle for clustering tasks. Despite the recent progress, the underlying mechanism of modularity maximization is still not well understood. In this work, we dig into the hidden success of modularity maximization for graph clustering. Our analysis reveals the strong connections between modularity maximization and graph contrastive learning, where positive and negative examples are naturally defined by modularity. In light of our results, we propose a community-aware graph clustering framework, coined øurs, which leverages modularity maximization as a contrastive pretext task to effectively uncover the underlying information of communities in graphs, while avoiding the problem of semantic drift. Extensive experiments on multiple graph datasets verify the effectiveness of øurs in terms of scalability and clustering performance compared to state-of-the-art graph clustering methods. Notably, øurs easily scales a sufficiently large graph with 100M nodes while outperforming strong baselines.

Recent studies have highlighted fairness issues in Graph Neural Networks (GNNs), where they produce discriminatory predictions against specific protected groups categorized by sensitive attributes such as race and age. While various efforts to enhance GNN fairness have made significant progress, these approaches are often tailored to specific sensitive attributes. Consequently, they necessitate retraining the model from scratch to accommodate changes in the sensitive attribute requirement, resulting in high computational costs. To gain deeper insights into this issue, we approach the graph fairness problem from a causal modeling perspective, where we identify the confounding effect induced by the sensitive attribute as the underlying reason. Motivated by this observation, we formulate the fairness problem in graphs from an invariant learning perspective, which aims to learn invariant representations across environments. Accordingly, we propose a graph fairness framework based on invariant learning, namely FairINV, which enables the training of fair GNNs to accommodate various sensitive attributes within a single training session. Specifically, FairINV incorporates sensitive attribute partition and trains fair GNNs by eliminating spurious correlations between the label and various sensitive attributes. Experimental results on several real-world datasets demonstrate that FairINV significantly outperforms state-of-the-art fairness approaches, underscoring its effectiveness. Our code is available via: https://github.com/ZzoomD/FairINV/.

Zezhong Wang, Fangkai Yang, Lu Wang, Pu Zhao, Hongru Wang, Liang Chen, Qingwei Lin, Kam-Fai Wong. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

Over the past few years, research on deep graph learning has shifted from static graphs to temporal graphs in response to real-world complex systems that exhibit dynamic behaviors. In practice, temporal graphs are formalized as an ordered sequence of static graph snapshots observed at discrete time points. Sequence models such as RNNs or Transformers have long been the predominant backbone networks for modeling such temporal graphs. Yet, despite the promising results, RNNs struggle with long-range dependencies, while transformers are burdened by quadratic computational complexity. Recently, state space models (SSMs), which are framed as discretized representations of an underlying continuous-time linear dynamical system, have garnered substantial attention and achieved breakthrough advancements in independent sequence modeling. In this work, we undertake a principled investigation that extends SSM theory to temporal graphs by integrating structural information into the online approximation objective via the adoption of a Laplacian regularization term. The emergent continuous-time system introduces novel algorithmic challenges, thereby necessitating our development of GraphSSM, a graph state space model for modeling the dynamics of temporal graphs. Extensive experimental results demonstrate the effectiveness of our GraphSSM framework across various temporal graph benchmarks.

Graph contrastive learning (GCL) has emerged as a representative paradigm in graph self-supervised learning, where negative samples are commonly regarded as the key to preventing model collapse and producing distinguishable representations. Recent studies have shown that GCL without negative samples can achieve state-of-the-art performance as well as scalability improvement, with bootstrapped graph latent (BGRL) as a prominent step forward. However, BGRL relies on a complex architecture to maintain the ability to scatter representations, and the underlying mechanisms enabling the success remain largely unexplored. In this paper, we introduce an instance-level decorrelation perspective to tackle the aforementioned issue and leverage it as a springboard to reveal the potential unnecessary model complexity within BGRL. Based on our findings, we present SGCL, a simple yet effective GCL framework that utilizes the outputs from two consecutive iterations as positive pairs, eliminating the negative samples. SGCL only requires a single graph augmentation and a single graph encoder without additional parameters. Extensive experiments conducted on various graph benchmarks demonstrate that SGCL can achieve competitive performance with fewer parameters, lower time and space costs, and significant convergence speedup.

Graph neural networks (GNNs) are being increasingly used in many high-stakes tasks, and as a result, there is growing attention on their fairness recently. GNNs have been shown to be unfair as they tend to make discriminatory decisions toward certain demographic groups, divided by sensitive attributes such as gender and race. While recent works have been devoted to improving their fairness performance, they often require accessible demographic information. This greatly limits their applicability in real-world scenarios due to legal restrictions. To address this problem, we present a demographic-agnostic method to learn fair GNNs via knowledge distillation, namely FairGKD. Our work is motivated by the empirical observation that training GNNs on partial data (i.e., only node attributes or topology data) can improve their fairness, albeit at the cost of utility. To make a balanced trade-off between fairness and utility performance, we employ a set of fairness experts (i.e., GNNs trained on different partial data) to construct the synthetic teacher, which distills fairer and informative knowledge to guide the learning of the GNN student. Experiments on several benchmark datasets demonstrate that FairGKD, which does not require access to demographic information, significantly improves the fairness of GNNs by a large margin while maintaining their utility.\footnoteOur code is available via: \code.

Group fairness for Graph Neural Networks (GNNs), which emphasizes algorithmic decisions neither favoring nor harming certain groups defined by sensitive attributes (e.g., race and gender), has gained considerable attention. In particular, the objective of group fairness is to ensure that the decisions made by GNNs are independent of the sensitive attribute. To achieve this objective, most existing approaches involve eliminating sensitive attribute information in node representations or algorithmic decisions. However, such ways may also eliminate task-related information due to its inherent correlation with the sensitive attribute, leading to a sacrifice in utility. In this work, we focus on improving the fairness of GNNs while preserving task-related information and propose a fair GNN framework named FairSAD. Instead of eliminating sensitive attribute information, FairSAD enhances the fairness of GNNs via Sensitive Attribute Disentanglement (SAD), which separates the sensitive attribute-related information into an independent component to mitigate its impact. Additionally, FairSAD utilizes a channel masking mechanism to adaptively identify the sensitive attribute-related component and subsequently decorrelates it. Overall, FairSAD minimizes the impact of the sensitive attribute on GNN outcomes rather than eliminating sensitive attributes, thereby preserving task-related information associated with the sensitive attribute. Furthermore, experiments conducted on several real-world datasets demonstrate that FairSAD outperforms other state-of-the-art methods by a significant margin in terms of both fairness and utility performance. Our source code is available at https://github.com/ZzoomD/FairSAD.

Numerous mobile applications (apps) with different functions meet the various needs of users, but users have to spend a lot of time selecting suitable mobile apps. How to select relevant mobile apps for users has become an important issue. Existing studies mainly utilise context, user interest, privacy, security, version, and heterogeneous information to make mobile app recommendations. However, they have at least one of the following limitations: (1) Don't fully integrate the rich heterogeneous information; (2) Don't capture complex structural and semantic information; (3) Don't differentiate the importance of different semantic meta-graphs; (4) Don't consider the influence of different users' rating criteria. Therefore, the predictive performance of these methods is relatively limited. This paper considers the influence of different users' rating criteria for the same app and proposes a weighted meta-graph based mobile app recommendation approach by leveraging matrix factorisation and neural networks. Specifically, the similarity measurement between users and apps considers the difference in users' rating criteria under various semantic meta-graph patterns. The matrix factorisation technology is used to acquire the user's and the app's latent feature matrices. The importance of various semantic meta-graphs is distinguished by exploiting the weight learning. The neural network technology is employed to learn interactions between users and apps, thereby predicting the user's preference for unobserved apps. Experimental results demonstrate the superiority of the proposed approach, the effectiveness of considering differences in users' rating criteria, and the importance of differentiating various semantic meta-graphs.

Recommendation system plays a remarkable role in solving the problem of information overload on the Internet. Existing research demonstrates that a recommended list enclosed with appropriate explanations can enhance the transparency of the system and encourage users to make decisions. Although existing works have achieved effective results, they still suffer from at least one of the following limitations: the work either does not use sentiment information or review information, does not explicitly incorporate review-level sentiment information into the model, is based on review retrieval, and generates explanations in the form of templates or phrases. To tackle the above limitations, this article proposes a REview-level Sentiment information enhanced multiTask learning approach for Explainable Recommendation (RESTER). Specifically, it first considers the user’s review information and analyzes the sentiment polarity contained in the review. Then, the user/item’s identity feature, review feature, and sentiment information are fused into a multitask learning framework by leveraging the implicit correlation between the rating prediction and explanation generation tasks. Comprehensive experiments on datasets in three different domains have shown that the proposed model is superior to all other baselines in both rating prediction and explanation generation tasks.

Graph neural networks (GNNs) have shown intrinsic topology bias inherited from graph-structured data, where a majority of nodes are associated with specific sensitive attributes (e.g., age, and race). In this regard, GNNs make discriminatory decisions toward certain groups defined by the sensitive attribute. Over the past few years, efforts have been made to mitigate the fairness issue of GNNs caused by topology bias. While achieving impressive results, these works likely only scratch the surface of what is possible with well-tuned GNN models or heuristic graph preprocessing techniques. Despite modern GNNs being built upon the message-passing framework, few works dig into the hidden reasoning behind the fairness issue of GNNs from such a fundamental perspective. In this work, we empirically demonstrate that message aggregation with higher edge weight for intergroup edges improves model fairness but sacrifices utility. The above observations motivate us to propose and derive a simple yet effective message-passing scheme,FairAGG, leading to more fair GNNs with less comprised downstream performance. Specifically, FairAGG measures the contribution of graph topology on fairness usingShapley value, which facilitates fair aggregation through reweighting. Experiments on several real-world datasets demonstrate that FairAGG enhances the fairness of the GNNs model while maintaining competitive utility performance.

Recent years have seen a surge in research on dynamic graph representation learning, which aims to model temporal graphs that are dynamic and evolving constantly over time. However, current work typically models graph dynamics with recurrent neural networks (RNNs), making them suffer seriously from computation and memory overheads on large temporal graphs. So far, scalability of dynamic graph representation learning on large temporal graphs remains one of the major challenges. In this paper, we present a scalable framework, namely SpikeNet, to efficiently capture the temporal and structural patterns of temporal graphs. We explore a new direction in that we can capture the evolving dynamics of temporal graphs with spiking neural networks (SNNs) instead of RNNs. As a low-power alternative to RNNs, SNNs explicitly model graph dynamics as spike trains of neuron populations and enable spike-based propagation in an efficient way. Experiments on three large real-world temporal graph datasets demonstrate that SpikeNet outperforms strong baselines on the temporal node classification task with lower computational costs. Particularly, SpikeNet generalizes to a large temporal graph (2.7M nodes and 13.9M edges) with significantly fewer parameters and computation overheads.

Graph convolutional networks (GCNs) have been shown to be vulnerable to small adversarial perturbations, which becomes a severe threat and largely limits their applications in security-critical scenarios. To mitigate such a threat, considerable research efforts have been devoted to increasing the robustness of GCNs against adversarial attacks. However, current defense approaches are typically designed to prevent GCNs from untargeted adversarial attacks and focus on overall performance, making it challenging to protect important local nodes from more powerful targeted adversarial attacks. Additionally, a trade-off between robustness and performance is often made in existing research. Such limitations highlight the need for developing an effective and efficient approach that can defend local nodes against targeted attacks, without compromising the overall performance of GCNs. In this work, we present a simple yet effective method, named Graph Universal AdveRsarial Defense (GUARD). Unlike previous works, GUARD protects each individual node from attacks with a universal defensive patch, which is generated once and can be applied to any node (node-agnostic) in a graph. GUARD is fast, straightforward to implement without any change to network architecture nor any additional parameters, and is broadly applicable to any GCNs. Extensive experiments on four benchmark datasets demonstrate that GUARD significantly improves robustness for several established GCNs against multiple adversarial attacks and outperforms state-of-the-art defense methods by large margins.

Graph neural networks (GNNs) have achieved state-of-the-art performance in representation learning for graphs recently. However, the effectiveness of GNNs, which capitalize on the key operation of message propagation, highly depends on the quality of the topology structure. Most of the graphs in real-world scenarios follow a long-tailed distribution on their node degrees, that is, a vast majority of the nodes in the graph are tail nodes with only a few connected edges. GNNs produce inferior node representations for tail nodes due to the lack of sufficient structural information. In the pursuit of promoting the performance of GNNs for tail nodes, we explore how the deficiency of structural information deteriorates the performance of tail nodes and propose a general structural augmentation based tailno de representation learning framework, dubbed as øurs, which can jointly learn to augment the graph structure and extract more informative representations for tail nodes. Extensive experiments on six public benchmark datasets demonstrate that øurs outperforms the state-of-the-art methods for tail node representation learning.

Large language models (LLMs) outperform information retrieval techniques for downstream knowledge-intensive tasks when being prompted to generate world knowledge.Yet, community concerns abound regarding the factuality and potential implications of using this uncensored knowledge.In light of this, we introduce CONNER, a COmpreheNsive kNowledge Evaluation fRamework, designed to systematically and automatically evaluate generated knowledge from six important perspectives -Factuality, Relevance, Coherence, Informativeness, Helpfulness and Validity.We conduct an extensive empirical analysis of the generated knowledge from three different types of LLMs on two widely-studied knowledge-intensive tasks, i.e., open-domain question answering and knowledge-grounded dialogue.Surprisingly, our study reveals that the factuality of generated knowledge, even if lower, does not significantly hinder downstream tasks.Instead, the relevance and coherence of the outputs are more important than small factual mistakes.Further, we show how to use CONNER to improve knowledge-intensive tasks by designing two strategies: Prompt Engineering and Knowledge Selection.Our evaluation code and LLMgenerated knowledge with human annotations will be released 1 to facilitate future research.

Personalized recommendation is becoming increasingly important in online information systems in the current era of information explosion. In real-world scenarios, when a user considers which items to consume, the decision choice may be affected by her friends. For example, she may ask her friends for suggestions or be attracted by products purchased by one friend. As such, to provide satisfactory recommendation service, it is important to account for the evidence in social relations when they are available to use. Several prior efforts have been made to leverage social relations to build the recommender system and verified their utility. However, most existing methods, such as the well-known TrustSVD, leverage only first-order social relations, i.e., the direct neighbors that are connected to the target user. The high-order social relations, e.g., the friends of friends, which are very informative to reveal user preference, have been largely ignored.

The data generated in many real-world applications can be modeled as heterogeneous graphs of multi-typed entities (nodes) and relations (links). Nowadays, such data are commonly generated and stored by distributed clients, making direct centralized model training unpractical. While the data in each client are prone to biased local distributions, generalizable global models are still in frequent need for large-scale applications. However, the large number of clients enforce significant computational overhead due to the communication and synchronization among the clients, whereas the biased local data distributions indicate that not all clients and parameters should be computed and updated at all times. Motivated by specifically designed preliminary studies on training a state-of-the-art heterogeneous graph neural network (HGN) with the vanilla FedAvg framework, in this work, we propose to leverage the characteristics of heterogeneous graphs by designing dynamic activation strategies for the clients and parameters during the federated training of HGN, named FedDA. Moreover, we design a novel disentangled model D-HGN to enable type-oriented activation of model parameters for FedDA. The effectiveness and efficiency of our proposed techniques are backed by both theoretical and empirical analysis– We theoretically analyze the validity and convergence of FedDA and mathematically illustrate its efficiency gain; meanwhile, we demonstrate the significant performance gains of FedDA and corroborate its efficiency gains with extensive experiments over multiple realistic FL settings synthesized based on real-world heterogeneous graphs.

Graph convolutional networks (GCNs) play a critical role in improving the performance of collaborative filtering. They leverage the concept of aggregating neighbor information to capture user preferences on bipartite graphs by stacking multiple convolutional layers. However, this requirement for layer stacking often leads to a long training time for convergence, and results in indistinguishable representations with significant performance deterioration due to the problem of oversmoothing. Additionally, the noise of interactions will be amplified by the stacking of convolutional layers through message passing. To address these issues, we propose a simple, plug-and-play-Neighborhood Structure -Embedding approach, named NSE, which utilizes first-order adjacency information to construct structural embeddings. By explicitly incorporating local topologically statistical information before message passing, the embeddings propagated at GCNs have better topology-structure awareness. This leads to an improved optimization path and greater robustness against noise propagation. Experimental results demonstrate significant performance improvements by employing our proposed NSE in graph collaborative filtering models. Particularly, the NSE-enhanced LGCN shows performance gains of 5.06% and 4.86% on the Yelp and Amazon-Books datasets, respectively. The average training convergence speed is improved by 204.8%. NSE-enhanced graph collaborative filtering has also demonstrated excellent robustness against both noise and oversmoothing.

Anomaly detection aims to distinguish abnormal instances that deviate significantly from the majority of benign ones. As instances that appear in the real world are naturally connected and can be represented with graphs, graph neural networks become increasingly popular in tackling the anomaly detection problem. Despite the promising results, research on anomaly detection has almost exclusively focused on static graphs while the mining of anomalous patterns from dynamic graphs is rarely studied but has significant application value. In addition, anomaly detection is typically tackled from semi-supervised perspectives due to the lack of sufficient labeled data. However, most proposed methods are limited to merely exploiting labeled data, leaving a large number of unlabeled samples unexplored. In this work, we present semi-supervised anomaly detection (SAD), an end-to-end framework for anomaly detection on dynamic graphs. By a combination of a time-equipped memory bank and a pseudo-label contrastive learning module, SAD is able to fully exploit the potential of large unlabeled samples and uncover underlying anomalies on evolving graph streams. Extensive experiments on four real-world datasets demonstrate that SAD efficiently discovers anomalies from dynamic graphs and outperforms existing advanced methods even when provided with only little labeled data.

The last years have witnessed the emergence of a promising self-supervised learning strategy, referred to as masked autoencoding. However, there is a lack of theoretical understanding of how masking matters on graph autoencoders (GAEs). In this work, we present masked graph autoencoder (MaskGAE), a self-supervised learning framework for graph-structured data. Different from standard GAEs, MaskGAE adopts masked graph modeling (MGM) as a principled pretext task - masking a portion of edges and attempting to reconstruct the missing part with partially visible, unmasked graph structure. To understand whether MGM can help GAEs learn better representations, we provide both theoretical and empirical evidence to comprehensively justify the benefits of this pretext task. Theoretically, we establish close connections between GAEs and contrastive learning, showing that MGM significantly improves the self-supervised learning scheme of GAEs. Empirically, we conduct extensive experiments on a variety of graph benchmarks, demonstrating the superiority of MaskGAE over several state-of-the-arts on both link prediction and node classification tasks.

Federated Recommender Systems (FedRecs) are considered privacy-preserving techniques to collaboratively learn a recommendation model without sharing user data. Since all participants can directly influence the systems by uploading gradients, FedRecs are vulnerable to poisoning attacks of malicious clients. However, most existing poisoning attacks on FedRecs are either based on some prior knowledge or with less effectiveness. To reveal the real vulnerability of FedRecs, in this paper, we present a new poisoning attack method to manipulate target items' ranks and exposure rates effectively in the top-K recommendation without relying on any prior knowledge. Specifically, our attack manipulates target items' exposure rate by a group of synthetic malicious users who upload poisoned gradients considering target items' alternative products. We conduct extensive experiments with two widely used FedRecs (Fed-NCF and Fed-LightGCN) on two real-world recommendation datasets. The experimental results show that our attack can significantly improve the exposure rate of unpopular target items with extremely fewer malicious users and fewer global epochs than state-of-the-art attacks. In addition to disclosing the security hole, we design a novel countermeasure for poisoning attacks on FedRecs. Specifically, we propose a hierarchical gradient clipping with sparsified updating to defend against existing poisoning attacks. The empirical results demonstrate that the proposed defending mechanism improves the robustness of FedRecs.

Abstract Blockchain and blockchain‐based decentralised applications have been attracting increasing attention recently. In public blockchain systems, users usually connect to third‐party peers or run a peer to join the P2P blockchain network. However, connecting to unreliable blockchain peers will lead to resource waste and even loss of cryptocurrencies by repeated transactions. In order to select reliable blockchain peers, it is urgently needed to evaluate and predict their reliability of them. Faced with this problem, we propose hybrid blockchain reliability prediction (H‐BRP), a Hybrid Blockchain Reliability Prediction model, to extract the blockchain reliability factors and then make the personalised prediction for each user. Comprehensive experiments conducted on 100 blockchain requesters and 200 blockchain peers demonstrate the effectiveness of the proposed H‐BRP model. Further, the implementation and dataset of 2,000,000 test cases are released.

Recent studies demonstrate that Graph Neural Networks (GNNs) are vulnerable to slight but adversarially designed perturbations, known asadversarial examples. To address this issue, robust training methods against adversarial examples have received considerable attention in the literature.Adversarial Training (AT)is a successful approach to learning a robust model using adversarially perturbed training samples. Existing AT methods on GNNs typically construct adversarial perturbations in terms of graph structures or node features. However, they are less effective and fraught with challenges on graph data due to the discreteness of graph structure and the relationships between connected examples. In this work, we seek to address these challenges and proposeSpectralAdversarialTraining (SAT), a simple yet effective adversarial training approach for GNNs. SAT first adopts a low-rank approximation of the graph structure based on spectral decomposition, and then constructs adversarial perturbations in the spectral domain rather than directly manipulating the original graph structure. To investigate its effectiveness, we employ SAT on three widely used GNNs. Experimental results on four public graph datasets demonstrate that SAT significantly improves the robustness of GNNs against adversarial attacks without sacrificing classification accuracy and training efficiency.

Recent studies have shown that graph neural networks (GNNs) are vulnerable against perturbations due to lack of robustness and can therefore be easily fooled. Currently, most works on attacking GNNs are mainly using gradient information to guide the attack and achieve outstanding performance. However, the high complexity of time and space makes them unmanageable for large scale graphs and becomes the major bottleneck that prevents the practical usage. We argue that the main reason is that they have to use the whole graph for attacks, resulting in the increasing time and space complexity as the data scale grows. In this work, we propose an efficient Simplified Gradient-based Attack (SGA) method to bridge this gap. SGA can cause the GNNs to misclassify specific target nodes through a multi-stage attack framework, which needs only a much smaller subgraph. In addition, we present a practical metric named Degree Assortativity Change (DAC) to measure the impacts of adversarial attacks on graph data. We evaluate our attack method on four real-world graph networks by attacking several commonly used GNNs. The experimental results demonstrate that SGA can achieve significant time and memory efficiency improvements while maintaining competitive attack performance compared to state-of-art attack techniques.

Although some existing models are proposed to exploit reviews for improving performance for recommender systems, few of them can handle the following issues led by the insufficient review data: (i) The regular training process does not exactly fit the scenario of preference prediction with few historical behaviors. (ii) Extracting informative and sufficient semantic features from limited review texts is a challenging work. To alleviate these issues, this paper proposes an enhanced prototypical network, FS-EPN, that leverages reviews for recommendation under the few-shot setting. FS-EPN consists of an attentional prototypical network being the basic architecture, a sentiment encoder and a memory collector cooperating to capture the extra sentimental and collaborative information from both user and item perspectives for semantic information supplement. We train FS-EPN under the meta-learning framework, which models the training process in the episodic manner to mimic the few-shot test environment. Extensive experiments conducted on six publicly available datasets demonstrate the superior capability of FS-EPN over several state-of-the-art models in few-shot recommendation.

Graph Convolutional Networks (GCNs) achieve an impressive performance due to the remarkable representation ability in learning the graph information. However, GCNs, when implemented on a deep network, require expensive computation power, making them difficult to be deployed on battery-powered devices. In contrast, Spiking Neural Networks (SNNs), which perform a bio-fidelity inference process, offer an energy-efficient neural architecture. In this work, we propose SpikingGCN, an end-to-end framework that aims to integrate the embedding of GCNs with the biofidelity characteristics of SNNs. The original graph data are encoded into spike trains based on the incorporation of graph convolution. We further model biological information processing by utilizing a fully connected layer combined with neuron nodes. In a wide range of scenarios (e.g., citation networks, image graph classification, and recommender systems), our experimental results show that the proposed method could gain competitive performance against state-of-the-art approaches. Furthermore, we show that SpikingGCN on a neuromorphic chip can bring a clear advantage of energy efficiency into graph data analysis, which demonstrates its great potential to construct environment-friendly machine learning models.

Deep graph learning (DGL) has achieved remarkable progress in both business and scientific areas ranging from finance and e-commerce, to drug and advanced material discovery. Despite the progress, how to ensure various DGL algorithms behave in a socially responsible manner and meet regulatory compliance requirements becomes an emerging problem, especially in risk-sensitive domains. Trustworthy graph learning (TwGL) aims to solve the above problems from a technical viewpoint. In contrast to conventional graph learning which mainly cares about model performance, TwGL considers various reliability and safety aspects of DGL, including but not limited to adversarial robustness, explainability, and privacy protection. Whilst several previous tutorials have been made for the introduction of DGL in KDD, seldom is there a special focus on its safety aspects, including reliability, explainability, and privacy protection capability. This tutorial mainly covers the key achievements of trustworthy graph learning in recent years. Specifically, we will discuss three essential topics, that is, the reliability of DGL against inherent noise, distribution shift and adversarial attack, explainability methods, and privacy protection for DGL. Meanwhile, we will introduce some guidelines for applying DGL to risk-sensitive applications (e.g., AI drug discovery) to ensure GNN models behave in a trustworthy way. We hope our tutorial can offer a comprehensive review of recent advances in this area and also provide some useful suggestions to guide the developers to choose appropriate techniques for their applications.

The prevalence of online social network makes it compulsory to study how social relations affect user choice. However, most existing methods leverage only first-order social relations, that is, the direct neighbors that are connected to the target user. The high-order social relations, e.g., the friends of friends, which are very informative to reveal user preference, have been largely ignored. In this work, we focus on modeling the indirect influence from the high-order neighbors in social networks to improve the performance of item recommendation. Distinct from mainstream social recommenders that regularize the model learning with social relations, we instead propose to directly factor social relations in the predictive model, aiming at learning better user embeddings to improve recommendation. To address the challenge that high-order neighbors increase dramatically with the order size, we propose to recursively “propagate” embeddings along the social network, effectively injecting the influence of high-order neighbors into user representation. We conduct experiments on two real datasets of Yelp and Douban to verify ourHigh-Order Social Recommender(HOSR) model. Empirical results show that our HOSR significantly outperforms recent graph regularization-based recommenders NSCR and IF-BPR$^+$, and graph convolutional network-based social influence prediction model DeepInf, achieving new state-of-the-arts of the task.

In recent years, advances in Graph Convolutional Networks (GCNs) have given new insights into the development of social recommendation. However, many existing GCN-based social recommendation methods often directly apply GCN to capture user-item and user-user interactions, which probably have two main limitations: (a) Due to the power-law property of the degree distribution, the vanilla GCN with static normalized adjacency matrix has limitations in learning node representations, especially for the long-tail nodes; (b) multi-typed social relationships between users that are ubiquitous in the real world are rarely considered. In this article, we propose a novel Bilateral Filtering Heterogeneous Attention Network (BFHAN), which improves long-tail node representations and leverages multi-typed social relationships between user nodes. First, we propose a novel graph convolutional filter for the user-item bipartite network and extend it to the user-user homogeneous network. Further, we theoretically analyze the correlation between the convergence values of different graph convolutional filters and node degrees after stacking multiple layers. Second, we model multi-relational social interactions between users as the multiplex network and further propose a multiplex attention network to capture distinctive inter-layer influences for user representations. Last but not least, the experimental results demonstrate that our proposed method outperforms several state-of-the-art GCN-based methods for social recommendation tasks.

In recent years, there are great interests as well as many challenges in applying reinforcement learning (RL) to recommendation systems (RS). In this paper, we summarize three key practical challenges of large-scale RL-based recommender systems: massive state and action spaces, high-variance environment, and the unspecific reward setting in recommendation. All these problems remain largely unexplored in the existing literature and make the application of RL challenging. We develop a model-based reinforcement learning framework, called GoalRec. Inspired by the ideas of world model (model-based), value function estimation (model-free), and goal-based RL, a novel disentangled universal value function designed for item recommendation is proposed. It can generalize to various goals that the recommender may have, and disentangle the stochastic environmental dynamics and high-variance reward signals accordingly. As a part of the value function, free from the sparse and high-variance reward signals, a high-capacity reward-independent world model is trained to simulate complex environmental dynamics under a certain goal. Based on the predicted environmental dynamics, the disentangled universal value function is related to the user's future trajectory instead of a monolithic state and a scalar reward. We demonstrate the superiority of GoalRec over previous approaches in terms of the above three practical challenges in a series of simulations and a real application.

Cross-domain recommendation technique is a promising way to alleviate data sparsity issues by transferring knowledge from an auxiliary domain to a target domain. However, most existing works focus on utilizing the same users among different domains, while ignoring domain-specific users which forms the majority in real-world circumstances. In this paper, we propose a novel cross-domain learning approach--Relation Expansion based Cross-Domain Recommendation (ReCDR) to improve recommendation accuracies on small-overlapped domains. ReCDR first models the interactions in each domain as a local graph. It then forms a shared network by expanding out relationships using pre-trained node similarities. On the enhanced graph, ReCDR adopts a hierarchical attention mechanism. The output embedding will finally be combined with the local feature to balance the result for dual-target task. The proposed model is thoroughly evaluated on three real-world datasets. Experiments demonstrate superior performance compared to state-of-the-art methods.

The sequential recommendation has been widely used to predict users' preferences in the near future by utilizing their dynamic interactions with items. However, existing methods only consider single-typed interactions (e.g., purchase), ignoring the rich heterogeneous information such as multi-typed interactions (e.g., click, purchase) and item attributes (e.g, category), which leads to a suboptimal model. We can integrate this rich information by introducing Dynamic Heterogeneous Information Networks (DHINs). Our solution contains three special designs: 1) Static Initialization; 2) Heterogeneous User Memory Network; 3) Two-level attention mechanism. Extensive experiments conducted on two real-world datasets show that our model outperforms other state-of-the-art solutions. Furthermore, we provide some insights into parameter settings and model interpretability.

Recent studies have shown that Graph Convolutional Networks (GCNs) are vulnerable to adversarial attacks on the graph structure. Although multiple works have been proposed to improve their robustness against such structural adversarial attacks, the reasons for the success of the attacks remain unclear. In this work, we theoretically and empirically demonstrate that structural adversarial examples can be attributed to the non-robust aggregation scheme (i.e., the weighted mean) of GCNs. Specifically, our analysis takes advantage of the breakdown point which can quantitatively measure the robustness of aggregation schemes. The key insight is that weighted mean, as the basic design of GCNs, has a low breakdown point and its output can be dramatically changed by injecting a single edge. We show that adopting the aggregation scheme with a high breakdown point (e.g., median or trimmed mean) could significantly enhance the robustness of GCNs against structural attacks. Extensive experiments on four real-world datasets demonstrate that such a simple but effective method achieves the best robustness performance compared to state-of-the-art models.

Many data mining and analytical tasks rely on the abstraction of networks (graphs) to summarize relational structures among individuals (nodes). Since relational data are often sensitive, we aim to seek effective approaches to generate utility-preserved yet privacy-protected structured data. In this paper, we leverage the differential privacy (DP) framework to formulate and enforce rigorous privacy constraints on deep graph generation models, with a focus on edge-DP to guarantee individual link privacy. In particular, we enforce edge-DP by injecting designated noise to the gradients of a link reconstruction based graph generation model, while ensuring data utility by improving structure learning with structure-oriented graph discrimination. Extensive experiments on two real-world network datasets show that our proposed DPGGAN model is able to generate graphs with effectively preserved global structure and rigorously protected individual link privacy.

Recommender systems rely on user behavior data like ratings and clicks to build personalization model. However, the collected data is observational rather than experimental, causing various biases in the data which significantly affect the learned model. Most existing work for recommendation debiasing, such as the inverse propensity scoring and imputation approaches, focuses on one or two specific biases, lacking the universal capacity that can account for mixed or even unknown biases in the data.

Representation learning on user-item graph for recommendation has evolved from using single ID or interaction history to exploiting higher-order neighbors. This leads to the success of graph convolution networks (GCNs) for recommendation such as PinSage and LightGCN. Despite effectiveness, we argue that they suffer from two limitations: (1) high-degree nodes exert larger impact on the representation learning, deteriorating the recommendations of low-degree (long-tail) items; and (2) representations are vulnerable to noisy interactions, as the neighborhood aggregation scheme further enlarges the impact of observed edges.

Heterogeneous information networks consist of multiple types of edges and nodes, which have a strong ability to represent the rich semantics underpinning network structures. Recently, the dynamics of networks has been studied in many tasks such as social media analysis and recommender systems. However, existing methods mainly focus on the static networks or dynamic homogeneous networks, which are incapable or inefficient in modeling dynamic heterogeneous information networks. In this paper, we propose a method named Dynamic Heterogeneous Information Network Embedding (DyHINE), which can update embeddings when the network evolves. The method contains two key designs: (1) A dynamic time-series embedding module which employs a hierarchical attention mechanism to aggregate neighbor features and temporal random walks to capture dynamic interactions; (2) An online real-time updating module which efficiently updates the computed embeddings via a dynamic operator. Experiments on three real-world datasets demonstrate the effectiveness of our model compared with state-of-the-art methods on the task of temporal link prediction.

These years much effort has been devoted to improving the accuracy or relevance of the recommendation system. Diversity, a crucial factor which measures the dissimilarity among the recommended items, received rather little scrutiny. Directly related to user satisfaction, diversification is usually taken into consideration after generating the candidate items. However, this decoupled design of diversification and candidate generation makes the whole system suboptimal. In this paper, we aim at pushing the diversification to the upstream candidate generation stage, with the help of Graph Convolutional Networks (GCN). Although GCN based recommendation algorithms have shown great power in modeling complex collaborative filtering effect to improve the accuracy of recommendation, how diversity changes is ignored in those advanced works. We propose to perform rebalanced neighbor discovering, category-boosted negative sampling and adversarial learning on top of GCN. We conduct extensive experiments on real-world datasets. Experimental results verify the effectiveness of our proposed method on diversification. Further ablation studies validate that our proposed method significantly alleviates the accuracy-diversity dilemma.

Blockchain has attracted an increasing amount of researches, and there are lots of refreshing implementations in different fields. Cryptocurrency as its representative implementation, suffers the economic loss due to phishing scams. In our work, accounts and transactions are treated as nodes and edges, thus detection of phishing accounts can be modeled as a node classification problem. Correspondingly, we propose a detecting method based on Graph Convolutional Network and autoencoder to precisely distinguish phishing accounts. Experiments on different large-scale real-world datasets from Ethereum show that our proposed model consistently performs promising results compared with related methods.

In business domains, bundling is one of the most important marketing strategies to conduct product promotions, which is commonly used in online e-commerce and offline retailers. Existing recommender systems mostly focus on recommending individual items that users may be interested in. In this paper, we target at a practical but less explored recommendation problem named bundle recommendation, which aims to offer a combination of items to users. To tackle this specific recommendation problem in the context of the virtual mall in online games, we formalize it as a link prediction problem on a user-item-bundle tripartite graph constructed from the historical interactions, and solve it with a neural network model that can learn directly on the graph-structure data. Extensive experiments on three public datasets and one industrial game dataset demonstrate the effectiveness of the proposed method. Further, the bundle recommendation model has been deployed in production for more than one year in a popular online game developed by Netease Games, and the launch of the model yields more than 60% improvement on conversion rate of bundles, and a relative improvement of more than 15% on gross merchandise volume (GMV).

Win prediction and performance evaluation are two core subjects in the sport analytics. Traditionally, they are treated separately and studied by two independent communities. However, this is not the intuitive way how humans interpret the matches: we predict the match results with the competition carrying on, and simultaneously evaluate each action based on the game context and its downstream impact. Predicting the match outcomes and evaluating the actions are coupled tasks, and the more accurately we predict, the better the evaluation is

Behavior tracing or predicting is a key component in various application scenarios like online user modeling and ubiquitous computing, which significantly benefits the system design (e.g., resource pre-caching) and improves the user experience (e.g., personalized recommendation). Traditional behavior tracing methods like Markovian and sequential models take recent behaviors as input and infer the next move by using the most real-time information. However, these existing methods rarely comprehensively model the low-level temporal irregularity in the recent behavior sequence, i.e., the unevenly distributed time intervals between consecutive behaviors, and the high-level periodicity in the long-term activity cycle, i.e., the periodic behavior patterns of each user.

Recently, deep learning-based models have been widely studied for click-through rate (CTR) prediction and lead to improved prediction accuracy in many industrial applications. However, current research focuses primarily on building complex network architectures to better capture sophisticated feature interactions and dynamic user behaviors. The increased model complexity may slow down online inference and hinder its adoption in real-time applications. Instead, our work targets at a new model training strategy based on knowledge distillation (KD). KD is a teacher-student learning framework to transfer knowledge learned from a teacher model to a student model. The KD strategy not only allows us to simplify the student model as a vanilla DNN model but also achieves significant accuracy improvements over the state-of-the-art teacher models. The benefits thus motivate us to further explore the use of a powerful ensemble of teachers for more accurate student model training. We also propose some novel techniques to facilitate ensembled CTR prediction, including teacher gating and early stopping by distillation loss. We conduct comprehensive experiments against 12 existing models and across three industrial datasets. Both offline and online A/B testing results show the effectiveness of our KD-based training strategy.

Traditional recommendation systems estimate user preference on items from past interaction history, thus suffering from the limitations of obtaining fine-grained and dynamic user preference. Conversational recommendation system (CRS) brings revolutions to those limitations by enabling the system to directly ask users about their preferred attributes on items. However, existing CRS methods do not make full use of such advantage --- they only use the attribute feedback in rather implicit ways such as updating the latent user representation. In this paper, we propose Conversational Path Reasoning (CPR), a generic framework that models conversational recommendation as an interactive path reasoning problem on a graph. It walks through the attribute vertices by following user feedback, utilizing the user preferred attributes in an explicit way. By leveraging on the graph structure, CPR is able to prune off many irrelevant candidate attributes, leading to a better chance of hitting user-preferred attributes. To demonstrate how CPR works, we propose a simple yet effective instantiation named SCPR (Simple CPR). We perform empirical studies on the multi-round conversational recommendation scenario, the most realistic CRS setting so far that considers multiple rounds of asking attributes and recommending items. Through extensive experiments on two datasets Yelp and LastFM, we validate the effectiveness of our SCPR, which significantly outperforms the state-of-the-art CRS methods EAR and CRM. In particular, we find that the more attributes there are, the more advantages our method can achieve.

Factorization machines (FMs) have been widely adopted to model the discrete feature interactions in recommender systems. Despite their great success, currently there is no study of their robustness to discrete adversarial perturbations. Whether modifying a certain number of the discrete input features has a dramatic effect on the FM's prediction? Although there exist robust training methods for FMs, they neglect the discrete property of input features and lack of an effective mechanism to verify the model robustness.

Fault detection is a fundamental requirement for Industrial Internet of Things (IIoT), such as the process industry. This article first reviews the recent studies focusing on applying the fault detection techniques to the IIoT networks. However, we find that numerous studies focus on the resource utilization and workload allocation. The fault detection toward IIoT facilities is still in its immature stage because the existing approaches are not accurate enough for the stringent fault detection in IIoT networks. To this end, we present a novel algorithm, named Gaussian Bernoulli restricted Boltzmann machines (GBRBMs)-based deep neural network (DNN), to transform the fault detection into a classification problem. The real trace-driven experiments show that the proposed scheme outperforms other baseline machine learning methods. We anticipate that this article can inspire blooming studies on the related topics of smart IIoT networks.

Predicting what items a user will consume in the next time (i.e., next-item recommendation) is a crucial task for recommender systems. While the factorization method is a popular choice in recommendation, several recent efforts have shown that the inner product does not satisfy the triangle inequality, which may hurt the model's generalization ability. TransRec is a promising method to overcome this issue, which learns a distance metric to predict the strength of user-item interactions. Nevertheless, such method only uses the latest consumed item to model a user's short-term preference, which is insufficient for modeling fidelity. In this article, we propose a simple yet effective method named attentive translation model, to explicitly exploit high-order sequential information for next-item recommendation. Specifically, we construct a user-specific translation vector by accounting for multiple recent items, which encode more information about a user's short-term preference than the latest item. To aggregate multiple items into one representation, we devise a position-aware attention mechanism, learning different weights on items at different orders in a personalized way. Extensive experiments on four real-world datasets show that our method significantly outperforms several state-of-the-art methods.

A large number of spatial knowledge graphs (SKGs) are available from spatially enriched knowledge bases, e.g., DBpedia and YAGO2. This provides a great chance to understand valuable information about the regions surrounding us. However, it is hard to comprehend SKGs due to the explosively growing volume and the complication of the graph structures. Thus we study the problem of similar region search (SRS), which is an easy-to-use but effective way to explore spatial data. The effectiveness of SRS highly depends on how to measure the region similarity. However, existing approaches cannot make use of the rich information contained in SKGs thus may lead to incorrect results. In this paper, we propose a spatial knowledge representation learning method for region similarity, namely SKRL4RS. SKRL4RS firstly encodes the spatial entities of an SKG into a vector space to make it easier to extract useful features. Then regions are represented by 3-D tensors using the spatial entity embeddings together with geographical information. Finally, region tensors are fed into the conventional triplet network to learn the feature vectors of regions. The region similarity measure learned by SKRL4RS can capture the hierarchical types, semantic relatedness, and relative locations of spatial entities inside a region. Experimental results on two real-world datasets show that our SKRL4RS outperforms the state-of-the-art by a significant margin in terms of the accuracy of measuring region similarity.

Service recommendation is widely used to locate developers' desired services. Previous methods mainly focus on employing collaborative filtering (CF) techniques to recommend services to developers. However, these methods have some problems, such as being sensitive to sparse data and having limited predictive ability to new developers. Generative adversarial network (GAN) can solve the above mentioned problems, since it can learn the data distribution from a limited amount of data and generate a new developer's preference score for a service, even if he/she has not invoked the service. In this paper, we propose a novel GAN based service recommendation method. It first constructs a heterogeneous information network (HIN) by utilizing mashup information, service information and their respective attribute information. Then, it samples meta-paths of different semantic relationships and constructs similarity matrices between mashups and services through meta-paths based similarity measurement. Finally, by leveraging the adversarial training between the discriminator and the generator, the discriminator can effectively guide the generator to generate a preference vector for the developer, thus recommending a list of services for him/her according to his/her given mashup attribute information. Comprehensive experimental results on a real-world dataset demonstrate the superiority of the proposed method.

Most recommendation research has been concentrated on recommending single items to users, such as the considerable work on collaborative filtering that models the interaction between a user and an item. However, in many real-world scenarios, the platform needs to show users a set of items, e.g., the marketing strategy that offers multiple items for sale as one bundle.In this work, we consider recommending a set of items to a user, i.e., the Bundle Recommendation task, which concerns the interaction modeling between a user and a set of items. We contribute a neural network solution named DAM, short for Deep Attentive Multi-Task model, which is featured with two special designs: 1) We design a factorized attention network to aggregate the item embeddings in a bundle to obtain the bundle's representation; 2) We jointly model user-bundle interactions and user-item interactions in a multi-task manner to alleviate the scarcity of user-bundle interactions. Extensive experiments on a real-world dataset show that DAM outperforms the state-of-the-art solution, verifying the effectiveness of our attention design and multi-task learning in DAM.

With the explosive growth of services, including Web services, cloud services, APIs and mashups, discovering the appropriate services for consumers is becoming an imperative issue. The traditional service discovery approaches mainly face two challenges: 1) the single source of description documents limits the effectiveness of discovery due to the insufficiency of semantic information; 2) more factors should be considered with the generally increasing functional and nonfunctional requirements of consumers. In this paper, we propose a novel framework, called SMS, for effectively discovering the appropriate services by incorporating social media information. Specifically, we present different methods to measure four social factors (semantic similarity, popularity, activity, decay factor) collected from Twitter. Latent Semantic Indexing (LSI) model is applied to mine semantic information of services from meta-data of Twitter Lists that contains them. In addition, we assume the target query-service matching function as a linear combination of multiple social factors and design a weight learning algorithm to learn an optimal combination of the measured social factors. Comprehensive experiments based on a real-world dataset crawled from Twitter demonstrate the effectiveness of the proposed framework SMS, through some compared approaches.

Heterogeneous Information Network(HIN) has been employed in recommender system to represent heterogeneous types of data, and meta path has been proposed to capture semantic relationship among objects. When applying HIN to the recommendation, there are two problems: how to extract features from meta paths and how to properly fuse these features to further improve recommendations. Some recent work has employed deep neural network to learn user and item representation, and attention mechanism has been explored to integrate information for recommendation. Inspired by these work, in this paper, we propose Heterogeneous Neural Attentive Factorization Machine(HNAFM) to solve above problems. Specifically, we first calculate the commuting matrices based on meta paths and use multilayer perceptrons to learn user and item features. A hierarchical attention mechanism is employed to find the meta path that best describes user's preference and item's property. Comprehensive experiments based on real-world datasets demonstrate that the proposed HNAFM significantly outperforms state-of-the-art rating prediction methods.

It is important to transfer the knowledge from label-rich source domain to unlabeled target domain due to the expensive cost of manual labeling efforts. Prior domain adaptation methods address this problem through aligning the global distribution statistics between source domain and target domain, but a drawback of prior methods is that they ignore the semantic information contained in samples, e.g., features of backpacks in target domain might be mapped near features of cars in source domain. In this paper, we present moving semantic transfer network, which learn semantic representations for unlabeled target samples by aligning labeled source centroid and pseudo-labeled target centroid. Features in same class but different domains are expected to be mapped nearby, resulting in an improved target classification accuracy. Moving average centroid alignment is cautiously designed to compensate the insufficient categorical information within each mini batch. Experiments testify that our model yields state of the art results on standard datasets.

With the wide adoption of SOA (Service Oriented Architecture), a massive amount of innovative applications emerge in the Internet. One of the popular representations is mashup. It is a new application created by combining different kinds of services. There exist multiple typed objects (e.g., mashup, service, category, tag, provider and description) and relations (e.g., compose and composed by relation between mashups and services, provide and provided by relation between services and providers), which constitute a heterogeneous information network (HIN) naturally. Several approaches already exist for recommending services for users but they are limited to consider only one or two kinds of relations between mashups and services. To apply the rich semantics and enhance recommendation performance, in this paper, we propose a Factorization Machine based service Recommendation approach, called FMRec, on HIN. Specifically, we firstly apply counting-based similarities for meta paths to capture the multiple semantic meanings between mashups and services. And then, we employ matrix factorization to the similarity matrices built by different kinds of meta paths to obtain the mashup latent features and service latent features. Finally, we leverage factorization machine model with a group lasso regularization term to learn the ratings between mashups and services. Comprehensive experiments are conducted on a real-world dataset, indicating that our proposed service recommendation approach significantly improves the quality of the recommendation results compared with existing methods.

With the rapid prevalence of smart mobile devices and the dramatic proliferation of mobile applications (Apps), App recommendation becomes an emergent task that will benefit different stockholders of mobile App ecosystems. Unlike traditional items, Apps have privileges to access a user's sensitive resources (e.g., contacts, messages and locations) which may lead to security risk or privacy leak. Thus, users' choosing of Apps are influenced by not only their personal interests but also their privacy preferences. Moreover, user privacy preferences vary with App categories. In this paper, we propose a mobile sparse additive generative model (Mobi-SAGE) to recommend Apps by considering both user interests and category-aware user privacy preferences. We collected a real-world dataset from 360 App store - the biggest Android App platform in China, and conduct extensive experiments on it. The experimental results show that our Mobi-SAGE consistently and significantly outperforms the state-of-the-art approaches, which implies the importance of exploiting category-aware user privacy preferences.

With the rapid development of mobile apps, the availability of a large number of mobile apps in application stores brings challenges to locate appropriate apps for users. Providing accurate mobile app recommendation for users becomes an imperative task. Conventional approaches mainly focus on learning users' preferences and app features to predict the user-app ratings. However, most of them did not consider the interactions among the context information of apps. To address this issue, we propose a broad learning approach for Context-Aware app recommendation with Tensor Analysis (CATA). Specifically, we utilize a tensor-based framework to effectively integrate app category information and multi-view features on users and apps, respectively, to facilitate the performance of rating prediction. The multidimensional structure is employed to capture the hidden relationships among the app categories and the multiview features. We develop an efficient factorization method which applies Tucker decomposition to learn the full-order interactions among the app categories and features. Furthermore, we employ a group ℓ1-norm regularization to learn the group-wise feature importance of each view with respect to each app category. Experiments on a real-world mobile app dataset demonstrate the effectiveness of the proposed method.

With the proliferation of mobile application (app) markets (e.g., Google Play, Apple App Store), predicting user preferences on apps becomes a challenging problem. Different from previous work, we assume that a user likes an app because he/she likes certain features of the app (e.g., permission, genre, topic). Based on this assumption, we propose a feature-oriented approach to predict user preferences on apps. Specifically, we transform the original app rating matrix to feature rating data and predict the unknown ratings on the features through a latent factor model, instead of directly predicting ratings on apps. The predicted user ratings on features can be used to generate the ratings on apps. Two integration methods are presented to give different significance for feature preferences. The approach has some obvious advantages: as it integrates feature information to analyze the details of user preference, it can generalize better as the feature rating data is denser, and improve the interpretation of the prediction of app ratings. Experimental results on a real-world dataset demonstrate the effectiveness of the proposed approach.

With an increasing number of Web services on-line, service recommendation becomes an important approach to help user discover suitable services. However, current service recommendation approaches only consider user's historical preferences and functional or non-functional properties of services. The compositional side of services to form mashups is overlooked. However, the service composition information is a valuable information for recommendation since it gives an implicit measure of relatedness of services. In this paper, we recognize the importance of mashups on service recommendation and propose to jointly model historical preference of users on mashups and services, compositional information of services as mashups, and functional information of services and mashups in a single framework. Specifically, we extract topics from functional descriptions of services and model the relations between user, mashup and service and topics as a quadripartite graph. We use a personalized ranking on graph algorithm to learn the proposed model and simultaneously recommend services and mashups for user. We conduct a comprehensive experimental study using real world data from ProgrammbleWeb and results show that our recommendation method outperforms other representative recommendation approaches.

As web-enabled software becomes the standard for business processes, the ways organizations, partners and customers interface with it have become a critical differentiator in the market place, i.e., API Economy. With the rapid proliferation of APIs, it is increasingly important for users to effectively manage objective APIs in kinds of API markets, e.g., ProgramableWeb (PW), Mashape, etc. In this paper, to facilitate the process of API management, we propose a graphbased recommendation approach called ATRec to automatically assign tags to unlabeled APIs by exploiting both graph structure information and semantic similarity. Specifically, ATRec first leverages the multi-type relations (i.e., among APIs, mashups, and mashup assigned tags) to construct a heterogeneous network, in which a Random Walk with Restart (RWR) model is applied to alleviate the total cold start problem where no API has ever been tagged. Furthermore, we apply the recommended API tags in two API management scenarios (API search, API recommendation). Comprehensive experiments based on a real dataset crawled from PW demonstrate the effectiveness of the proposed approach.

Modern service industry (MSI) is an information and knowledge intensive service industry, relying on information technology and modern management philosophy. The development of MSI is of high significance for promoting rapid global economy, accelerating social progress, and building an innovation-oriented society and harmonious realm. This paper first presents the history and trends of MIS in China in terms of the worldwide MSI development status. It then proposes the concept of Crossover Services based upon a survey of 62 MSI-related listed firms in China, whose business models, products, and services evidently exploit the concept. After elaborating on the crossover, convergence, and complex characteristics of crossover services, it proposes a technical framework that facilities addressing the scientific issues and technical challenges on crossover service realization. Finally, it illustrates how a new cloud-based middleware platform, named JTang++, supports the realization of crossover services.

With the development of Service Computing and Big Data research, more and more heterogeneous data generated in the process of Service Computing attracts our attention. Combining correlated data sources may help improve the performance of a given task. For example, in service recommendation, one can combine (1) user profile data (e.g. Genders, age, etc.), (2) user log data (e.g., Click through data, service invocation records, etc.), (3) QoS data (e.g. Response time, cost, etc.), (4) service functional description (e.g., Service name, WSDL document, etc.) and (5) service tagging data (i.e., Tags annotated by users) to build a recommendation model. All these data sources provide informative but heterogeneous features. For instance, user profile and QoS data usually have nominal features reflecting users' background and services' qualities, log data provides term-based features about users' historical behaviors, and service functional description and tagging data have term-based features reflecting services' functionalities and users' collective opinions. Given multiple heterogeneous data sources, one important challenge is to find a unified feature subspace to capture the knowledge from all data sources. To handle this problem, in this paper, we propose a Heterogeneous Feature Selection framework, named as WS-HFS, in which the consensus and the weight of different sources are both considered. Moreover, we apply the proposed framework to Web service clustering as a case study, and compare it with the state of the art approaches. The comprehensive experiments based on real data demonstrate the effectiveness of WS-HFS.

With the wide adoption of Service-Oriented Architecture (SOA), the number of web accessible services and their compositions is increasing rapidly. Among huge number of services, how to recommend appropriate ones for automatic composition satisfying users' need is challenging. We investigate services and their compositions in Programmable Web which characterize services as APIs and their compositions as mashups. We study the problem of recommending suitable APIs satisfying users' need for mash up creation. To this end, we propose a manifold ranking framework for API recommendation. First, we categorize existing mashups into functionally similar clusters. Then we recommend APIs for each mash up cluster using manifold ranking algorithm which incorporate the relationships between mashups, between APIs and between mashups and APIs. Intuitively, we take three factors into consideration: (1) We recommend APIs that are in functionally similar mashups. (2) We recommend APIs that are popular in the mashups. (3) We recommend APIs that are similar to each other. Finally, we map a user's requirement for mash up creation to a mash up cluster and recommend APIs generated by the algorithm to user. Experiments based on real dataset crawled from Programmble Web demonstrate the effectiveness of the proposed approach in terms of precision, recall, and NDCG.

Application Programming Interfaces (APIs), which are emerging web services in general, are increasing with a rapid speed in recent years. With so many APIs, many management platforms have been developed and deployed, leading to the boom of API markets, that are similar to the mobile App markets. Meanwhile, it has become more and more difficult to select and manage APIs. In reality, most existing management platforms typically recommend currently popular APIs to developers. However, the fact that popularity of API varies over time is ignored in those platforms, leading to the difficulty of recommending APIs that are just released but may be popular in the near future. To tackle this challenge, an approach of predicting the popularity of APIs is proposed in this paper. Predicting the popularity of API can not only be used for API ranking, recommendation and selection, but also make it more convenient for API providers and consumers to manage or select API respectively. In this paper, we propose a time-aware linear model to predict the API popularity, using time series feature of APIs and API's self-features such as its' provider ranking and description features, which are called heterogeneous features in our paper. Comprehensive experiments have been conducted on a real-world Programmable Web dataset with 613 real APIs. The experimental results show that our model has a better performance, when compared with some other state-of-the-art prediction models.

The past decade has witnessed a fast growth of web-based services, making discovery of user desired services from a large and diverse service space a fundamental challenge. Service clustering has been demonstrated as a promising solution by automatically detecting functionally similar services so that they can be searched and discovered together. In this way, both the efficiency and accuracy of service discovery can be improved. However, the autonomous nature of service providers leads to highly diverse usage of terms in their respective service descriptions. Furthermore, a typical service description is comprised of very limited terms due to the small number of (and focused) functionalities offered by the service. These unique characteristics make service descriptions different from regular text documents, which poses additional challenges when clustering large-scale services. Recent works show that service clustering can benefit from discovery and use of functionality-related latent factors to represent services as opposed to a large and diverse set of terms. Nonetheless, how to determine the total number of latent functional factors and sparsely assign them to each service description arises as a central challenge, especially for a large service space where there is no easy way to enumerate the types of different functionalities. In this paper, we propose a machine learning method that automatically learns the number of latent functional factors in a service space. It also enforces the sparsity constraint, which allows each service to be represented by a small number of latent functional factors. The sparsity constraint is in line with the fact that most real-world services only provide limited functionalities. We conduct extensive experiments on two sets of real-world service data to demonstrate the effectiveness of the proposed service clustering approach.

Massive amounts of data are being generated on social media sites, such as Twitter and Facebook. These data can be used to better understand people (e.g., personality traits, perceptions, and preferences) and predict their behavior. As a result, a deeper understanding of users and their behavior can benefit a wide range of intelligent applications, such as advertising, social recommender systems, and personalized knowledge management. These applications will also benefit individual users themselves and optimize their experience across a wide variety of domains, such as retail, healthcare, and education. Since mining and understanding user behavior from social media often requires interdisciplinary effort, including machine learning, text mining, human-computer interaction, and social science, our workshop aims to bring together researchers and practitioners from multiple fields to discuss the creation of deeper models of individual users by mining the content that they publish and the social networking behavior that they exhibit.

Web service composition helps users integrate services to create new large-granularity and value-added composite services. Most recent studies have focused on automatic AI-Planning-based static or dynamic composition at functional- or process-level. However in industry, most business applications are still composed manually or semi-automatically with abundant domain expertise. Consequently, to build a good and reliable composite service is really a time-consuming and professional task. Inspired by the Instant Search of Google, we propose an Instant recommendation approach to provide optimal suggestions while a composition process incrementally proceeds. In our model, we fully utilize the execution log of composite services, and intend to identify appropriate services which have been proved to be more reliable and robust, therefore those services have higher probability to fulfill users' demands. To find the top-k possible composite services in real-time, we adopt the A* search algorithm with various pruning heuristics to dynamically expand the search space efficiently. Experiments on a real-world dataset with 15,959 real Web services crawled from the Internet demonstrate the effectiveness and efficiency of the proposed approach.

Online news reading has become the major method to know about the world as web provide more information than other media like TV and radio. However, traditional online news reading interface is inconvenient for many types of people, especially for those who are disabled or taking a bus. This paper presents a mobile application iNewsBox enabling users to listen to news collected from the Internet. In order to simplify necessary interactions of getting valuable news, we also propose a framework for using implicit feedback to recommend news in this paper. Experiment shows our algorithms in iNewsBox are effective.

Quality-of-service-based (QoS) service selection is an important issue of service-oriented computing. A common premise of previous research is that the QoS values of services to target users are supposed to be all known. However, many of QoS values are unknown in reality. This paper presents a neighborhood-based collaborative filtering approach to predict such unknown values for QoS-based selection. Compared with existing methods, the proposed method has three new features: 1) the adjusted-cosine-based similarity calculation to remove the impact of different QoS scale; 2) a data smoothing process to improve prediction accuracy; and 3) a similarity fusion approach to handle the data sparsity problem. In addition, a two-phase neighbor selection strategy is proposed to improve its scalability. An extensive performance study based on a public data set demonstrates its effectiveness.

In this paper, we propose a novel framework WSCRec, in which historical information (i.e., execution logs) are utilized to facilitate the process of Web service composition. According to the execution logs of composite services, appropriate services which have been proved to be more reliable and robust and have higher probability to fulfill users' demands are located.

In this paper, we propose a time-sensitive probability skyline (TPS) approach to recommend services with uncertainty. We project services to n-dimensional data space and recommend services in TPS. Experimental evaluation on real data shows the great performance of TPS in service recommendation by comparing the experiment result with results of other approaches.

Skyline attracts more and more attention from academic circle and industrial circle because of its application in multi-criterion decision support, preference answering and data analysis. However, it seems unnecessary to recommend all services in skyline while the number of skyline points is large. The number of services in skyline is always large for the reason that comparability decreases with the increase of data dimensionality. Users always want to get only 2 or 3 recommendations instead of all services in skyline. Motivated by this, we propose to compute the representative skyline which contains some points that best describe the contour of the full skyline. In this paper, we propose a new definition which we call “similarity-based representative skyline”. We provide an algorithm SBRSA, which is based on a traversal approach to compute the value of similarity. In particular, we propose an algorithm to maintain the result of SBRSA in dynamic data environment. An extensive performance study using real and synthetic service data is reported to verify its great performance in representation and computing cost.