Graph neural networks (GNNs) have achieved significant success in graph representation learning and have been applied to various domains. However, many real-world graphs contain sensitive personal information, such as user profiles in social networks, raising serious privacy concerns when graph learning is performed using GNNs. To address this issue, locally private graph learning protocols have gained considerable attention. These protocols leverage the privacy advantages of local differential privacy (LDP) and the effectiveness of GNN's message-passing in calibrating noisy data, offering strict privacy guarantees for users' local data while maintaining high utility (e.g., node classification accuracy) for graph learning. Despite these advantages, such protocols may be vulnerable to data poisoning attacks, a threat that has not been considered in previous research. Identifying and addressing these threats is crucial for ensuring the robustness and security of privacy-preserving graph learning frameworks. This work introduces the first data poisoning attack targeting locally private graph learning protocols. The attacker injects fake users into the protocol, manipulates these fake users to establish links with genuine users, and sends carefully crafted data to the server, ultimately compromising the utility of private graph learning. The effectiveness of the attack is demonstrated both theoretically and empirically. In addition, several defense strategies have also been explored, but their limited effectiveness highlights the need for more robust defenses.

Hypergraphs are the natural description of higher-order interactions among objects, widely applied in social network analysis, cross-modal retrieval, etc. Hypergraph Neural Networks (HGNNs) have become the dominant solution for learning on hypergraphs. Traditional HGNNs are extended from message passing graph neural networks, following the homophily assumption, and thus struggle with the prevalent heterophilic hypergraphs that call for long-range dependence modeling. Existing solutions enlarge the message flow through the hypergraph bottleneck, mitigating the oversquashing issue and capturing long-range dependence. However, they often accelerate the loss of representation distinguishability in the repeated aggregations, leading to oversmoothing. This dilemma motivates an interesting question: Can we develop a unified mechanism that is agnostic to both homophilic and heterophilic hypergraphs? In this paper, we achieve the best of both worlds through the lens of Riemannian geometry, which provides the potential to adjust the message passing behavior in different regions. The key insight lies in the connection between oversquashing and hypergraph bottleneck within the framework of Riemannian manifold heat flow. Building on this, we propose the novel idea of locally adapting the bottlenecks of different subhypergraphs. The core innovation of the proposed mechanism is the design of an adaptive local (heat) exchanger. Specifically, it captures the rich long-range dependencies via the Robin condition, and preserves the representation distinguishability via source terms, thereby enabling heterophily-agnostic message passing with theoretical guarantees. Based on this theoretical foundation, we present a novel Heat-Exchanger with Adaptive Locality for Hypergraph Neural Network (HealHGNN), designed as a node-hyperedge bidirectional systems with linear complexity in the number of nodes and hyperedges. Extensive experiments on both homophilic and heterophilic cases show that HealHGNN achieves the state-of-the-art performance.

Retrieval-Augmented Generation (RAG) improves the performance of Large Language Models (LLMs) by retrieving and integrating relevant information from external knowledge bases, which helps generate more accurate responses. However, RAG is vulnerable to retrieval poisoning attacks , where attackers can induce LLM to produce inaccurate responses by injecting malicious documents into the retrieval process. In this article, we propose ShieldRAG , a novel defense framework designed to counteract retrieval poisoning attacks by reshaping the retrieval embedding space. ShieldRAG leverages a dual-strategy effect realized via a majority-consensus mechanism: ① Push : Implicitly forces the embedding of a user query away from malicious documents by filtering out their minority signals, reducing their influence. ② Pull : Aligns the embedding of a user query closer to that of benign documents, reinforcing accurate retrieval. These strategies work synergistically to preserve retrieval integrity and enhance the quality of LLM-generated responses. Specifically, ShieldRAG operates through three key steps: Sliding Retrieval Explanation Generation , Keyword Aggregation , and Query Targeting Optimization . These three steps collectively ensure the effective integration of information from benign sources while filtering out malicious interference, thereby significantly enhancing the robustness of RAG systems against retrieval poisoning attacks. We evaluate ShieldRAG on four open-domain Question Answering (QA) datasets: Natural Questions, MS-MARCO, HotpotQA, and 2WikiMultiHopQA, using seven representative LLMs. Extensive experiments demonstrate that ShieldRAG significantly improves response accuracy while mitigating adversarial effects, showcasing strong generalization across multiple datasets and LLM architectures.

Instruction tuning has become a cornerstone for unlocking the full potential of large language models. Among the key factors, complex and diverse instructions play a crucial role in aligning these models with a wide range of downstream tasks. However, current methodologies for constructing large-scale instruction datasets tend to favor powerful models, such as GPT-4, based on the empirical assumption that larger models inherently possess superior capabilities. In this study, we challenge this prevailing assumption and delve into the untapped potential of smaller language models (SLMs) in the context of instruction evolution. Through extensive experiments across three distinct scenarios of instruction evolution, we find that SLMs can generate more effective instructions compared to their larger counterparts. Further analysis reveals that SLMs exhibit a broader output space during instruction evolution, leading to the creation of more complex and diverse instructional variants. Additionally, we observe that existing evaluation metrics fall short in capturing the nuanced impact of instructions. To address this limitation, we propose Instruction Complex-Aware IFD (IC-IFD), an enhanced framework that incorporates instruction complexity into the original IFD score. This approach enables a more accurate assessment of the effectiveness of instruction data, paving the way for more refined instruction tuning strategies.

Extreme Gradient Boosting (XGBoost) demonstrates excellent performance in practice and is widely used in both industry and academic research. This extensive application has led to a growing interest in employing multi-party data to develop more robust XGBoost models. In response to increasing concerns about privacy leakage, secure vertical federated XGBoost is proposed. It employs secure multi-party computation techniques, such as secret sharing (SS), to allow multiple parties holding vertically partitioned data, i.e., disjoint features on the same samples, to collaborate in constructing an XGBoost model. However, the running efficiency is the primary obstacle to the practical application of existing protocols, especially in multi-party settings. The reason is that these protocols not only require the execution of data-oblivious computations to protect intermediate results, leading to high computational complexity, but also involve a large number of SS-based non-linear operations with high overheads, e.g., division operations in gain score calculation and comparison operations in best split selection. To this end, we present a secure and efficient multi-party protocol for vertical federated XGBoost, called SecureXGB, which can perform the collaborative training of an XGBoost model in an SS-friendly manner. In SecureXGB, we first propose a parallelizable multi-party permutation method, which can secretly and efficiently permute all samples before model training to reduce the reliance on data-oblivious computations. Then, we design a linear gain score that can be evaluated without involving division operations and has equivalent utility to the original gain score. Finally, we develop a synchronous best split selection method to secretly identify the best split with the maximum gain score using a minimal number of comparison operations. Experimental results demonstrate that SecureXGB can achieve better training efficiency than state-of-the-art protocols without the loss of model accuracy.

Multi-passage machine reading comprehension (MRC) aims to answer a question by multiple passages. Existing multi-passage MRC approaches have shown that employing passages with and without golden answers (i.e., labeled and unlabeled passages) for model training can improve prediction accuracy. However, when using the unlabeled passages, they either incur the wrong labeling problem or treat the labeled and unlabeled passages equally. In addition, they ignore the original passage information to verify the correctness of the answer. In this paper, we present MLDV-MRC, a novel approach for multi-passage MRC viaMulti-taskLearning andDualVerification. MLDV-MRC adopts the extract-then-select framework, where an extractor is first used to predict answer candidates, then a selector is used to choose the final answer. For the extractor, we adopt multi-task learning with generative adversarial training to train it by using both labeled and unlabeled passages. To train the extractor by backpropagation, we propose a hybrid method which combines boundary-based and content-based extracting methods to produce the answer candidate set and its representation. For the selector, we propose to leverage both the information from answer candidates and original passages to verify the final answer. In particular, we propose a global-local memory-augmented neural network to build the representations of original passages, which fuses the passage-level information and word-level information. The experimental results on three open-domain QA datasets confirm the effectiveness of our approach.

Social network alignment, identifying social accounts of the same individual across different social networks, shows fundamental importance in a wide spectrum of applications, such as link prediction and information diffusion. Individuals more often than not join in multiple social networks, and it is in fact much too expensive or even impossible to acquiring supervision for guiding the alignment. To the best of our knowledge, few method in the literature can align multiple social networks without supervision. In this article, we propose to study the problem of unsupervised multiple social network alignment. To address this problem, we propose a novel unsupervised model of joint Matrix factorization with a diagonal Cone under orthogonal Constraint, referred to as MC 2 . Its core idea is to embed and align multiple social networks in the common subspace via an unsupervised approach. Specifically, in MC 2 model, we first design a matrix optimization to infer the common subspace from different social networks. To address the nonconvex optimization, we then design an efficient alternating algorithm by leveraging its inherent functional property. Through extensive experiments on real-world datasets, we demonstrate that the proposed MC 2 model significantly outperforms the state-of-the-art methods.

With the deep penetration of the Internet and mobile devices, preference rankings are being collected on a massive scale by diverse data collectors for various business demands. However, users’ preference rankings in many applications are highly sensitive. Without proper privacy protection mechanisms, it either puts individual privacy in jeopardy or hampers business opportunities due to users’ unwillingness to share their true rankings. In this paper, we initiate the study of collecting preference rankings under local differential privacy. The key technical challenge comes from the fact that the number of possible rankings could be large in practical settings, leading to excessive injected noise. To solve this problem, we present a novel approach SAFARI, whose main idea is to collect a set of distributions over small domains which are carefully chosen based on the riffle independent (RI) model to approximate the overall distribution of users’ rankings, and then generate a synthetic ranking dataset from the obtained distributions. By working on small domains instead of a large domain, SAFARI can significantly reduce the magnitude of added noise. In SAFARI, we design two transformation rules, namely Rule I and Rule II, to instruct users to transform their data to provide the information about the distributions of the small domains. In particular, we propose a method called LADE to precisely estimate the required distributions used for the structure learning of RI model. We also propose a new LDP method called SAFA for frequency estimation over multiple attributes that have small domains. We formally prove that SAFARI guarantees$\varepsilon$-local differential privacy. Extensive experiments on real datasets confirm the effectiveness of SAFARI.

Social network alignment, aligning different social networks on their common users, is receiving dramatic attentions from both academic and industry. All existing studies consider the social network to be static and neglect its inherent dynamics. In fact, the dynamics of social networks contain the discriminative pattern of an individual, which can be leveraged to facilitate social network alignment. Hence, we for the first time propose to study the problem of aligning dynamic social networks. Towards this end, we propose a novel Dynamic Graph autoencoder based dynamic social network Alignment approach, referred to as DGA, unfolding the fruitful dynamics of social networks for user alignment. However, it faces challenges in both modeling and optimization: (1) To model the intra-network dynamics, we design a novel dynamic graph autoencoder to learn user embeddings with complex network dynamics. (2) To model the inter-network alignment, we design a unified optimization framework over proposed dynamic graph autoencoders, constructing a common subspace for user alignment across different networks. (3) To address this optimization problem, we design an effective alternating algorithm with solid theoretical guarantees. We conduct extensive experiments on real-world datasets and show that the proposed approach substantially outperforms the state-of-the-art methods.

Given a set of local sequential datasets held by multiple parties, we study the problem of publishing a synthetic dataset that preserves approximate sequentiality information of the integrated dataset while satisfying differential privacy for each local dataset. The existing solutions for publishing differentially private sequential data in the centralized setting mostly adopt tree-based approaches. Such approaches rely on different tree structures that encode sequential data's statistical information. The construction of a tree structure is normally done by recursively splitting nodes whose noisyscores(e.g., entropy or count) are larger than a given threshold. However, extending similar ideas to the multi-party setting is challenging. First, the comparison between noisy scores and a given threshold needs to be done in a distributed manner without letting the parties know the noisy scores, while satisfying differential privacy for each local dataset. Second, in the multi-party setting the large number of node splitting decisions incurs prohibitive computation costs. In addressing the above challenges, we presentDPST, a distributed prediction suffix tree construction solution. In DPST, we first introduce a novel node splitting decision method that calculates the comparison result under encryption with substantially improved efficiency. Then we present a novel batch-based tree construction approach to reduce computation costs. In order to achieve high parallel performance without incurring any extra communication cost, we introduce theconjunctionandslidemethods to ensure that each batch contains a stable number of carefully arrangeddecision tasks. To further reduce communication and computation costs, we propose a prefix-based pre-pruning method to reduce the number of nodes that need to be judged whether to split by an interactive protocol. Extensive experiments on real datasets demonstrate that our DPST solution offers desirable data utility with low computation and communication costs.

Network alignment aims to discover nodes in different networks belonging to the same identity. In recent years, the network alignment problem has aroused significant attentions in both industry and academia. However, the continuous exploding of network data brings two challenges in solving the network alignment problem, i.e., large network scale and scarce labeled data. To bridge this gap, in this paper we propose a novel approach termed asMulti-granularItyNetwork alIgnment based on coNtrastive learninG(MINING). Specifically, in MINING, we first design multi-granularity alignment framework to solve the issue of large network scale. Then, we design intra- and inter-network contrastive learning to solve the issue of scarce labeled data. Moreover, we provide theoretical proofs to demonstrate the effectiveness of MINING. Finally, we conduct extensive experiments on the benchmark datasets of Facebook-Twitter, AMiner-LinkedIn and DBpedia$_{\text{ZH}}$-DBpedia$_{\text{EN}}$, and results show that MINING can averagely achieve 15.93% higher$\operatorname{Hits@}k$and 14.82% higher$\operatorname{MRR@}k$compared with the state-of-the-art methods.

Recently, aligning users among different social networks has received significant attention. However, most of the existing studies do not consider users' behavior information during the aligning procedure and thus still suffer from poor learning performance. In fact, we observe that social network alignment and user behavior analysis can benefit from each other. Motivated by such an observation, we propose to jointly study the social network alignment and user behavior analysis problem in this paper. We design a novel framework named BANANA-RGB. In this framework, to capture users' multi-scale behavior information in each social network, we train a variant of the hierarchical periodic memory network with personalized memorization. To leverage behavior analysis for social network alignment, we design a tensor fusion network-based alignment component to improve the performance. To further leverage social network alignment for behavior analysis, we design a gating-based cross-network behavior fusion component to integrate users' behavior information in different social networks based on the alignment result. We iteratively train the above two components to make the two tasks benefit from each other. Extensive experiments on real-world datasets demonstrate that our proposed approach outperforms the state-of-the-art methods.

In this paper, we tackle the problem of collecting individual trajectories under local differential privacy. The key challenge is how to achieve high utility of the collected trajectories while satisfying the strong privacy guarantee. To overcome this challenge, we present a novel approach, which is referred to as PrivTC. In PrivTC, we first design a locally differentially private grid construction method to instruct the aggregator to lay an appropriate grid on the given geospatial domain. Then we propose a locally differentially private spectral learning method to help the aggregator learn the Hidden Markov Model (HMM) from users' trajectories discretized by the constructed grid. Finally, the aggregator generates a synthetic trajectory dataset as a surrogate for the original one from the learned HMM. Extensive experiments on real datasets confirm the effectiveness of PrivTC.

Event argument extraction (EAE) is an important information extraction task, which aims to identify the arguments of an event described in a given text and classify the roles played by them. A key characteristic in realistic EAE data is that the instance numbers of different roles follow an obvious long-tail distribution. However, the training and evaluation paradigms of existing EAE models either prone to neglect the performance on "tail roles'', or change the role instance distribution for model training to an unrealistic uniform distribution. Though some generic methods can alleviate the class imbalance in long-tail datasets, they usually sacrifice the performance of "head classes'' as a trade-off. To address the above issues, we propose to train our model on realistic long-tail EAE datasets, and evaluate the average performance over all roles. Inspired by the Mixture of Experts (MOE), we propose a Routing-Balanced Dual Expert Framework (RBDEF), which divides all roles into "head" and "tail" two scopes and assigns the classifications of head and tail roles to two separate experts. In inference, each encoded instance will be allocated to one of the two experts by a routing mechanism. To reduce routing errors caused by the imbalance of role instances, we design a Balanced Routing Mechanism (BRM), which transfers several head roles to the tail expert to balance the load of routing, and employs a tri-filter routing strategy to reduce the misallocation of the tail expert's instances. To enable an effective learning of tail roles with scarce instances, we devise Target-Specialized Meta Learning (TSML) to train the tail expert. Different from other meta learning algorithms that only search a generic parameter initialization equally applying to infinite tasks, TSML can adaptively adjust its search path to obtain a specialized initialization for the tail expert, thereby expanding the benefits to the learning of tail roles. In experiments, RBDEF significantly outperforms the state-of-the-art EAE models and advanced methods for long-tail data.

In social networks, one of the most significant challenges is how to estimate the tweet popularity. Prior studies focus on leveraging different aspects of just a single tweet, while ignoring the impact of historical tweets. In this article, we propose to leverage such historical information and rethink the problem of tweet popularity estimation. From historical information, there are two important factors that can be extracted: (1) user reputation feature, which can represent coarse-grained level of tweet popularity and (2) tweet related features, which can represent fine-grained level of tweet popularity. To incorporate these two factors from historical information, we design a novel deep neural architecture, a Bi-layered LSTM for tweet popUlarity Estimation, called DeepBlue. Specifically, we first propose a user-reputation aware mechanism to combine coarse-grained and fine-grained level estimation into a united LSTM model. We also design a content attention mechanism to consider different impacts of historical tweets in terms of content similarity. We then propose a time aware mechanism to address the time interval irregularity issue. Finally, we apply the Poisson regression model to obtain the overall loss for tweet popularity estimation. Extensive experiments demonstrate the superiority of our proposed approach to other state-of-the-arts in terms of MAE and SRC.

Relation extraction (RE), an important information extraction task, faced the great challenge brought by limited annotation data. To this end, distant supervision was proposed to automatically label RE data, and thus largely increased the number of annotated instances. Unfortunately, lots of noise relation annotations brought by automatic labeling become a new obstacle. Some recent studies have shown that the teacher-student framework of knowledge distillation can alleviate the interference of noise relation annotations via label softening. Nevertheless, we find that they still suffer from two problems: propagation of inaccurate dark knowledge and constraint of a unified distillation temperature . In this article, we propose a simple and effective Multi-instance Dynamic Temperature Distillation (MiDTD) framework, which is model-agnostic and mainly involves two modules: multi-instance target fusion (MiTF) and dynamic temperature regulation (DTR). MiTF combines the teacher’s predictions for multiple sentences with the same entity pair to amend the inaccurate dark knowledge in each student’s target. DTR allocates alterable distillation temperatures to different training instances to enable the softness of most student’s targets to be regulated to a moderate range. In experiments, we construct three concrete MiDTD instantiations with BERT, PCNN, and BiLSTM-based RE models, and the distilled students significantly outperform their teachers and the state-of-the-art (SOTA) methods.

User profiling is playing an increasingly important role in real-world applications. Previous works have shown that integrating user information from multiple social networks helps to significantly improve the performance of user profiling. However, these studies either ignore the different contributions of various features in different profiling tasks or need to train one model for each task. What’s more, the assumption of the strong relatedness between user profiling tasks limits their application. These phenomena make inferring comprehensive user attributes still an open problem. In this paper, we propose a novel method, called Hierarchical Attention-based Model with sparse-sharing-based muLti-task sElf-Training algorithm (HAMLET), for comprehensive user profiling. More specifically, we first employ a hierarchical attention-based network as our base network to represent users. It assigns various features from different social networks with different weights for different users during the fusing procedure. Then, we propose a multi-task self-training algorithm that takes advantage of both task correlations and self-training to obtain better performance. We conduct extensive experiments on two real-world datasets and verify the superiority of HAMLET for user profiling.

Given a set of local sequential datasets held by multiple parties, we study the problem of publishing a synthetic dataset that preserves approximate sequentiality information of the integrated dataset while satisfying differential privacy for each local dataset. The existing solutions for publishing differentially private sequential data in the centralized setting mostly adopt tree-based approaches. Such approaches rely on different tree structures that encode sequential data's statistical information. The construction of a tree structure is normally done by recursively splitting nodes whose noisy scores (e.g., entropy or count) are larger than a given threshold. However, extending similar ideas to the multi-party setting is challenging. First, the comparison between noisy scores and a given threshold needs to be done in a distributed manner without letting the parties know the noisy scores, while satisfying differential privacy for each local dataset. Second, in the multi-party setting the large number of node splitting decisions incurs prohibitive computation costs. In addressing the above challenges, we present DPST, a distributed prediction suffix tree construction solution. In DPST, we first introduce a novel node splitting decision method that calculates the comparison result under encryption with substantially improved efficiency. Then we present a novel batch-based tree construction approach to reduce the computation costs. In order to achieve high parallel performance without incurring any extra communication cost, we introduce the conjunction and slide methods to ensure that each batch contains a stable number of carefully arranged decision tasks. Extensive experiments on real datasets demonstrate that our DPST solution offers desirable data utility with low computation and communication costs.

Abstract Local community detection aims to find the communities that a given seed node belongs to. Most existing works on this problem are based on a very strict assumption that the seed node only belongs to a single community, but in real-world networks, nodes are likely to belong to multiple communities. In this paper, we first introduce a novel algorithm, HqsMLCD, that can detect multiple communities for a given seed node over static networks. HqsMLCD first finds the high-quality seeds which can detect better communities than the given seed node with the help of network representation, then expands the high-quality seeds one-by-one to get multiple communities, probably overlapping. Since dynamic networks also act an important role in practice, we extend the static HqsMLCD to handle dynamic networks and introduce HqsDMLCD. HqsDMLCD mainly integrates dynamic network embedding and dynamic local community detection into the static one. Experimental results on real-world networks demonstrate that our new method HqsMLCD outperforms the state-of-the-art multiple local community detection algorithms. And our dynamic method HqsDMLCD gets comparable results with the static method on real-world networks.

Recently, reconciling social networks, identifying the accounts belonging to the same individual across social networks, receives significant attention from both academic and industry. Most of the existing studies have limitations in the following three aspects: multiplicity, comprehensiveness and robustness. To address these limitations, we rethink this problem and, for the first time, robustly and comprehensively reconcile multiple social networks. In this paper, we propose two frameworks, MASTER and MASTER+, i.e., across Multiple social networks, integrate Attribute and STructure Embedding for Reconciliation. In MASTER, we first design a novel Constrained Dual Embedding model, simultaneously embedding and reconciling multiple social networks, to formulate this problem into a unified optimization. To address this optimization, we then design an effective NS-Alternating algorithm and prove it converges to KKT points. To further speed up MASTER, we propose a scalable framework, namely MASTER+. The core idea is to group accounts into clusters and then perform MASTER in each cluster in parallel. Specifically, we design an efficient Augmented Pre-Embedding model and Balance-aware Fuzzy Clustering algorithm for the high efficiency and the high accuracy. Extensive experiments demonstrate that both MASTER and MASTER+ outperform the state-of-the-art approaches. Moreover, MASTER+ inherits the effectiveness of MASTER and enjoys higher efficiency.

User profiling has very important applications for many downstream tasks, such as recommender system, behavior prediction and market strategy. Most existing methods only focus on modeling user profiles of one social network with plenty of data. However, user profiles are difficult to acquire, especially when the data is scarce. Modeling user profiles under such conditions often leads to poor performance. Fortunately, we observed that not only user attributes but also user relationships are useful for user profiling and benefit the results. Meanwhile, similar users have similar behavior in different social networks. Finding user dependencies between social networks will help to infer user profiles. Motivated by such observations, in this paper, we for the first time propose to study the user profiling problem from the transfer learning perspective. We design an efficient User Profile transferring acrOss Networks (UPON) framework, which transfers knowledge of user relationship from one social network with plenty of data to facilitate the user profiling on the other social network with scarce data. In UPON, we first design a novel graph convolutional networks based characteristic-aware domain attention model (GCN-CDAM) to find user dependencies within and between domains (referring to social networks). We then design a dual-domain weighted adversarial learning method to solve the domain shift problem existing in the transferring procedure. Experimental results on Twitter-Foursquare dataset demonstrate that UPON outperforms the state-of-the-art models.

Social network alignment shows fundamental importance in a wide spectrum of applications. To the best of our knowledge, existing studies mainly focus on network alignment at the individual user level, requiring abundant common information between shared individual users. For the networks that cannot meet such requirements, social community structures actually provide complementary and critical information at a slightly coarse-grained level, alignment of which will provide additional information for user alignment. In turn, user alignment also reveals more clues for community alignment. Hence, in this paper, we introduce the problem of joint social network alignment, which aims to align users and communities across social networks simultaneously. Key challenges lie in that (1) how to learn the representations of both users and communities, and (2) how to make user alignment and community alignment benefit from each other. To address these challenges, we first elaborate on the characteristics of real-world networks with the notion of δ-hyperbolicity, and show the superiority of hyperbolic space for representing social networks. Then, we present a novel hyperbolic embedding approach for the joint social network alignment, referred to as Perfect, in a unified optimization. Extensive experiments on real-world datasets show the superiority of Perfect in both user alignment and community alignment.

In this paper, we tackle the problem of answering multi-dimensional range queries under local differential privacy. There are three key technical challenges: capturing the correlations among attributes, avoiding the curse of dimensionality, and dealing with the large domains of attributes. None of the existing approaches satisfactorily deals with all three challenges. Overcoming these three challenges, we first propose an approach called Two-Dimensional Grids (TDG). Its main idea is to carefully use binning to partition the two-dimensional (2-D) domains of all attribute pairs into 2-D grids that can answer all 2-D range queries and then estimate the answer of a higher dimensional range query from the answers of the associated 2-D range queries. However, in order to reduce errors due to noises, coarse granularities are needed for each attribute in 2-D grids, losing fine-grained distribution information for individual attributes. To correct this deficiency, we further propose Hybrid-Dimensional Grids (HDG), which also introduces 1-D grids to capture finer-grained information on distribution of each individual attribute and combines information from 1-D and 2-D grids to answer range queries. To make HDG consistently effective, we provide a guideline for properly choosing granularities of grids based on an analysis of how different sources of errors are impacted by these choices. Extensive experiments conducted on real and synthetic datasets show that HDG can give a significant improvement over the existing approaches.

In this paper, we study the problem of publishing high-dimensional data in a distributed multi-party environment under differential privacy. In particular, with the assistance of a semi-trusted curator, the parties (i.e., local data owners) collectively generate a synthetic integrated dataset while satisfying ε-differential privacy. To solve this problem, we present a differentially private sequential update of Bayesian network (DP-SUBN) approach. In DP-SUBN, the parties and the curator collaboratively identify the Bayesian network N that best fits the integrated dataset in a sequential manner, from which a synthetic dataset can then be generated. The fundamental advantage of adopting the sequential update manner is that the parties can treat the intermediate results provided by previous parties as their prior knowledge to direct how to learn N. The core of DP-SUBN is the construction of the search frontier, which can be seen as a priori knowledge to guide the parties to update N. By exploiting the correlations of attribute pairs, we propose exact and heuristic methods to construct the search frontier. In particular, to privately quantify the correlations of attribute pairs without introducing too much noise, we first put forward a non-overlapping covering design (NOCD) method, and then devise a dynamic programming method for determining the optimal parameters used in NOCD. Through privacy analysis, we show that DP-SUBN satisfies ε-differential privacy. Extensive experiments on real datasets demonstrate that DP-SUBN offers desirable data utility with low communication cost.

Social network alignment, identifying social accounts of the same individual across different social networks, shows fundamental importance across a wide spectrum of applications. Individuals more often than not join in multiple social networks and it is in fact intractable or even impossible to acquiring supervision for guiding the alignment. However, to the best of our knowledge, none of existing methods can align multiple social networks without supervision. In this paper, we propose to study the problem of unsupervised multiple social network alignment. To address this problem, we propose a novel unsupervised model of Matrix factorization with diagonal Cone under orthogonal Constraint, referred to as MC2. Its core idea is to embed and align multiple social networks in the common subspace via an unsupervised approach. Specifically, in MC2model, we first design a matrix optimization to infer the common subspace from different social networks. To address the nonconvex optimization, we then design an efficient alternating algorithm by leveraging its inherent functional property. Through extensive experiments on real-world datasets, we demonstrate that the proposed MC2model significantly outperforms the state-of-the-art methods.

Social network alignment, aligning different social networks on their common users, is receiving dramatic attention from both academic and industry. All existing studies consider the social network to be static and neglect its inherent dynamics. In fact, the dynamics of social networks contain the discriminative pattern of an individual, which can be leveraged to facilitate social network alignment. Hence, we for the first time propose to study the problem of aligning dynamic social networks. Towards this end, we propose a novel Dynamic social Network Alignment (DNA) framework, a unified optimization approach over deep neural architectures, to unfold the fruitful dynamics to perform alignment. However, it faces tremendous challenges in both modeling and optimization: (1) To model the intra-network dynamics, we explore the local dynamics of the latent pattern in friending evolvement and the global consistency of the representation similarity with neighbors. We design a novel deep neural architecture to obtain the dual embedding capturing local dynamics and global consistency for each user. (2) To model the inter-network alignment, we exploit the underlying identity of an individual from the dual embedding in each dynamic social network. We design a unified optimization approach interplaying proposed deep neural architectures to construct a common subspace of identity embeddings. (3) To address this optimization problem, we design an effective alternating algorithm with solid theoretical guarantees. We conduct extensive experiments on real-world datasets and show that the proposed DNA framework substantially outperforms the state-of-the-art methods.

In social networks, one of the most significant challenges is how to estimate the tweet popularity. Prior studies about this problem focus on leveraging different aspects of just a single tweet, while ignoring the impact of historical tweets. In this paper, we propose to leverage such historical information and rethink the problem of tweet popularity estimation. From historical information, there are two important factors that can be extracted: (1) user reputation feature, which can represent coarse-grained level of tweet popularity; (2) tweet related features, which can represent fine-grained level of tweet popularity. To incorporate two factors from historical information, we design a novel deep neural architecture, a Bi-layered LSTM for tweet popularity Estimation, called DeepBlue. Specifically, we first propose a user-reputation aware mechanism to combine coarse-grained and fine-grained level estimation into a unified LSTM model. We then propose a time aware mechanism to address the time interval irregularity issue in standard LSTM. Finally, we apply the Poisson regression model to obtain the overall loss for tweet popularity estimation. Extensive experiments demonstrate the superiority of our proposed approach to other state-of-the-arts in terms of MAE and SRC.

In this paper, we initiate the study of collecting preference rankings under local differential privacy. The key technical challenge comes from the fact that the number of possible rankings increases factorially in the number of items to rank. In practical settings, this number could be large, leading to excessive injected noise. To solve this problem, we present a novel approach called SAFARI. The general idea is to collect a set of distributions over small domains which are carefully chosen based on the riffle independent model to approximate the overall distribution of users' rankings, and then generate a synthetic ranking dataset from the obtained distributions. By working on small domains instead of a large domain, SAFARI can significantly reduce the magnitude of added noise. Extensive experiments on real datasets confirm the effectiveness of SAFARI.

Supervised topic classifiers for Twitter and other media sources are important in a variety of long-term topic tracking tasks. Unfortunately, over long periods of time, features that are predictive during the training period may prove ephemeral and fail to generalize to prediction at future times. For example, if we trained a classifier to identify tweets concerning the topic of “Celebrity Death”, individual celebrity names and terms associated with these celebrities such as “Nelson Mandela” or “South Africa” would prove to be temporally unstable since they would not generalize over long periods of time; in contrast, terms like “RIP” (rest in peace) would prove to be temporally stable predictors of this topic over long periods of time. In this paper, we aim to design supervised learning methods for Twitter topic classifiers that are capable of automatically downweighting temporally unstable features to improve future generalization. To do this, we first begin with an oracular approach that chooses temporally stable features based on knowledge of both train and test data labels. We then search for feature metrics evaluated on only the training data that are capable of recovering the temporally stable features identified by our oracular definition. We next embed the top-performing metric as a temporal stability regularizer in logistic regression with the important property that the overall training objective retains convexity, hence enabling a globally optimal solution. Finally, we train our topic classifiers on 6 Twitter topics over roughly one year of data and evaluate on the following year of data, showing that logistic regression with our temporal stability regularizer generally outperforms logistic regression without such regularization across the full precision-recall continuum. Overall, these results establish a novel regularizer for training long-term temporally stable topic classifiers for Twitter and beyond.

It has been shown that stock price movements are influenced by news. To predict stock movements with news, many existing works rely only on the news title since the news content may contain irrelevancies which seriously degrade the prediction accuracy. However, we observe that there is still useful information in the content which is not reflected in the title, and simply ignoring the content will result in poor performance. In this paper, taking advantage of neural representation learning, we propose a hierarchical complementary attention network (HCAN) to capture valuable complementary information in news title and content for stock movement prediction. In HCAN, we adopt a two-level attention mechanism to quantify the importances of the words and sentences in a given news. Moreover, we design a novel measurement for calculating the attention weights to avoid capturing redundant information in the news title and content. Experimental results on news datasets show that our proposed model outperforms the state-of-the-art techniques.

Community Question Answering (CQA) has increasingly become an important service for people asking questions and providing answers online, which enables people to help each other by sharing knowledge. Recently, with accumulation of users and contents, much concern has arisen over the efficiency and answer quality of CQA services. To address this problem, question routing has been proposed which aims at routing new questions to suitable answerers, who have both high possibility and high ability to answer the questions. In this paper, we formulate question routing as a multi-objective ranking problem, and present a multi-objective learning-to-rank approach for question routing (MLQR), which can simultaneously optimize the answering possibility and answer quality of routed users. In MLQR, realizing that questions are relatively short and usually attached with tags, we first propose a tagword topic model (TTM) to derive topical representations of questions. Based on TTM, we then develop features for each question-user pair, which are captured at both platform level and thread level. In particular, the platform-level features summarize the information of a user from his/her history posts in the CQA platform, while the thread-level features model the pairwise competitions of a user with others in his/her answered threads. Finally, we extend a state-of-the-art learning-to-rank algorithm for training a multi-objective ranking model. Extensive experimental results on real-world datasets show that our MLQR can outperform state-of-the-art methods in terms of both answering possibility and answer quality.

In this paper, we study the location‐aware targeted influence maximization problem in social networks, which finds a seed set to maximize the influence spread over the targeted users. In particular, we consider those users who have both topic and geographical preferences on promotion products as targeted users. To efficiently solve this problem, one challenge is how to find the targeted users and compute their preferences efficiently for given requests. To address this challenge, we devise a TR‐tree index structure, where each tree node stores users' topic and geographical preferences. By traversing the TR‐tree in depth‐first order, we can efficiently find the targeted users. Another challenge of the problem is to devise algorithms for efficient seeds selection. We solve this challenge from two complementary directions. In one direction, we adopt the maximum influence arborescence (MIA) model to approximate the influence spread, and propose two efficient approximation algorithms with approximation ratio, which prune some candidate seeds with small influences by precomputing users' initial influences offline and estimating the upper bound of their marginal influences online. In the other direction, we propose a fast heuristic algorithm to improve efficiency. Experiments conducted on real‐world data sets demonstrate the effectiveness and efficiency of our proposed algorithms.

Mining frequent subgraphs from a collection of input graphs is an important task for exploratory data analysis on graph data. However, if the input graphs contain sensitive information, releasing discovered frequent subgraphs may pose considerable threats to individual privacy. In this paper, we study the problem of frequent subgraph mining (FSM) under the rigorous differential privacy model. We present a two-phase differentially private FSM algorithm, which is referred to as DFG. In DFG, frequent subgraphs are privately identified in the first phase, and the noisy support of each identified frequent subgraph is calculated in the second phase. In particular, to privately identity frequent subgraphs, we propose a frequent subgraph identification approach, which can improve the accuracy of discovered frequent subgraphs through candidate pruning. Moreover, to compute the noisy support of each identified frequent subgraph, we devise a lattice-based noisy support computation approach, which leverages the inclusion relations between the discovered frequent subgraphs to improve the accuracy of the noisy supports. Through formal privacy analysis, we prove that DFG satisfies ε-differential privacy. Extensive experimental results on real datasets show that DFG can privately find frequent subgraphs while achieving high data utility.

With the development of large-scale knowledge bases, people are building systems which give simple answers to questions based on consolidate facts. In this paper, we focus on simple questions, which ask about only a subject and relation in the knowledge base. Observing that certain parts of a question usually overlap with names of its corresponding subject and relation in the knowledge base, we argue that a question is formed by a mixture of copying and generation. To model that, we propose a sequence-to-sequence (seq2seq) architecture which encodes a candidate subject-relation pair and decodes it into the given question, where the decoding probability is used to select the best candidate. In our decoder, the copying mode points the subject or relation and duplicates its name, while the generating mode summarizes the meaning of the subject-relation pair and produces a word to smooth the question. Realizing that although sometimes a subject or relation is pointed, different names or keywords might be used, we also incorporate a paraphrasing mode to supplement the copying mode using an automatically mined lexicon. Extensive experiments on the largest dataset exhibit our better performance compared with the state-of-the-art methods.

Community Question Answering (CQA) has increasingly become an important service for people asking questions and providing answers online, which enables people to help each other by sharing knowledge. Recently, with accumulation of users and contents, much concern has arisen over the efficiency and answer quality of CQA services. To address this problem, question routing has been proposed which aims at routing new questions to suitable answerers, who have both high possibility and high ability to answer the questions. In this paper, we formulate question routing as a multi-objective ranking problem, and present a multi-objective learning-to-rank approach for question routing (MLQR), which can simultaneously optimize the answering possibility and answer quality of routed users. In MLQR, realizing that questions are relatively short and usually attached with tags, we first propose a tagword topic model (TTM) to derive topical representations of questions. Based on TTM, we then develop features for each question-user pair, which are captured at both platform level and thread level. In particular, the platform-level features summarize the information of a user from his/her history posts in the CQA platform, while the thread-level features model the pairwise competitions of a user with others in his/her answered threads. Finally, we extend a state-of-the-art learning-to-rank algorithm for training a multi-objective ranking model. Extensive experimental results on real-world datasets show that our MLQR can outperform state-of-the-art methods in terms of both answering possibility and answer quality.

In this paper, we study the novel problem of publishing high-dimensional data in a distributed multi-party environment under differential privacy. In particular, with the assistance of a semi-trusted curator, the involved parties (i.e., local data owners) collectively generate a synthetic integrated dataset while satisfying ε-differential privacy for any local dataset. To solve this problem, we present a differentially private sequential update of Bayesian network (DP-SUBN) solution. In DP-SUBN, the parties and the curator collaboratively identify the Bayesian network ℕ that best fits the integrated dataset D in a sequential manner, from which a synthetic dataset can then be generated. The fundamental advantage of adopting the sequential update manner is that the parties can treat the statistical results provided by previous parties as their prior knowledge to direct how to learn ℕ. The core of DP-SUBN is the construction of the search frontier, which can be seen as a priori knowledge to guide the parties to update ℕ. To improve the fitness of ℕ and reduce the communication cost, we introduce a correlation-aware search frontier construction (CSFC) approach, where attribute pairs with strong correlations are used to construct the search frontier. In particular, to privately quantify the correlations of attribute pairs without introducing too much noise, we first propose a non-overlapping covering design (NOCD) method, and then introduce a dynamic programming method to find the optimal parameters used in NOCD to ensure that the injected noise is minimum. Through formal privacy analysis, we show that DP-SUBN satisfies ε-differential privacy for any local dataset. Extensive experiments on a real dataset demonstrate that DP-SUBN offers desirable data utility with low communication cost.

Frequent itemset mining (FIM) is one of the most fundamental problems in data mining. It has practical importance in a wide range of application areas such as decision support, Web usage mining, bioinformatics, etc. Given a database, where each transaction contains a set of items, FIM tries to find itemsets that occur in transactions more frequently than a given threshold. Despite valuable insights the discovery of frequent itemsets can potentially provide, if the data is sensitive (e.g., web browsing history and medical records), releasing the discovered frequent itemsets might pose considerable threats to individual privacy.

Mining frequent subgraphs from a collection of input graphs is an important topic in data mining research. However, if the input graphs contain sensitive information, releasing frequent subgraphs may pose considerable threats to individual's privacy. In this paper, we study the problem of frequent subgraph mining (FGM) under the rigorous differential privacy model. We introduce a novel differentially private FGM algorithm, which is referred to as DFG. In this algorithm, we first privately identify frequent subgraphs from input graphs, and then compute the noisy support of each identified frequent subgraph. In particular, to privately identify frequent subgraphs, we present a frequent subgraph identification approach which can improve the utility of frequent subgraph identifications through candidates pruning. Moreover, to compute the noisy support of each identified frequent subgraph, we devise a lattice-based noisy support derivation approach, where a series of methods has been proposed to improve the accuracy of the noisy supports. Through formal privacy analysis, we prove that our DFG algorithm satisfies ε-differential privacy. Extensive experimental results on real datasets show that the DFG algorithm can privately find frequent subgraphs with high data utility.

In this paper, we study the problem of mining frequent sequences under the rigorous differential privacy model. We explore the possibility of designing a differentially private frequent sequence mining (FSM) algorithm which can achieve both high data utility and a high degree of privacy. We found, in differentially private FSM, the amount of required noise is proportionate to the number of candidate sequences. If we could effectively prune those unpromising candidate sequences, the utility and privacy tradeoff can be significantly improved. To this end, by leveraging a sampling-based candidate pruning technique, we proposePFS$^2$, a novel differentially private FSM algorithm. It is the first algorithm that supports the general gap-constrained FSM in the context of differential privacy. The gap constraints in FSM can be used to limit the mining results to a controlled set of frequent sequences. In ourPFS$^2$algorithm, the core is to utilize sample databases to prune the candidate sequences generated based on the downward closure property. In particular, we use the noisy local support of candidate sequences in the sample databases to estimate which candidate sequences are potentially frequent. To improve the accuracy of such private estimations, a gap-aware sequence shrinking method is proposed to enforce the length constraint on the sample databases. Moreover, to calibrate the amount of noise required by differential privacy, a gap-aware sensitivity computation method is proposed to obtain the sensitivity of the local support computations with different gap constraints. Furthermore, to decrease the probability of misestimating frequent sequences as infrequent, a threshold relaxation method is proposed to relax the user-specified threshold for the sample databases. Through formal privacy analysis, we show that ourPFS$^2$algorithm is$\epsilon$-differentially private. Extensive experiments on real datasets illustrate that ourPFS$^2$algorithm can privately find frequent sequences with high accuracy.

In this paper, we study the problem of mining frequent sequences under the rigorous differential privacy model. We explore the possibility of designing a differentially private frequent sequence mining (FSM) algorithm which can achieve both high data utility and a high degree of privacy. We found, in differentially private FSM, the amount of required noise is proportionate to the number of candidate sequences. If we could effectively reduce the number of unpromising candidate sequences, the utility and privacy tradeoff can be significantly improved. To this end, by leveraging a sampling-based candidate pruning technique, we propose a novel differentially private FSM algorithm, which is referred to as PFS2. The core of our algorithm is to utilize sample databases to further prune the candidate sequences generated based on the downward closure property. In particular, we use the noisy local support of candidate sequences in the sample databases to estimate which sequences are potentially frequent. To improve the accuracy of such private estimations, a sequence shrinking method is proposed to enforce the length constraint on the sample databases. Moreover, to decrease the probability of misestimating frequent sequences as infrequent, a threshold relaxation method is proposed to relax the user-specified threshold for the sample databases. Through formal privacy analysis, we show that our PFS2algorithm is ϵ-differentially private. Extensive experiments on real datasets illustrate that our PFS2algorithm can privately find frequent sequences with high accuracy.

Co-clustering has emerged to be a powerful data mining tool for two-dimensional co-occurrence and dyadic data. However, co-clustering algorithms often require significant computational resources and have been dismissed as impractical for large data sets. Existing studies have provided strong empirical evidence that expectation-maximization (EM) algorithms (e.g., k-means algorithm) with sequential updates can significantly reduce the computational cost without degrading the resulting solution. Motivated by this observation, we introduce sequential updates for alternate minimization co-clustering (AMCC) algorithms which are variants of EM algorithms, and also show that AMCC algorithms with sequential updates converge. We then propose two approaches to parallelize AMCC algorithms with sequential updates in a distributed environment. Both approaches are proved to maintain the convergence properties of AMCC algorithms. Based on these two approaches, we present a new distributed framework, Co-ClusterD, which supports efficient implementations of AMCC algorithms with sequential updates. We design and implement Co-ClusterD, and show its efficiency through two AMCC algorithms: fast nonnegative matrix tri-factorization (FNMTF) and information theoretic co-clustering (ITCC). We evaluate our framework on both a local cluster of machines and the Amazon EC2 cloud. Empirical results show that AMCC algorithms implemented in Co-ClusterD can achieve a much faster convergence and often obtain better results than their traditional concurrent counterparts.

Recently, there has been a growing interest in designing differentially private data mining algorithms. Frequent itemset mining (FIM) is one of the most fundamental problems in data mining. In this paper, we explore the possibility of designing a differentially private FIM algorithm which can not only achieve high data utility and a high degree of privacy, but also offer high time efficiency. To this end, we propose a differentially private FIM algorithm based on the FP-growth algorithm, which is referred to as PFP-growth. The PFP-growth algorithm consists of a preprocessing phase and a mining phase. In the preprocessing phase, to improve the utility and privacy tradeoff, a novel smart splitting method is proposed to transform the database. For a given database, the preprocessing phase needs to be performed only once. In the mining phase, to offset the information loss caused by transaction splitting, we devise a run-time estimation method to estimate the actual support of itemsets in the original database. In addition, by leveraging the downward closure property, we put forward a dynamic reduction method to dynamically reduce the amount of noise added to guarantee privacy during the mining process. Through formal privacy analysis, we show that our PFP-growth algorithm is ε-differentially private. Extensive experiments on real datasets illustrate that our PFP-growth algorithm substantially outperforms the state-of-the-art techniques.

Co-clustering is a powerful data mining tool for co-occurrence and dyadic data. As data sets become increasingly large, the scalability of co-clustering becomes more and more important. In this paper, we propose two approaches to parallelize co-clustering with sequential updates in a distributed environment. Based on these two approaches, we present a new distributed framework, Co-ClusterD, that supports efficient implementations of co-clustering algorithms with sequential updates. We design and implement Co-ClusterD, and show its efficiency through two co-clustering algorithms: fast nonnegative matrix tri-factorization (FNMTF) and information theoretic co-clustering (ITCC). We evaluate our framework on both a local cluster of machines and the Amazon EC2 cloud. Our evaluation shows that co-clustering algorithms implemented in Co-ClusterD can achieve better results and run faster than their traditional concurrent counterparts.