Large language models (LLMs) have transformed recommender systems through strong semantic understanding and generalization. However, the design of item identifiers remains a critical bottleneck that directly affects recommendation quality. Traditional metadata-based identifiers introduce length variability and semantic ambiguity, whereas existing collaborative indexing (CID) approaches often neglect item attributes, show limited cross-dataset generalizability, and incur high computational cost at scale. To address these limitations, we propose a Graph Neural Network (GNN)–based item indexing framework with three coordinated innovations. First, we construct attribute-enriched co-occurrence graphs and use a GNN encoder to fuse item features with collaborative signals, yielding semantically informed representations that work well for attribute-rich catalogs. Second, we replace recursive spectral clustering with hierarchical agglomerative clustering on GNN embeddings, enabling direct control of index length via tree depth and reducing hyperparameter tuning across datasets. Third, we exploit localized message passing rather than global eigendecomposition, which provides considerably better runtime efficiency and is amenable to mini-batch training, supporting online index updates as interactions evolve. Across five benchmarks, GID achieves strong average ranking performance, showing larger improvements on sparse and attribute-rich datasets while remaining competitive in dense settings. The framework is robust under both seen and unseen prompt templates, which supports practical LLM-based recommendation. On sequential recommendation, GID improves HR@10 by 7.9% on average over the strongest baseline in each dataset.

The convergence of social networks and the Internet of Things (IoT) forms a complex digital ecosystem that links physical sensor data with human social interactions. However, the growing presence of social bots and fake sensor accounts introduces deceptive behaviors that distort relationships and undermine trust in IoT-enabled social platforms. Existing graph neural network (GNN) approaches have shown promise in social bot detection, yet most are built on the homophily assumption, which constrains their capacity to capture adversarial relationships between genuine and malicious entities. To address this challenge, we propose CAGNet, a Community-Aware Gated Network that jointly models temporal and structural properties. Specifically, user posts are represented as temporal sequences through a recurrent neural network to capture dynamic behavioral patterns, while local clustering coefficients and neighbor feature deviations guide an adaptive fusion of neighbor information and node residuals. Empirical results demonstrate that CAGNet effectively captures both temporal and structural dependencies, enhancing the model’s resilience against adversarial manipulation and improving the reliability of social bot detection in IoT-driven environments. Furthermore, we introduce the expert-annotated Chinese Weibo benchmark for social bot detection, offering a high-quality, reusable resource for evaluation. Experimental results show that the proposed method improves accuracy by 2.01% over current methods.

Multi-task learning (MTL) is a standard learning paradigm in machine learning. The central idea of MTL is to capture the shared knowledge among multiple tasks for mitigating the problem of data sparsity where the annotated samples for each task are quite limited. Recent studies indicate that graph multi-task learning (GMTL) yields the promising improvement over previous MTL methods. GMTL represents tasks on a task relation graph, and further leverages graph neural networks (GNNs) to learn complex task relationships. Although GMTL achieves the better performance, the construction of task relation graph heavily depends on simple heuristic tricks, which results in the existence of spurious task correlations and the absence of true edges between tasks with strong connections. This problem largely limits the effectiveness of GMTL. To this end, we propose the Generative Causality-driven Network (GCNet), a novel framework that progressively learns the causal structure between tasks to discover which tasks are beneficial to be jointly trained for improving generalization ability and model robustness. To be specific, in the feature space, GCNet first introduces a feature-level generator to generate the structure prior for reducing learning difficulty. Afterwards, GCNet develops a output-level generator which is parameterized as a new causal energy-based model (EBM) to refine the learned structure prior in the output space driven by causality. Benefiting from our proposed causal framework, we theoretically derive an intervention contrastive estimation for training this causal EBM efficiently. Experiments are conducted on multiple synthetic and real-world datasets. Extensive empirical results and model analyses demonstrate the superior performance of GCNet over several competitive MTL baselines.

Graph anomaly detection (GAD) refers to identifying abnormal graph nodes or edges that heavily deviate from normal observations. Existing approaches inevitably suffer from the influence of imbalanced data and privacy protection. This shortcoming poses challenges in optimizing node embeddings and detecting multitype anomalies simultaneously, resulting in decreased accuracy of existing GAD models. To address this shortcoming, we introduce a new federated learning model for graph anomaly detection (FedGAD). FedGAD enables collaborative unsupervised learning among decentralized data centers without requiring direct access to the distributed subgraphs. Specifically, FedGAD masks and reconstructs the neighborhood features to enhance the knowledge of node representations. Considering the data diversity across distributed clients, we also design a cross-clients' node representation module that enables nodes to reconstruct neighbors by leveraging information from other clients. Furthermore, we use a multiscale contrastive learning function, which includes both structure-level and contextual-level learning functions, to detect graph anomalies in the condition that subgraphs located at different clients show imbalanced data distributions. Experimental results on seven benchmark datasets demonstrate the superior performance of FedGAD compared with baseline methods, verifying its capability of improving GAD performance.

Personalized federated learning (PFL) on graphs is an emerging field focusing on the collaborative development of architectures across multiple clients, each with distinct graph data distributions while adhering to strict privacy standards. This area often requires extensive expert intervention in model design, which is a significant limitation. Recent advancements have aimed to automate the search for graph neural network architectures, incorporating large language models (LLMs) for their advanced reasoning and self-reflection capabilities. However, two technical challenges persist. First, although LLMs are effective in natural language processing, their ability to meet the complex demands of graph neural architecture search (GNAS) is still being explored. Second, while LLMs can guide the architecture search process, they do not directly solve the issue of client drift due to heterogeneous data distributions. To address these challenges, we introduce a novel method, Personalized Federated Graph Neural Architecture Search (PFGNAS). This approach employs a task-specific prompt to identify and integrate optimal GNN architectures continuously. To counteract client drift, PFGNAS utilizes a weight-sharing strategy of supernet, which optimizes the local architectures while ensuring client-specific personalization. Extensive evaluations show that PFGNAS significantly outperforms traditional PFL methods, highlighting the advantages of integrating LLMs into personalized federated learning environments.

Instruction tuning for large language models (LLMs) can drive them to produce results consistent with human goals in specific downstream tasks. However, the process of continual instruction tuning (CIT) for LLMs may bring about the catastrophic forgetting (CF) problem, where previously learned abilities are degraded. Recent methods try to alleviate the CF problem by modifying models or replaying data, which may only remember the surface-level pattern of instructions and get confused on held-out tasks. In this paper, we propose a novel continual instruction tuning method based on Key-part Information Gain (KPIG). Our method computes the information gain on masked parts to dynamically replay data and refine the training objective, which enables LLMs to capture task-aware information relevant to the correct response and alleviate overfitting to general descriptions in instructions. In addition, we propose two metrics, P-score and V-score, to measure the generalization and instruction-following abilities of LLMs. Experiments demonstrate our method achieves superior performance on both seen and held-out tasks.

Heterogeneous graphs are increasingly used to represent complex real-world scenarios with diverse entities and interactions by meta structures. Recently, the search of meta structures is combined with graph neural architecture search to automatically extract the semantic knowledge for various tasks in heterogeneous graphs. However, prior research primarily focuses on identifying meta structures that are universally applicable across all nodes in a graph, neglecting the variations in meta structure selection that arise from the unique features and topology of individual nodes. To address this challenge, we introduce a Node-Centric approach to search Meta Structures in heterogeneous graphs (NC-MS for short). NC-MS implements a node level method that discover meaningful meta structures tailored to each node, capturing subtle differences in meta structure choices between nodes and providing nuanced identification. Additionally, NC-MS utilizes an efficient and differentiable network to enhance operational efficiency. Empirical studies across three real-world datasets validate the superiority of NC-MS, demonstrating its ability to outperform existing models in heterogeneous graph neural networks.

Graph Anomaly Detection (GAD) identifies deviant patterns including anomalous nodes, edges, and subgraphs in graph data, with significant applications in social networks, cybersecurity, and financial risk control. While spectral methods have proven effective for homogeneous graph anomaly detection, their application to heterogeneous graphs remains challenging due to structural complexity and semantic richness. Existing heterogeneous graph anomaly detection methods either rely on manually designed meta-paths or decompose the graph into homogeneous subgraphs, leading to limited flexibility or loss of structural integrity. To address these limitations, we propose the Graph Wavelet Transformer (GWT), a novel spectral-based approach that integrates global graph properties and spectral analysis without requiring meta-path information. GWT employs a three-stage process: heterogeneous-to-homogeneous graph conversion, global dependency modeling via graph transformers, and spectral-aware feature enhancement focusing on frequency band components. Extensive experiments on multiple benchmarks demonstrate that GWT significantly outperforms ten baseline methods, providing a new paradigm for heterogeneous graph anomaly detection that preserves structural completeness while achieving computational efficiency.

Anomaly detection in continuous-time event sequences is a crucial task in safety-critical applications. While existing methods primarily focus on developing a superior test statistic, they fail to provide guarantees regarding the false positive rate (FPR), which undermines their reliability in practical deployments. In this paper, we propose CADES (Conformal Anomaly Detection in Event Sequences), a novel test procedure based on conformal inference for the studied task with finite-sample FPR control. Specifically, by using the time-rescaling theorem, we design two powerful non-conformity scores tailored to event sequences, which exhibit complementary sensitivities to different abnormal patterns. CADES combines these scores with Bonferroni correction to leverage their respective strengths and addresses non-identifiability issues of existing methods. Theoretically, we prove the validity of CADES and further provide strong guarantees on calibration-conditional FPR control. Experimental results on synthetic and real-world datasets, covering various types of anomalies, demonstrate that CADES outperforms state-of-the-art methods while maintaining FPR control.

The past decade has witnessed rapid advancements in cross-modal retrieval, with significant progress made in accurately measuring the similarity between cross-modal pairs. However, the persistent hubness problem, a phenomenon where a small number of targets frequently appear as nearest neighbors to numerous queries, continues to hinder the precision of similarity measurements. Despite several proposed methods to reduce hubness, their underlying mechanisms remain poorly understood. To bridge this gap, we analyze the widely-adopted Inverted Softmax approach and demonstrate its effectiveness in balancing target probabilities during retrieval. Building on these insights, we propose a probability-balancing framework for more effective hubness reduction. We contend that balancing target probabilities alone is inadequate and, therefore, extend the framework to balance both query and target probabilities by introducing Sinkhorn Normalization (SN). Notably, we extend SN to scenarios where the true query distribution is unknown, showing that current methods, which rely solely on a query bank to estimate target hubness, produce suboptimal results due to a significant distributional gap between the query bank and targets. To mitigate this issue, we introduce Dual Bank Sinkhorn Normalization (DBSN), incorporating a corresponding target bank alongside the query bank to narrow this distributional gap. Our comprehensive evaluation across various cross-modal retrieval tasks, including image-text retrieval, video-text retrieval, and audio-text retrieval, demonstrates consistent performance improvements, validating the effectiveness of both SN and DBSN. All codes are publicly available at https://github.com/ppanzx/DBSN.

Graph-level out-of-distribution (OOD) detection, which attempts to identify OOD graphs originated from an unknown distribution, is a vital building block for safety-critical applications in Web and society. Current approaches concentrate on how to learn better graph representations, but fail to provide any statistically guarantee on detection results, therefore impeding their deployments in the scenario where detection errors would result in serious consequences. To overcome this critical issue, we propose the Conformal Graph-level Out-of-distribution Detection (CGOD), extending the theory of conformal prediction to graph-level OOD detection with a rigorous control over the false positive rate. In CGOD, we develop a new aggregated non-conformity score function based on the proposed adaptive data augmentation. Through the guidance from two designed metrics, i.e., score consistency and representation diversity, our augmentation strategy can generate multiple non-conformity scores, and aggregating these generated non-conformity scores together is robust to the misleading information. Meanwhile, our score function can perceive the subsequent process of conformal inference, enabling the aggregated non-conformity score to be adaptive to different input graphs and deriving a more accurate score estimation. We conduct experiments on multiple real-world datasets with different empirical settings. Extensive results and model analyses demonstrate the superior performance of our approach over several competitive baselines.

Message passing scheme is a general idea for Graph Neural Networks (GNNs) to learn node representations. During message passing, given a target node, we transform and aggregate the feature vectors of its neighbors and generate a representation vector for the target node. However, real-world graph data is usually constructed from complicated scenarios based on manually pre-defined rules; it is often the case that noisy information gets involved in message passing, thereby resulting in sub-optimal performance for GNNs and also impacting their trustworthiness and reliability. In this study, we present an effective learnable threshold technique that explicitly optimizes heterogeneous graph structure with the goal to maximize performance improvement of GNNs for downstream tasks. We give an explanation about the design of the learnable threshold and show the ability that our model can be applied to large-scale graphs. Experiments on seven datasets show that our model has a powerful ability to deal with homogeneous graphs with low homophily ratio and dense graphs. With the verification of robustness analysis, our model can resist the noisy information, which proves the robustness of our model.

With the rapid development of mobile network technology and wearable mobile devices, user-scenario interactions generate a large amount of user behavioral data in the form of multivariate event sequences. Due to data isolation, these multi-scenario events need to be jointly trained to achieve better prediction results. However, traditional federated learning methods face significant challenges when handling distributed event sequences. And the effectiveness of existing modeling approaches for event sequences in federated contexts has not been thoroughly explored. To this end, we propose Federated Multivariate Temporal Point Processes (FedMTPP), which enables learning from distributed event sequences within a novel federated learning framework and leverages efficient event modeling technology, MTPP, to forecast future events. Specifically, FedMTPP restores the temporal structure of the original event sequence by rearranging event embeddings and redesigns the autoregressive-based hidden representation computation in traditional MTPP, making it more suitable for federated prediction tasks. Additionally, FedMTPP incorporates advanced encryption techniques to effectively safeguard user privacy and security. Experimental results on both synthetic and real datasets demonstrate that FedMTPP substantially improves the performance of local models and achieves results comparable to state-of-the-art centralized MTPP methods.

Recently link weight prediction has attracted an increasing research interest due to its merits in quantifying the strength between nodes within a graph. Nonetheless, current link weight prediction methods focus solely on graph topology, disregarding node feature information embedded in graphs. In real-world applications, we often collect heterogeneous graph data where multiple types of nodes linked by multiple types of edges are available for analysis, and it is essential and challenging to quantify the proximity of different types of nodes. To solve this challenge, we present a new model for Heterogeneous Graph Link Weight Prediction (HLWP for short). In HLWP, message passing in heterogeneous graph neural networks is described as a meta structure, which can be effectively designed by Differentiable Neural Architecture Search (DARTS) algorithms. Thus, HLWP can enhance the message passing in heterogeneous graphs by DARTS. In addition, HLWP employs a perturbation-based algorithm to enhance stability and precision. Through empirical experiments conducted on three real-world datasets, we demonstrate that HLWP achieves accurate predictions of link weights. Our results highlight the superiority of HLWP over existing methods for link weight prediction and baseline GNN models in terms of accurately predicting link weights within heterogeneous graphs.

Methods based on graph neural networks for solving combinatorial optimization (CO) problems have exhibited promising results in tackling a range of NP-hard problems, eliminating the necessity for reliance on manually created domain knowledge. Existing models including reinforcement learning (RL) framework assume that combinatorial instances in the training set contribute equally during training. Nevertheless, there is considerable variation in the quality of training instances, and the performance of models may suffer from the inclusion of low-quality training instances. This paper expands the current scope of neural solvers for CO problems through the incorporation of curriculum learning (CL). To alleviate the adverse impact of low-quality training instances, we propose CL4CO which utilizes CL strategy, a selective training method, to train models based on the rank of instances' quality in neural Combinatorial Optimization framework. Also, we introduce several candidate topology-aware metrics based on heterophily ratio and evaluation of clustering for the training scheduler. Furthermore, it is noteworthy to emphasize that it has potential to enhance the generalization capacity of RL-based baselines and we give a experimental validation. This enhancement plugin from the fact that CL empowers the acquired RL-based solver to effectively leverage commonly shared features within the same class of CO. Empirically, we conduct a case study on MaxCut, a classical discrete Oil-vector CO, to verify our findings and our results demonstrate that CL4CO is efficient and superiority with good generalization ability.

We present a novel perspective on temporal point processes (TPPs) by reformulating their intensity processes as solutions to stochastic differential equations (SDEs). In particular, we first prove the equivalent SDE formulations of several classical TPPs, including Poisson processes, Hawkes processes, and self-correcting processes. Based on these proofs, we introduce a unified TPP framework called Neural Jump-Diffusion Temporal Point Process (NJDTPP), whose intensity process is governed by a neural jump-diffusion SDE (NJDSDE) where the drift, diffusion, and jump coefficient functions are parameterized by neural networks. Compared to previous works, NJDTPP exhibits model flexibility in capturing intensity dynamics without relying on any specific functional form, and provides theoretical guarantees regarding the existence and uniqueness of the solution to the proposed NJDSDE. Experiments on both synthetic and real-world datasets demonstrate that NJDTPP is capable of capturing the dynamics of intensity processes in different scenarios and significantly outperforms the state-of-the-art TPP models in prediction tasks.

In the context of privacy-aware social web services, secure causal reasoning on graphs is an essential task, for example, identifying the key influential nodes/users through which information, ideas, trends spread across the network. Despite the significant advancements achieved by neural network-based methods in these domains, existing research often assumes complete observable of graph data, an assumption that conflicts with privacy concerns in social web services. This assumption may not hold in numerous real-world social analysis applications due to privacy restrictions. In such scenarios, some graph links might remain unobserved until the model is trained, posing a critical challenge for privacy preservation. To address this issue, we introduce a novel model named Causal Effect on Coarsened Graphs (Cecoar), which is specifically designed for privacy-aware social web services. This model employs a new type of coarsened graph that accurately represents partially observed nodes and edges, aligning with the privacy constraints of social web services. We have developed three crucial modules to enable causal estimation on coarsened graphs: (1) a graph sampling module, (2) a confounder encoder, and (3) a graph-based encoder with causal prediction layers. Furthermore, we present a comprehensive theoretical analysis of the proposed model, demonstrating its performance and generalization capabilities. We evaluate our model on two benchmark datasets for causal estimation, and the results confirm its effectiveness in handling graph data containing private information. The code is in https://github.com/liu-yang-maker/Cecoar-ICWS24.

Anonymous networks play a crucial role in preserving information privacy, yet simultaneously spark technical conflicts and gamesmanship between their maintainers and regulatory authorities. To solve the above conflicts and economic losses in real networks, it is imminent to realize adversarial exercises in anonymous networks and validate new technologies and scenarios in the cyber range. It is well known that the two dominant types of anonymity networks are Tor and Invisible Internet Project (I2P). This paper establishes a cyber range based on I2P to realize a task-driven automated scenario construction technique. The proposed task-driven automated scenario construction technique can solve the problem of decoupling between business scenarios and I2P cyber range network infrastructure. Specifically, our approach allows users to upload models/executable code to extend flexibly and fine-grained control I2P nodes, protocols, and traffic behavior characteristics. Then it empowers users to define application scenarios as needed, enabling them to handle intricate business scenarios programmatically. Therefore, it further supports diverse adversarial exercises and the validation of new technologies and scenarios within the dark web.

Uncovering temporal dependency patterns behind event sequences plays a key role in predicting event types and event times. Recently, transformers based models have been used to describe point processes, such as the Transformer Hawkes Processes (THP models). However, existing THP models assume that data are collected in a central server and can be always seen during model training. Indeed, event sequence data are often located at different data centers which can not be shared directly due to the risk of privacy leakage. To this end, we combine in this paper the THP models with federated learning, enabling collaborative learning from a large amount of distributed event sequence data. Experiments show that our approach surpasses single data source training while preserving data privacy. For clients lacking certain types of event sequence data, our method performs much more stable than previous centralized training models.

Graph classification plays an important role in a wide range of applications from biological prediction to social analysis. Traditional graph classification models built on graph kernels are hampered by the challenge of poor generalization as they are heavily dependent on the dedicated design of handcrafted features. Recently, graph neural networks (GNNs) become a new class of tools for analyzing graph data and have achieved promising performance. However, it is necessary to collect a large number of labeled graph data for training an accurate GNN, which is often unaffordable in real-world applications. Therefore, it is an open question to build GNNs under the condition of few-shot learning where only a few labeled graphs are available. In this article, we introduce a new Structure-aware Prototypical Neural Process (SPNP for short) for a few-shot graph classification. Specifically, at the encoding stage, SPNP first employs GNNs to capture graph structure information. Then, SPNP incorporates such structural priors into the latent path and the deterministic path for representing stochastic processes. At the decoding stage, SPNP uses a new prototypical decoder to define a metric space where unseen graphs can be predicted effectively. The proposed decoder, which contains a self-attention mechanism to learn the intraclass dependence between graphs, can enhance the class-level representations, especially for new classes. Furthermore, benefited from such a flexible encoding-decoding architecture, SPNP can directly map the context samples to a predictive distribution without any complicated operations used in previous methods. Extensive experiments demonstrate that SPNP achieves consistent and significant improvements over state-of-the-art methods. Further discussions are provided toward model efficiency and more detailed analysis.

Decision-focused learning (DFL) combines both machine learning and combinatorial optimization so as to enhance the quality of decision-making. In general, DFL adds an optimization layer after the neural network and solves a focused combinatorial optimization problem. The optimization layer is usually based on KKT conditions or surrogate functions. However, the optimizer associated with the DFL is inflexible and requires a large amount of expert knowledge. Furthermore, it has been shown that the optimizer exhibits a lack of robustness and differentiability when confronted with complex tasks, especially in the graph domain. To solve this problem, we study a more generic situation by taking the optimization layer as a black-box operator on graphs. Then, we design a mixed zeroth-order optimization to differentiate the layer. Specifically, we combine DFL with graph neural networks and present a decision-focused graph neural network named ZO4Graph based on a mixed zeroth-order optimizer. Empirically, we conduct extensive experiments compared with two-stage models on community detection. Numerical results demonstrate that our proposed framework outperforms its peers.

Anomaly detection in multi-type event sequences is a crucial and challenging problem with important applications in various domains, including cybersecurity, finance and healthcare. Temporal point process has emerged as a powerful technique for modeling event sequences and has gained considerable attention in the field of anomaly detection. However, existing temporal point process approaches are either inapplicable to multi-type event sequence data or incur the loss of valuable information in subsequences associated with specific event types. To this end, we propose a novel Multiple Hypothesis Testing based Anomaly Detection method (MultiAD) to detect anomalous multi-type event sequences. The basic idea of MultiAD is to capture the underlying distribution of normal sequences using a neural multivariate point process, based on which the original hypothesis testing problem can be converted into a multiple hypothesis testing using the multivariate time rescaling theorem. By conducting multiple hypothesis tests on the time-rescaled subsequences, MultiAD makes full use of the valuable information contained within individual subsequences. Moreover, we claim that the existing test statistic ignores the sequential information of inter-event time intervals and propose new statistics to address this shortcoming. Finally, we employ the kernel method to obtain a smooth estimator of the distribution of the proposed statistics under the null hypothesis. This ensures a more accurate and reliable computation of the p-value, providing robust statistical inference. Extensive experimental results demonstrate that MultiAD significantly outperforms the state-of-the-art methods on both synthetic and real-world data.

Graph neural networks (GNNs) are powerful models to handle graph data and can achieve state-of-the-art in many critical tasks including node classification and link prediction. However, existing graph neural networks still face both challenges of over-smoothing and over-squashing based on previous literature. To this end, we propose a new Curvature-based topology-aware Dropout sampling technique named CurvDrop, in which we integrate the Discrete Ricci Curvature into graph neural networks to enable more expressive graph models. Also, this work can improve graph neural networks by quantifying connections in graphs and using structural information such as community structures in graphs. As a result, our method can tackle the both challenges of over-smoothing and over-squashing with theoretical justification. Also, numerous experiments on public datasets show the effectiveness and robustness of our proposed method. The code and data are released in https://github.com/liu-yang-maker/Curvature-based-Dropout.

Data augmentation has been widely studied as it can be used to improve the generalizability of graph representation learning models. However, existing works focus only on the data augmentation on homogeneous graphs. Data augmentation for heterogeneous graphs remains under-explored. Considering that heterogeneous graphs contain different types of nodes and links, ignoring the type information and directly applying the data augmentation methods of homogeneous graphs to heterogeneous graphs will lead to suboptimal results. In this paper, we propose a novel Multi-Aspect Heterogeneous Graph Augmentation framework named MAHGA. Specifically, MAHGA consists of two core augmentation strategies: structure-level augmentation and metapath-level augmentation. Structure-level augmentation pays attention to network schema aspect and designs a relation-aware conditional variational auto-encoder that can generate synthetic features of neighbors to augment the nodes and the node types with scarce links. Metapath-level augmentation concentrates on metapath aspect, which constructs metapath reachable graphs for different metapaths and estimates the graphons of them. By sampling and mixing up based on the graphons, MAHGA yields intra-metapath and inter-metapath augmentation. Finally, we conduct extensive experiments on multiple benchmarks to validate the effectiveness of MAHGA. Experimental results demonstrate that our method improves the performances across a set of heterogeneous graph learning models and datasets.

Graph neural networks (GNNs) are popularly used to analyze non-Euclidean graph data. Despite their successes, the design of graph neural networks requires heavy manual work and rich domain knowledge. Recently, neural architecture search algorithms are widely used to automatically design neural architectures for CNNs and RNNs. Inspired by the success of neural architecture search algorithms, we present a graph neural architecture search algorithm GraphNAS that enables automatic design of the best graph neural architecture based on reinforcement learning. Specifically, GraphNAS uses a recurrent network as the controller to generate variable-length strings that describe the architectures of graph neural networks, and trains the recurrent network with policy gradient to maximize the expected accuracy of the generated architectures on a validation data set. Moreover, based on GraphNAS, we design a new GraphNAS++ model using distributed neural architecture search. Compared with GraphNAS that generates and evaluates only one candidate architecture at each iteration, GraphNAS++ generates a mini-batch of candidate architectures and evaluates them in a distributed computing environment until convergence. Experiments on real-world datasets demonstrate that GraphNAS can design a novel network architecture that rivals the best human-invented architecture. Moreover, GraphNAS++ can speed up the design process at least five times by using the distributed training framework with GPUs.

Heterogeneous graphs are commonly used to describe networked data with multiple types of nodes and edges. Heterogeneous Graph Neural Networks (HGNNs) are powerful tools for analyzing heterogeneous graphs. However, designing neural architectures of HGNNs requires extensive domain knowledge and time-consuming manual work. Recently, neural architecture search algorithms have become popular in automatically designing neural architectures for homogeneous graph neural networks. In this paper, we present a Heterogeneous Graph Neural Architecture Search algorithm (HGNAS for short) which allows the automatic design of heterogeneous graph neural architectures. Specifically, HGNAS first designs a new search space based on existing popular HGNNs. Then, HGNAS uses a policy network as the controller to sample and find the best neural architecture from the designed search space by maximizing the expected accuracy of the selected architectures on a given validation dataset. Moreover, we design a new method HGNAS++ to improve the efficiency of HGNAS by training the RNN controller within a generative adversarial learning framework. The basic idea of HGNAS++ is to embed a pairwise ranker into the reinforcement learning based architecture search algorithm. The pairwise ranker can be taken as a discriminator which selects more accurate architectures between pairs of candidate architectures. Then, the RNN controller can be updated more efficiently by only using a relatively small number of candidate architectures selected by the pairwise ranker. Experiments on real-world heterogeneous graph datasets show that HGNAS is capable of designing novel HGNNs that beat the best human-invented HGNNs. On the benchmark datasets, HGNAS++ improves HGNAS in terms of evaluation cost, with a reduction of 50% of the evaluated candidate architectures and a decrease of 24% in search time on average. As a byproduct, HGNAS++ can find sparse yet powerful neural architectures for HGNNs.

Unsupervised cross-domain NER task aims to solve the issues when data in a new domain are fully-unlabeled. It leverages labeled data from source domain to predict entities in unlabeled target domain. Since training models on large domain corpus is time-consuming, in this paper, we consider an alternative way by introducing syntactic dependency structure. Such information is more accessible and can be shared between sentences from different domains. We propose a novel framework with dependency-aware GNN (DGNN) to learn these common structures from source domain and adapt them to target domain, alleviating the data scarcity issue and bridging the domain gap. Experimental results show that our method outperforms state-of-the-art methods.

Semantic parsing is a mainstream method of knowledge base question answering task that first generates a set of logical forms according to question and knowledge base (KB), and then selects the most matching one to get answers. However, existing selection methods are usually based on word-level matching, which cannot capture the structural information or solve the long-term dependency problem of entities. To solve this problem, we propose a syntax-based graph matching method, which explicitly models both question and logical form as graphs, and performs matching at both word-level and structure-level. The multi-level matching strategy not only captures the structural and semantic relations between entities but also explores their intrinsic relations. Such a design can greatly minimize the gap between question and most relevant knowledge from KB, and hence can reason more accurately. We conduct extensive experiments on several benchmarks and demonstrate the effectiveness of our proposed method.

Structured sequences are a popular data representation, used to model complex data such as traffic networks. A key machine learning task for structured sequences is node classification, that is predicting the class labels of unlabeled nodes. Though many node classification models were proposed, they assume a closed world setting, that all class labels appear in the training data. But in the real-world, the presence of never-before-seen class labels in testing data can considerably degrade a classifier’s accuracy. A promising solution to this issue is to build classifiers for an open-world setting, where samples with unknown class labels are continuously observed such that training and testing data may have different class label spaces. Several approaches have been proposed for open-world learning problems in computer vision and natural language processing, but they cannot be applied directly to structured sequences due to the complexity of their non-Euclidean properties and their dynamic nature. This paper addresses this important research gap by proposing a novel Open-world Structured Sequence node Classification (OSSC) model, to learn from structured sequences in an open-world setting. OSSC captures the structural and temporal information via a GCN-based dynamic variational framework. A latent distribution sequence is learned for each node using both stochastic states and deterministic states, to capture the evolution of node attributes and topology, followed by a sampling process to generate node representations. An open-world classification loss is further adopted to ensure that node representations are sensitive to unknown classes. And a combination of Openmax and Softmax is utilized to recognize nodes from unknown classes and to classify others to one of the known classes. Experiments on real-world datasets show that the proposed OSSC method is capable of learning accurate open-world node classifiers from structured sequence data.

Recently, the field of natural language processing (NLP) has grown rapidly, driven by massive datasets. At the same time, the need for automatic summarization systems has been rapidly increasing as the amount of textual information on the web and in large data centers became intractable for human readers. However, the lack of large-scale and high-quality Chinese datasets remain a critical bottleneck for further research on automatic text summarization. To close this gap, we searched domestic and foreign Chinese news websites and designed the FCFS (Format and Content Filtering System) to crawl and filter these records to construct NEWSFARM. The NEWSFARM is a large-scale Chinese long news summarization corpus, containing more than 220K Chinese long news and summaries written by professional editors or authors, all of which are released to the public1. We calculated the static metrics and designed many experiments with the baseline models to evaluate the dataset. By comparing with the common datasets, the results not only demonstrate the usefulness and challenges of the proposed corpus for automatic text summarization but also validate the superiority of FCFS.

Guaranteed Display (GD) is widely used in e-commerce marketing for advertisers to acquire an agreed-upon number of impressions with target audiences. With the main objective to maximize the contract delivery rate under contract constraints, user interest (such as click-through rate and conversion rate) is also essential to improve the long-time return on investment for advertisers and the e-commerce platform. In this paper, we design an adaptive unified allocation framework (AUAF) by not only considering supply of audience impressions in request-level but also avoiding over-allocation of audience impressions. Specifically, our allocation model simultaneously optimizes the contract delivery and the match between advertisements and user interests with explicit constraint to prevent unnecessary allocation. Facing the challenge of serving billion-scale requests per day, a parameter-server based parallel optimization algorithm is also developed, enabling the proposed allocation model to be efficiently optimized and incrementally updated in minutes. Thus, the offline optimization results and the online decisions can be synchronized for real-time serving. In other words, our approach can achieve adaptive pacing that is consistent with the optimal allocation solution. Our extensive experimental results demonstrate that the proposed AUAF framework can improve both contract delivery rate and average click-through rate (CTR), which we use to measure the user interest in this paper. The improvements on CTR are statistically significant in comparison with existing methods. Moreover, since March 2020, AUAF has been deployed in the guaranteed display advertising system of Alibaba, bringing more than 10% increase on CTR without loss of contract delivery rate, which has resulted in significant value creation for the business.

In this demo, we present an online multi-modal community query engine (MQE) on Alibaba's billion-scale heterogeneous network. MQE has two distinct features in comparison with existing community query engines. Firstly, MQE supports multimodal community search on heterogeneous graphs with keyword and image queries. Secondly, to facilitate community understanding in real business scenarios, MQE generates natural language descriptions for the retrieved community in combination with other useful demographic information. The distinct features of MQE benefit many downstream applications in Alibaba's e-commerce platform like recommendation. Our experiments confirm the effectiveness and efficiency of MQE on graphs with billions of edges.

On-demand food delivery (OFD) platforms have greatly impacted the food service industry, where OFD recommendation systems play a central role in enhancing user experience and raising revenues. OFD recommendation, compared with existing online e-commerce recommendation systems, needs to put more emphasis on fulfillment time related variables, because the order fulfillment cycle time (OFCT) which refers to the time elapsed between a user placing a food order and receiving the food significantly influences a user's choice from the recommended items. In this paper, we investigate the OFCT related information and propose a Fulfillment-Time-Aware Personalized Ranking (FTAPR) method for recommendation. FTAPR mainly consists of three components. First, Transformers are used to estimate OFCT based on a large amount of user order sequences. Then, the predicted OFCT and other OFCT related features are fused and encoded by a deep & cross network to learn fulfillment time related feature representation. At the last step, the time bias representation from the deep & cross network is integrated into the ranking system to deliver final search results. Extensive offline and online experiments on real-world datasets collected from one of China's largest OFD platforms Ele.me show the superiority of our model, e.g., an online A/B testing shows that FTAPR brings 1.3% and 2.5% gains in CTR and CVR compared with baselines.

Malicious Web crawlers threaten information system due to heavily taking up bandwidth resources and stealing private user data. Ele.me, a prevalent on-demand food delivery platform in China, suffers from the negative impact of crawlers. The crawler detection systems face two major challenges: spatial patterns of the crawler behaviors and limited labeled data for training. In this paper, we present efficient solutions to tackle these challenges. Specifically, we propose a new Attributed Action Net (AANet for short) model to detect Location-Based Services~(LBS) crawlers and a three-stage learning framework to train the model. AANet consists of three different embedding modules, including the action token sequence, temporal-spatial attributes of users, and the context information of the raw data. We have deployed the model at Ele.me, and both offline experiments and online A/B tests show that the proposed method is superior to the state-of-the-art models for sequence data classification on the food delivery platform.

Cross-domain slot filling is a challenging task in spoken language understanding due to the differences in text genre across domains. In this paper, we attempt to solve this task by exploiting the syntactic structures of user utterances, because these syntactic structures are actually accessible and can be shared between utterances from different domains. To this end, we propose a novel Syntactic Structure Encoder (SSE) module and incorporate it into a detection-prediction framework. SSE introduces graph convolutional network (GCN) to learn the common structures from multiple source domains, which are helpful to better adaptation on the target domain. Experimental results conducted on SNIPS dataset show that our model significantly outperforms the state-of-the-art approach in cross-domain slot filling. Specifically, our model outperforms the best model by ∼4% and ∼5% F1-scores under the 20-example and 50-example settings, respectively.

Heterogeneous Graph Neural networks (HGNNs) have been popularly used in processing complicated networks such as academic networks, social networks, and knowledge graphs. Despite their success, the design of the neural architectures of HGNNs still requires rich domain knowledge and heavy manual work. In this paper, we propose a Heterogeneous Graph Neural Architecture Search algorithm (HGNAS for short) which enables automatic design of the best neural architectures with minimal human effort. Specifically, HGNAS first defines a general HGNN framework on top of existing popular HGNNs. A search space of HGNAS is designed based on the general framework that includes multiple groups of message encoding and aggregation functions. Then, HGNAS uses a policy network as the controller to sample and find the best neural architecture from the designed search space by maximizing the expected accuracy of the selected architectures on a validation dataset. Moreover, we introduce effective methods to improve HGNAS from three aspects, i.e., the optimization of hyper-parameters, the improvement of search space, and the selection of message receptive fields. Experiments on public datasets show that HGNAS is capable of designing novel HGNNs that rival the best human-invented HGNNs. More interestingly, HGNAS finds some sparse yet powerful neural architectures for HGNNs on the benchmark datasets.

Multivariate Hawkes processes (MHPs) are classic methods to learn temporal patterns in event sequences of different entities. Traditional MHPs with explicit parametric intensity functions are friendly to model interpretability. However, recent Deep MHPs which employ various variants of recurrent neural networks are hardly to understand, albeit more expressive towards event sequences. The lack of model interpretability of Deep MHPs leads to a limited comprehension of complicated dynamics between events. To this end, we present a new Disentangled Deep Multivariate Hawkes Process $(\mathrm{D}^{2}$ MHP) to enhance model expressiveness and meanwhile maintain model interpretability. $\mathrm{D}^{2}$ MHP achieves state disentanglement by disentangling the latent representation of an event sequence into static and dynamic latent variables, and matches these latent variables to interpretable factors in the intensity function. Moreover, considering that an entity typically has multiple identities, $\mathrm{D}^{2}$ MHP further splits these latent variables into factorized representations, each of which is associated with a corresponding identity. Experiments on real-world datasets show that $\mathrm{D}^{2}$ MHP yields significant and consistent improvements over state-of-the-art baselines. We also demonstrate model interpretability via the detailed analysis.

In recent years, temporal knowledge graph (TKG) reasoning has received significant attention. Most existing methods assume that all timestamps and corresponding graphs are available during training, which makes it difficult to predict future events. To address this issue, recent works learn to infer future events based on historical information. However, these methods do not comprehensively consider the latent patterns behind temporal changes, to pass historical information selectively, update representations appropriately and predict events accurately. In this paper, we propose the Historical Information Passing (HIP) network to predict future events. HIP network passes information from temporal, structural and repetitive perspectives, which are used to model the temporal evolution of events, the interactions of events at the same time step, and the known events respectively. In particular, our method considers the updating of relation representations and adopts three scoring functions corresponding to the above dimensions. Experimental results on five benchmark datasets show the superiority of HIP network, and the significant improvements on Hits@1 prove that our method can more accurately predict what is going to happen.

The interaction of multiple drugs could lead to serious events, which causes injuries and huge medical costs. Accurate prediction of drug-drug interaction (DDI) events can help clinicians make effective decisions and establish appropriate therapy programs. Recently, many AI-based techniques have been proposed for predicting DDI associated events. However, most existing methods pay less attention to the potential correlations between DDI events and other multimodal data such as targets and enzymes. To address this problem, we propose a Multimodal Deep Neural Network (MDNN) for DDI events prediction. In MDNN, we design a two-pathway framework including drug knowledge graph (DKG) based pathway and heterogeneous feature (HF) based pathway to obtain drug multimodal representations. Finally, a multimodal fusion neural layer is designed to explore the complementary among the drug multimodal representations. We conduct extensive experiments on real-world dataset. The results show that MDNN can accurately predict DDI events and outperform the state-of-the-art models.

Live-streaming platforms have recently gained significant popularity by attracting an increasing number of young users and have become a very promising form of online shopping. Similar to the traditional online shopping platforms such as Taobao, live-streaming platforms also suffer from online malicious fraudulent behaviors where many transactions are not genuine. The existing anti-fraud models proposed to recognize fraudulent transactions on traditional online shopping platforms are inapplicable on live-streaming platforms. This is mainly because live-streaming platforms are characterized by a unique type of heterogeneous live-streaming networks where multiple heterogeneous types of nodes such as users, live-streamers, and products are connected with multiple different types of edges associated with edge features. In this paper, we propose a new approach based on a heterogeneous graph neural network for LIve-streaming Fraud dEtection (called LIFE). LIFE designs an innovative heterogeneous graph learning model that fully utilizes various heterogeneous information of shopping transactions, users, streamers, and items from a given live-streaming platform. Moreover, a label propagation algorithm is employed within our LIFE framework to handle the limited number of labeled fraudulent transactions for model training. Extensive experimental results on a large-scale Taobao live-streaming platform demonstrate that the proposed method is superior to the baseline models in terms of fraud detection effectiveness on live-streaming platforms. Furthermore, we conduct a case study to show that the proposed method is able to effectively detect fraud communities for live-streaming e-commerce platforms.

The large-scale recommender system mainly consists of two stages: matching and ranking. The matching stage (also known as the retrieval step) identifies a small fraction of relevant items from billion-scale item corpus in low latency and computational cost. Item-to-item collaborative filtering (item-based CF) and embedding-based retrieval (EBR) have been long used in the industrial matching stage owing to its efficiency. However, item-based CF is hard to meet personalization, while EBR has difficulty in satisfying diversity. In this paper, we propose a novel matching architecture, Path-based Deep Network (named PDN), through incorporating both personalization and diversity to enhance matching performance. Specifically, PDN is comprised of two modules: Trigger Net and Similarity Net. PDN utilizes Trigger Net to capture the user's interest in each of his/her interacted item. Similarity Net is devised to evaluate the similarity between each interacted item and the target item based on these items' profile and CF information. The final relevance between the user and the target item is calculated by explicitly considering user's diverse interests, \ie aggregating the relevance weights of the related two-hop paths (one hop of a path corresponds to user-item interaction and the other to item-item relevance). Furthermore, we describe the architecture design of the proposed PDN in a leading real-world E-Commerce service (Mobile Taobao App). Based on offline evaluations and online A/B test, we show that PDN outperforms the existing solutions for the same task. The online results also demonstrate that PDN can retrieve more personalized and more diverse items to significantly improve user engagement. Currently, PDN system has been successfully deployed at Mobile Taobao App and handling major online traffic.

In the emerging business of food delivery, rider traffic accidents raise financial cost and social traffic burden. Although there has been much effort on traffic accident forecasting using temporal-spatial prediction models, none of the existing work studies the problem of detecting the takeaway rider accidents based on food delivery trajectory data. In this article, we aim to detect whether a takeaway rider meets an accident on a certain time period based on trajectories of food delivery and riders’ contextual information. The food delivery data has a heterogeneous information structure and carries contextual information such as weather and delivery history, and trajectory data are collected as a spatial-temporal sequence. In this article, we propose a TakeAway Rider Accident detection fusion network TARA-Net to jointly model these heterogeneous and spatial-temporal sequence data. We utilize the residual network to extract basic contextual information features and take advantage of a transformer encoder to capture trajectory features. These embedding features are concatenated into a pyramidal feed-forward neural network. We jointly train the above three components to combine the benefits of spatial-temporal trajectory data and sparse basic contextual data for early detecting traffic accidents. Furthermore, although traffic accidents rarely happen in food delivery, we propose a sampling mechanism to alleviate the imbalance of samples when training the model. We evaluate the model on a transportation mode classification dataset Geolife and a real-world Ele.me dataset with over 3 million riders. The experimental results show that the proposed model is superior to the state-of-the-art.

Graph neural networks (GNNs) emerged recently as a powerful tool for analyzing non-Euclidean data such as social network data. Despite their success, the design of graph neural networks requires heavy manual work and domain knowledge. In this paper, we present a graph neural architecture search method (GraphNAS) that enables automatic design of the best graph neural architecture based on reinforcement learning. Specifically, GraphNAS uses a recurrent network to generate variable-length strings that describe the architectures of graph neural networks, and trains the recurrent network with policy gradient to maximize the expected accuracy of the generated architectures on a validation data set. Furthermore, to improve the search efficiency of GraphNAS on big networks, GraphNAS restricts the search space from an entire architecture space to a sequential concatenation of the best search results built on each single architecture layer. Experiments on real-world datasets demonstrate that GraphNAS can design a novel network architecture that rivals the best human-invented architecture in terms of validation set accuracy. Moreover, in a transfer learning task we observe that graph neural architectures designed by GraphNAS, when transferred to new datasets, still gain improvement in terms of prediction accuracy.

Discrete network embedding emerged recently as a new direction of network representation learning. Compared with traditional network embedding models, discrete network embedding aims to compress model size and accelerate model inference by learning a set of short binary codes for network vertices. However, existing discrete network embedding methods usually assume that the network structures (e.g., edge weights) are readily available. In real-world scenarios such as social networks, sometimes it is impossible to collect explicit network structure information and it usually needs to be inferred from implicit data such as information cascades in the networks. To address this issue, we present an end-to-end discrete network embedding model for latent networks DELN that can learn binary representations from underlying information cascades. The essential idea is to infer a latent Weisfeiler-Lehman proximity matrix that captures node dependence based on information cascades and then to factorize the latent Weisfiler-Lehman matrix under the binary node representation constraint. Since the learning problem is a mixed integer optimization problem, an efficient maximal likelihood estimation based cyclic coordinate descent (MLE-CCD) algorithm is used as the solution. Experiments on real-world datasets show that the proposed model outperforms the state-of-the-art network embedding methods.

Most Visual Question Answering (VQA) models suffer from the language prior problem, which is caused by inherent data biases. Specifically, VQA models tend to answer questions (e.g., what color is the banana?) based on the high-frequency answers (e.g., yellow) ignoring image contents. Existing approaches tackle this problem by creating delicate models or introducing additional visual annotations to reduce question dependency and strengthen image dependency. However, they are still subject to the language prior problem since the data biases have not been fundamentally addressed. In this paper, we introduce a self-supervised learning framework to solve this problem. Concretely, we first automatically generate labeled data to balance the biased data, and then propose a self-supervised auxiliary task to utilize the balanced data to assist the VQA model to overcome language priors. Our method can compensate for the data biases by generating balanced data without introducing external annotations. Experimental results show that our method achieves state-of-the-art performance, improving the overall accuracy from 49.50% to 57.59% on the most commonly used benchmark VQA-CP v2. In other words, we can increase the performance of annotation-based methods by 16% without using external annotations. Our code is available on GitHub.

As a fundamental task in NLP, recognizing implicit discourse relations remains a challenging problem for years. One of the most important reasons is the limited amounts of annotated data. On one hand, most existing methods use multi-task methods to enlarge data. External datasets are often fully introduced to the training process which may lead to a negative transfer of noisy data. On the other hand, some previous researches on selecting data mostly focus on designing rules with heuristic methods. It should be difficult to cover all aspects of good samples. Another drawback is that data selection can hardly generally fit well in any other recognition model since it was specialized with a specific discourse relation classifier.In this paper, we propose a novel selective data enhanced model (SDE) based on reinforcement learning. Our model is a general framework, composed of two parts: 1) Discourse relation classifier is designed to identify relations, including multi-level representation module and relation recognition module. 2) Pseudo labeled data selector is designed to pick out data that can enhance the discourse relation classifier. We conduct joint learning alternately to optimize both of the classifier and the selector. Our model is able to expand data selectively. And classifier part can be replaced with any other complicated networks. To further exploit interact signals between arguments, we also present a multi-level representation based on BERT. Experiments show that our model achieves better performance than state-of-the-art methods.

Understanding the target audience for popular television series is valuable for online video platform to manage advertising sales, purchase video copyrights, and compete with other video service platforms. Existing studies in this domain generally focus on using data mining and machine learning techniques to recommend television series to individual users or predict the popularity of television series. Knowing only the popularity of television series may, however, limit our ability to answer more in-depth questions and develop more intelligent applications. In this paper, we develop a data-driven framework to model and predict audience competition patterns for popular online television series. Specifically, we first construct a sequence of dynamic competition networks of television series by mining the detailed viewership records. Then, we design the Dynamic Deep Network Factorization (DDNF), a hybrid modeling framework for predicting the future competition networks. Our framework adopts the deep neural network (DNN) and the knowledge-base (KB) embedding to incorporate static features, and integrates the Long Short-Term Memory (LSTM) network to learn dynamic features of the television series. Finally, extensive experiments on real-world data sets validate the effectiveness of our approach compared with state-of-the-art baselines in predicting the audience competition for existing and new television series.

The latent friend recommendation in online social media is interesting, yet challenging, because the user-item ratings and the user-user relationships are both sparse. In this paper, we propose a new dual implicit mining-based latent friend recommendation model that simultaneously considers the implicit interest topics of users and the implicit link relationships between the users in the local topic cliques. Specifically, we first propose an algorithm called all reviews from a user and all tags from their corresponding items to learn the implicit interest topics of the users and their corresponding topic weights, then compute the user interest topic similarity using a symmetric Jensen-Shannon divergence. After that, we adopt the proposed weighted local random walk with restart algorithm to analyze the implicit link relationships between the users in the local topic cliques and calculate the weighted link relationship similarity between the users. Combining the user interest topic similarity with the weighted link relationship similarity in a unified way, we get the final latent friend recommendation list. The experiments on real-world datasets demonstrate that the proposed method outperforms the state-of-the-art latent friend recommendation methods under four different types of evaluation metrics.

Attributed network embedding plays an important role in transferring network data into compact vectors for effective network analysis. Existing attributed network embedding models are designed either in continuous Euclidean spaces which introduce data redundancy or in binary coding spaces which incur significant loss of representation accuracy. To this end, we present a new Low-Bit Quantization for Attributed Network Representation Learning model (LQANR for short) that can learn compact node representations with low bitwidth values while preserving high representation accuracy. Specifically, we formulate a new representation learning function based on matrix factorization that can jointly learn the low-bit node representations and the layer aggregation weights under the low-bit quantization constraint. Because the new learning function falls into the category of mixed integer optimization, we propose an efficient mixed-integer based alternating direction method of multipliers (ADMM) algorithm as the solution. Experiments on real-world node classification and link prediction tasks validate the promising results of the proposed LQANR model.

In this paper, we present a novel sequence-to-sequence architecture with multi-head attention for automatic summarization of long text. Summaries generated by previous abstractive methods have the problems of duplicate and missing original information commonly. To address these problems, we propose a multi-head attention summarization (MHAS) model, which uses multi-head attention mechanism to learn relevant information in different representation subspaces. The MHAS model can consider the previously predicted words when generating new words to avoid generating a summary of redundant repetition words. And it can learn the internal structure of the article by adding self-attention layer to the traditional encoder and decoder and make the model better preserve the original information. We also integrate the multi-head attention distribution into pointer network creatively to improve the performance of the model. Experiments are conducted on CNN/Daily Mail dataset, which is a long text English corpora. Experimental results show that our proposed model outperforms the previous extractive and abstractive models.

Time series has been a popular research topic over the past decade. Salient subsequences of time series that can benefit the learning task, e.g., classification or clustering, are called shapelets. Shapelet-based time series learning extracts these types of salient subsequences with highly informative features from a time series. Most existing methods for shapelet discovery must scan a large pool of candidate subsequences, which is a time-consuming process. A recent work, [1] , uses regression learning to discover shapelets in a time series; however, it only considers learning shapelets from labeled time series data. This paper proposes an Unsupervised Salient Subsequence Learning (USSL) model that discovers shapelets without the effort of labeling. We developed this new learning function by integrating the strengths of shapelet learning, shapelet regularization, spectral analysis and pseudo-label to simultaneously and automatically learn shapelets to help clustering unlabeled time series better. The optimization model is iteratively solved via a coordinate descent algorithm. Experiments show that our USSL can learn meaningful shapelets, with promising results on real-world and synthetic data that surpass current state-of-the-art unsupervised time series learning methods.

This paper formulates the problem of learning discriminative features (i.e., segments) from networked time-series data, considering the linked information among time series. For example, social network users are considered to be social sensors that continuously generate social signals represented as a time series. The discriminative segments are often referred to as shapelets in a time series. Extracting shapelets for time-series analysis has been widely studied. However, existing works on shapelet selection assume that the time series are independent and identically distributed. This assumption restricts their applications to social networked time-series analysis since a user's actions can be correlated to his/her social affiliations. In this paper, we propose a novel network regularized least squares (NetRLS) feature selection model that combines typical time-series data and user network data for analysis. Experiments on real-world Twitter, Weibo, and DBLP networked time-series data demonstrate the performance of the proposed method. NetRLS performs better than the representative baselines on four evaluation criteria, namely classification accuracy, area under the curve (AUC), F1-score, and statistical significance analysis. NetRLS also has competitive running time as the baselines.

Attributed network embedding enables joint representation learning of node links and attributes. Existing attributed network embedding models are designed in continuous Euclidean spaces which often introduce data redundancy and impose challenges to storage and computation costs. To this end, we present a Binarized Attributed Network Embedding model (BANE for short) to learn binary node representation. Specifically, we define a new Weisfeiler-Lehman proximity matrix to capture data dependence between node links and attributes by aggregating the information of node attributes and links from neighboring nodes to a given target node in a layer-wise manner. Based on the Weisfeiler-Lehman proximity matrix, we formulate a new Weisfiler-Lehman matrix factorization learning function under the binary node representation constraint. The learning problem is a mixed integer optimization and an efficient cyclic coordinate descent (CCD) algorithm is used as the solution. Node classification and link prediction experiments on real-world datasets show that the proposed BANE model outperforms the state-of-the-art network embedding methods.

How to solve the classification problem with only label proportions has recently drawn increasing attention in the machine learning field. In this paper, we propose an ensemble learning strategy to deal with the learning problem with label proportions (LLP). In detail, we first give a loss function based on different weights for LLP, and then construct the corresponding weak classifier, at the same time, estimate its conditional probabilities by a standard logistic function. At last, by introducing the maximum likelihood estimation, we propose a new anyboost learning system for LLP (called Adaboost-LLP). Unlike traditional methods, our method does not make any restrictive assumptions on training set; at the same time, compared with alter- SVM, Adaboost-LLP exploits more extra weight information and uses multiple weak classifiers that can be solved efficiently to combine a strong classifier. All experiments show that our method outperforms the existing methods in both accuracy and training time.

Feature selection aims to select a subset of features from high-dimensional data according to a predefined selecting criterion. Sparse learning has been proven to be a powerful technique in feature selection. Sparse regularizer, as a key component of sparse learning, has been studied for several years. Although convex regularizers have been used in many works, there are some cases where nonconvex regularizers outperform convex regularizers. To make the process of selecting relevant features more effective, we propose a novel nonconvex sparse metric on matrices as the sparsity regularization in this paper. The new nonconvex regularizer could be written as the difference of the $\ell _{2,1}$ norm and the Frobenius ( $\ell _{2,2}$ ) norm, which is named the $\ell _{2,1-2}$ . To find the solution of the resulting nonconvex formula, we design an iterative algorithm in the framework of ConCave-Convex Procedure (CCCP) and prove its strong global convergence. An adopted alternating direction method of multipliers is embedded to solve the sequence of convex subproblems in CCCP efficiently. Using the scaled cluster indictors of data points as pseudolabels, we also apply $\ell _{2,1-2}$ to the unsupervised case. To the best of our knowledge, it is the first work considering nonconvex regularization for matrices in the unsupervised learning scenario. Numerical experiments are performed on real-world data sets to demonstrate the effectiveness of the proposed method.

Multiview learning (MVL), by exploiting the complementary information among multiple feature sets, can improve the performance of many existing learning tasks. Support vector machine (SVM)-based models have been frequently used for MVL. A typical SVM-based MVL model is SVM-2K, which extends SVM for MVL by using the distance minimization version of kernel canonical correlation analysis. However, SVM-2K cannot fully unleash the power of the complementary information among different feature views. Recently, a framework of learning using privileged information (LUPI) has been proposed to model data with complementary information. Motivated by LUPI, we propose a new multiview privileged SVM model, multi-view privileged SVM model (PSVM-2V), for MVL. This brings a new perspective that extends LUPI to MVL. The optimization of PSVM-2V can be solved by the classical quadratic programming solver. We theoretically analyze the performance of PSVM-2V from the viewpoints of the consensus principle, the generalization error bound, and the SVM-2K learning model. Experimental results on 95 binary data sets demonstrate the effectiveness of the proposed method.

With the Internet becoming the dominant channel for business and life, web proxies are also increasingly used for illegal purposes such as propagating malware, impersonate phishing pages to steal sensitive data or redirect victims to other malicious targets. In this paper, using thousands of web proxy URLs crawled, we performed a large-scale study on the DOM (Document Object Model) structure features. Our study reveals the existence of the dedicated web proxy DOM among hosts that play orchestrating roles in proxy activities. Motivated by their distinctive features in DOM and URL, we developed an automatical stepping-stone detection system-ProxyDetector. Specially, we explored the potential benefits of considering DOM-based features, which improved 25% recall rate than before. We extensively evaluated ProxyDetector with four methods on a diverse spectrum of corpora with 2,068 web proxy sites and 26,066 legitimate sites. Capable of achieving over 95% precision of web proxy sites with a high recall rate of 96.5% on average, our ProxyDetector has been demonstrated to be an effective solution of detecting the web proxy sites.

WiFi fingerprint-based localization is regarded as one of the most promising techniques for indoor localization. However, this raises serious privacy concerns. Current approaches to mitigate the privacy concerns rely on the encryption with large calculation consumption. In this paper, we propose a data obfuscation mechanism based on the generalized version of differential privacy. We extend the standard definition to the indoor WiFi fingerprint data for spatial counting where the inputs belong to multiple dimensions of numerical data in a limited range. With a given privacy budget, the proposed method generalizes the original dataset, and then specializes it using differential privacy. As the designed novel scheme expand the range for specialization, the data set released by the proposed algorithm can yield better mining results. Furthermore, experimental results give out comparisons between nonuniform and uniform ε selection scheme, and find uniform ε selection scheme can fully use the privacy budget in our situation.

In this paper we study a new problem of online discovering diffusion provenances in large networks. Existing work on network diffusion provenance identification focuses on offline learning where data collected from network detectors are static and a snapshot of the network is available before learning. However, an offline learning model does not meet the need for early warning, real-time awareness, or a real-time response to malicious information spreading in networks. To this end, we propose an online regression model for real-time diffusion provenance identification. Specifically, we first use offline collected network cascades to infer the edge transmission weights, and then use an online l 1 non-convex regression model as the identification model. The proposed methods are empirically evaluated on both synthetic and real-world networks. Experimental results demonstrate the effectiveness of the proposed model.

Clustering has been widely used in data analysis. A majority of existing clustering approaches assume that the number of clusters is given in advance. Recently, a novel clustering framework is proposed which can automatically learn the number of clusters from training data. Based on these works, we propose a nonsmooth penalized clustering model via ℓp(0p-norm-based regularization to control the tradeoff between the model fit and the number of clusters. We theoretically prove that the new model can guarantee the sparseness of cluster centers. To increase its practicality for practical use, we adhere to an easy-to-compute criterion and follow a strategy to narrow down the search interval of cross validation. To address the nonsmoothness and nonconvexness of the cost function, we propose a simple smoothing trust region algorithm and present its convergent and computational complexity analysis. Numerical studies on both simulated and practical data sets provide support to our theoretical results and demonstrate the advantages of our new method.

Graph classification is an important tool for analyzing data with structure dependency, where subgraphs are often used as features for learning. In reality, the dimension of the subgraphs crucially depends on the threshold setting of the frequency support parameter, and the number may become extremely large. As a result, subgraphs may be incrementally discovered to form a feature stream and require the underlying graph classifier to effectively discover representative subgraph features from the subgraph feature stream. In this paper, we propose a primal-dual incremental subgraph feature selection algorithm (ISF) based on a max-margin graph classifier. The ISF algorithm constructs a sequence of solutions that are both primal and dual feasible. Each primal-dual pair shrinks the dual gap and renders a better solution for the optimal subgraph feature set. To avoid bias of ISF algorithm on short-pattern subgraph features, we present a new incremental subgraph join feature selection algorithm (ISJF) by forcing graph classifiers to join short-pattern subgraphs and generate long-pattern subgraph features. We evaluate the performance of the proposed models on both synthetic networks and real-world social network data sets. Experimental results demonstrate the effectiveness of the proposed methods.

Multi-instance learning (MIL) is useful for tackling labeling ambiguity in learning tasks, by allowing a bag of instances to share one label. Recently, bag mapping methods, which transform a bag to a single instance in a new space via instance selection, have drawn significant attentions. To date, most existing works are developed based on the original space, i.e., utilizing all instances for bag mapping, and instance selection is indirectly tied to the MIL objective. As a result, it is hard to guarantee the distinguish capacity of the selected instances in the new bag mapping space for MIL. In this paper, we propose a direct discriminative mapping approach for multi-instance learning (MILDM), which identifies instances to directly distinguish bags in the new mapping space. Experiments and comparisons on real-world learning tasks demonstrate the algorithm performance.

In this paper we theoretically study the minimum Differentially Resolving Set (DRS) problem derived from the classical sensor placement optimization problem in network source locating. A DRS of a graph G = (V, E) is defined as a subset S ⊆ V where any two elements in V can be distinguished by their different differential characteristic sets defined on S. The minimum DRS problem aims to find a DRS S in the graph G with minimum total weight Σv∈S w(v). In this paper we establish a group of Integer Linear Programming (ILP) models as the solution. By the weighted set cover theory, we propose an approximation algorithm with the Θ(ln n) approximability for the minimum DRS problem on general graphs, where n is the graph size.

Network traces are one of the most exhaustive data sources for the forensic investigation of computer security incidents. Recent advances in capturing the network traces techniques have facilitated the forensic processing, including the reconstruction. Unfortunately, off-line Web-downloading chain reconstruction could not meet the demands for real time processing. Furthermore, the packets in prior studies are mainly captured on the client or server side, which is difficult to monitor the network traffic in the real-time. Consequently, how to online reconstruct from the packets captured by the gateway is getting challenging. In this paper, based on the packets from the gateway, we propose a novel system, CookieMiner. CookieMiner first identifies the web-downloading resources, and then use the cookies to reconstruct the web-downloading chains reversely. All cookies would be firstly split into a series of tokens by semicolon and a threshold value will be set by the SET_K algorithm according to the number of tokens. Next, the HTTP packets whose tokens' number is greater than the threshold will be sorted by timestamp and then inserted into the corresponding web-downloading chain. Finally, the most frequent URLs are extracted as entry points from all the chains with the same web-downloading resources. In addition, by a user study involving 6 pair-wise downloading applications, CookieMiner can reconstruct the web-downloading chains and find their entry points with high precision and low false positive rate.

Social recommendation explores social information to improve the quality of a recommender system. It can be further divided into explicit and implicit social network recommendation. The former assumes the existence of explicit social connections between users in addition to the rating data. The latter one assumes the availability of only the ratings but not the social connections between users since the explicit social information data may not necessarily be available and usually are binary decision values (e.g., whether two people are friends), while the strength of their relationships is missing. Most of the works in this field use only rating data to infer the latent social networks. They ignore the dynamic nature of users that the preferences of users drift over time distinctly. To this end, we propose a new Implicit Dynamic Social Recommendation (IDSR) model, which infers latent social network from cascade data. It can sufficiently mine the information contained in time by mining the cascade data and identify the dynamic changes in the users in time by using the latest updated social network to make recommendations. Experiments and comparisons on three real-world datasets show that the proposed model outperforms the state-of-the-art solutions in both explicit and implicit scenarios.

In this paper, we study a new problem of continuous learning from doubly-streaming data where both data volume and feature space increase over time. We refer to the doubly-streaming data as trapezoidal data streams and the corresponding learning problem as online learning from trapezoidal data streams. The problem is challenging because both data volume and data dimension increase over time, and existing online learning[1],[2], online feature selection[3], and streaming feature selection algorithms[4],[5]are inapplicable. We propose a new Online Learning with Streaming Features algorithm (OL$_{SF}$for short) and its two variants, which combine online learning[1],[2]and streaming feature selection[4],[5]to enable learning from trapezoidal data streams with infinite training instances and features. When a new training instance carrying new features arrives, a classifier updates the existing features by following the passive-aggressive update rule[2]and updates the new features by following the structural risk minimization principle. Feature sparsity is then introduced by using the projected truncation technique. We derive performance bounds of the OL$_{SF}$algorithm and its variants. We also conduct experiments on real-world data sets to show the performance of the proposed algorithms.

Witnessing the wide spread of malicious information in large networks, we develop an efficient method to detect anomalous diffusion sources and thus protect networks from security and privacy attacks. To date, most existing work on diffusion sources detection are based on the assumption that network snapshots that reflect information diffusion can be obtained continuously. However, obtaining snapshots of an entire network needs to deploy detectors on all network nodes and thus is very expensive. Alternatively, in this article, we study the diffusion sources locating problem by learning from information diffusion data collected from only a small subset of network nodes. Specifically, we present a new regression learning model that can detect anomalous diffusion sources by jointly solving five challenges, that is, unknown number of source nodes, few activated detectors, unknown initial propagation time, uncertain propagation path and uncertain propagation time delay. We theoretically analyze the strength of the model and derive performance bounds. We empirically test and compare the model using both synthetic and real-world networks to demonstrate its performance.

Graph classification is an important tool for analysing structured and semi-structured data, where subgraphs are commonly used as the feature representation. However, the number and size of subgraph features crucially depend on the threshold parameters of frequent subgraph mining algorithms. Any improper setting of the parameters will generate many trivial short-pattern subgraph fragments which dominate the feature space, distort graph classifiers and bury interesting long-pattern subgraphs. In this paper, we propose a new Subgraph Join Feature Selection (SJFS) algorithm. The SJFS algorithm, by forcing graph classifiers to join short-pattern subgraph fragments, can defrag trivial subgraph features and deliver long-pattern interesting subgraphs. Experimental results on both synthetic and real-world social network graph data demonstrate the performance of the proposed method.

The problem of modeling topics on user behavior data in social networks has been widely studied in social marketing and social emotion analysis, where latent topic models are commonly used as the solutions. The user behavior data are highly related in time and space, which demands new latent topic models that consider both temporal and spatial distances. However, existing topic models either fail to model these two factors simultaneously, or cannot handle the high order dependence among user behaviors. In this paper we present a new nonparametric Bayesian model Time and Space Dependent Chinese Restaurant Processes (TSD-CRP for short). TSD-CRP can auto-select the number of topics and model high-order temporal and spatial dependence behind user behavior data. Empirical results on real-world data sets demonstrate the effectiveness of the proposed method.

We study a new problem of learning from doubly-streaming data where both data volume and feature space increase over time. We refer to the problem as mining trapezoidal data streams. The problem is challenging because both data volume and feature space are increasing, to which existing online learning, online feature selection and streaming feature selection algorithms are inapplicable. We propose a new Sparse Trapezoidal Streaming Data mining algorithm (STSD) and its two variants which combine online learning and online feature selection to enable learning trapezoidal data streams with infinite training instances and features. Specifically, when new training instances carrying new features arrive, the classifier updates the existing features by following the passive-aggressive update rule used in online learning and updates the new features with the structural risk minimization principle. Feature sparsity is also introduced using the projected truncation techniques. Extensive experiments on the demonstrated UCI data sets show the performance of the proposed algorithms.

In this paper we study a new problem of online diffusion source detection in social networks. Existing work on diffusion source detection focuses on offline learning, which assumes data collected from network detectors are static and a snapshot of network is available before learning. However, an offline learning model does not meet the needs of early warning, real-time awareness, and real-time response of malicious information spreading in social networks. In this paper, we combine online learning and regression-based detection methods for real-time diffusion source detection. Specifically, we propose a new ℓ1non-convex regression model as the learning function, and an Online Stochastic Sub-gradient algorithm (OSS for short). The proposed model is empirically evaluated on both synthetic and real-world networks. Experimental results demonstrate the effectiveness of the proposed model.

Ensemble learning is a common tool for data stream classification, mainly because of its inherent advantages of handling large volumes of stream data and concept drifting. Previous studies, to date, have been primarily focused on building accurate ensemble models from stream data. However, a linear scan of a large number of base classifiers in the ensemble during prediction incurs significant costs in response time, preventing ensemble learning from being practical for many real-world time-critical data stream applications, such as Web traffic stream monitoring, spam detection, and intrusion detection. In these applications, data streams usually arrive at a speed of GB/second, and it is necessary to classify each stream record in a timely manner. To address this problem, we propose a novel Ensemble-tree (E-tree for short) indexing structure to organize all base classifiers in an ensemble for fast prediction. On one hand, E-trees treat ensembles as spatial databases and employ an R-tree like height-balanced structure to reduce the expected prediction time from linear to sub-linear complexity. On the other hand, E-trees can be automatically updated by continuously integrating new classifiers and discarding outdated ones, well adapting to new trends and patterns underneath data streams. Theoretical analysis and empirical studies on both synthetic and real-world data streams demonstrate the performance of our approach.

Influence maximization, defined as finding a small subset of nodes that maximizes spread of influence in social networks, is NP-hard under both Independent Cascade (IC) and Linear Threshold (LT) models, where many greedy-based algorithms have been proposed with the best approximation guarantee. However, existing greedy-based algorithms are inefficient on large networks, as it demands heavy Monte-Carlo simulations of the spread functions for each node at the initial step [7]. In this paper, we establish new upper bounds to significantly reduce the number of Monte-Carlo simulations in greedy-based algorithms, especially at the initial step. We theoretically prove that the bound is tight and convergent when the summation of weights towards (or from) each node is less than 1. Based on the bound, we propose a new Upper Bound based Lazy Forward algorithm (UBLF in short) for discovering the top-k influential nodes in social networks. We test and compare UBLF with prior greedy algorithms, especially CELF [30]. Experimental results show that UBLF reduces more than 95 percent Monte-Carlo simulations of CELF and achieves about 2-10 times speedup when the seed set is small.

With the rapid growth of event-based social networks (EBSNs) like Meetup, the demand for event recommendation becomes increasingly urgent. In EBSNs, event recommendation plays a central role in recommending the most relevant events to users who are likely to participate in. Different from traditional recommendation problems, event recommendation encounters three new types of information, i.e., heterogenous online+offline social relationships, geographical features of events and implicit rating data from users. Yet combining the three types of data for offline event recommendation has not been considered. Therefore, we present a Bayesian latent factor model that can unify these data for event recommendation. Experimental results on real-world data sets show the performance of our method.

With the rapid growth of event-based social networks, the demand of event recommendation becomes increasingly important. Different from classic recommendation problems, event recommendation generally faces the problems of heterogenous online and offline social relationships among users and implicit feedback data. In this paper, we present a baysian probability model that can fully unleash the power of heterogenous social relations and efficiently tackle with implicit feedback characteristic for event recommendation. Experimental results on several real-world datasets demonstrate the utility of our method.

Max-margin matrix factorization (M3F) has been popularly applied to collaborative filtering for personalized recommendations. The nonparametric M3F model represents the latest progress of the M3F methods, which can auto-select the number of factors by using nonparametric techniques. However, existing non-parametric M3F methods assume a collection of user rating data can be fully obtained before training, and they are inapplicable for on-the-fly recommender systems where user rating data arrive continuously. In this paper, we present a new efficient online nonparametric 3F model for flexible recommendation. Specifically, we design an online nonparametric M3F model (OnM3F) based on the online Passive-Aggressive learning and solve the corresponding optimization problem by using the online stochastic gradient descent. Empirical studies on two large real-world data sets verify the effectiveness of the proposed method.

This paper investigates the cooperative beamforming in the amplify-and-forward-based cognitive relay network. Beamforming coefficients are designed to maximize the sum rate of the secondary users with a total transmit power constraint of the relays and an interference power constraint at the primary receiver. The sum rate maximization problem is firstly solved by characterizing the achievable rate region. This method is optimal but of great complexity. Then, an iteration algorithm based on semidefinite programming (SDP) and the first order Taylor polynomial which has lower complexity but comparable performance is introduced. Simulation results confirm the feasibility of these two methods for the sum rate maximization problem in the cognitive relay network and demonstrate that the lower complexity method finds the global optimum of the problem just as the rate region method does.

In this paper, we propose a filter-and-forward (FF)beamforming strategy in the secondary relays for the one-way cognitive relay network with frequency selective fading channels. Our aim is to optimize the beamforming coefficients and the secondary transmitted power to maximize the SINR at the secondary destination, while satisfying the interference constraint on the primary network and the total relays' transmitted power restriction. To solve this problem, we propose a two-step iterative algorithm. Simulation results show that comparing the amplifyand-forward (AF) strategy, the filter-and-forward strategy (FF) can get higher SINR at the secondary destination by using the algorithm that we proposed.

Detecting outliers from big data plays an important role in network security. Previous outlier detection algorithms are generally incapable of handling big data. In this paper we present an parallel outlier detection method for big data, based on a new parallel auto-encoder method. Specifically, we build a replicator model of the input data to obtain the representation of sample data. Then, the replicator model is used to measure the replicability of test data, where records having higher reconstruction errors are classified as outliers. Experimental results show the performance of the proposed parallel algorithm.

In this paper, we study a new problem on social network influence maximization. The problem is defined as, given a target user $w$, finding the top-k most influential nodes for the user. Different from existing influence maximization works which aim to find a small subset of nodes to maximize the spread of influence over the entire network (i.e., global optima), our problem aims to find a small subset of nodes which can maximize the influence spread to a given target user (i.e., local optima). The solution is critical for personalized services on social networks, where fully understanding of each specific user is essential. Although some global influence maximization models can be narrowed down as the solution, these methods often bias to the target node itself. To this end, in this paper we present a local influence maximization solution. We first provide a random function, with low variance guarantee, to randomly simulate the objective function of local influence maximization. Then, we present efficient algorithms with approximation guarantee. For online social network applications, we also present a scalable approximate algorithm by exploring the local cascade structure of the target user. We test the proposed algorithms on several real-world social networks. Experimental results validate the performance of the proposed algorithms.

Influence maximization, defined as finding a small subset of nodes that maximizes spread of influence in social networks, is NP-hard under both Linear Threshold (LT) and Independent Cascade (IC) models, where a line of greedy/heuristic algorithms have been proposed. The simple greedy algorithm [14] achieves an approximation ratio of 1-1/e. The advanced CELF algorithm [16], by exploiting the sub modular property of the spread function, runs 700 times faster than the simple greedy algorithm on average. However, CELF is still inefficient [4], as the first iteration calls for N times of spread estimations (N is the number of nodes in networks), which is computationally expensive especially for large networks. To this end, in this paper we derive an upper bound function for the spread function. The bound can be used to reduce the number of Monte-Carlo simulation calls in greedy algorithms, especially in the first iteration of initialization. Based on the upper bound, we propose an efficient Upper Bound based Lazy Forward algorithm (UBLF in short), by incorporating the bound into the CELF algorithm. We test and compare our algorithm with prior algorithms on real-world data sets. Experimental results demonstrate that UBLF, compared with CELF, reduces more than 95% Monte-Carlo simulations and achieves at least 2-5 times speed-raising when the seed set is small.

Behavioral targeting (BT) is a valuable tool for online advertising. In this paper, we study a new problem of incorporating social information into traditional behavior targeting models. Specifically, we present a social regularization based Poisson regression framework for behavior targeting. Based on the observation that social information can be diverse and competing, we furthermore present two specific social regularization terms: the average-based social regularization term and the individual-based social regularization term. To validate the effectiveness of the proposed models, we use the KDDCUP'12 behavior targeting data, issued by the Tecent company in China, as the test bed. The results demonstrate that the proposed models, by incorporating additional social network information, can achieve at least 5% improvement compared to the traditional Poisson regression based model from the CTR lift viewpoint, especially when the historical behavior data is sparse and insufficient.

With the rapid proliferation of diverse online social network sites, user recommendation has been received unprecedented attention. At present, the methods for user recommendation are mainly divided into two categories: recommending a new friend for a target user according to similar interest, or by friendships similarity between the two users. The first category methods have high recall but low precision, the second methods have high precision but low recall. In this paper, we proposed a new hybrid approach by incorporating users' interests and users' friendships together to recommend new friends for target users. Firstly, we use latent Dirichlet allocation (LDA) to model users' interests, and Weighted-PageRank Algorithm to model users' friendship network, and then merge these two factors into a hybrid model based on PageRank algorithm. This hybrid method models users' interests and users' friendships at the same time, and we demonstrate the effectiveness of our hybrid model by using some social network datasets.

Behavior targeting (BT) is a promising tool for online advertising. The state-of-the-art BT methods, which are mainly based on regression models, have two limitations. First, learning regression models for behavior targeting is difficult since user clicks are typically several orders of magnitude fewer than views. Second, the user interests are not fixed, but often transient and influenced by media and pop culture. In this paper, we propose to formulate behavior targeting as a classification problem. Specifically, we propose to use an SVM ensemble for behavior prediction. The challenge of using ensemble SVM for BT stems from the computational complexity (it takes 53 minutes in our experiments to predict behavior for 32 million users, which is inadequate for online application). To this end, we propose a fast ensemble SVM prediction framework, which builds an indexing structure for SVM ensemble to achieve sub-linear prediction time complexity. Experimental results on real-world large scale behavior targeting data demonstrate that the proposed method is efficient and outperforms existing linear regression based BT models.

Data stream classification has drawn increasing attention from the data mining community in recent years, where a large number of stream classification models were proposed. However, most existing models were merely focused on mining from single-label data streams. Mining from multi-label data streams has not been fully addressed yet. On the other hand, although some recent work touched the multi-label stream mining problem, they never consider the expensive labeling cost issue, preventing them from real-world applications. To this end, we study, in this paper, a challenging problem that mining from multi-label data streams with limited labeling resource. Specifically, we propose an ensemble-based active learning framework to handle the large volume of stream data, expensive labeling cost and concept drifting problems on multi-label data streams. Experiments on both synthetic and real world data sets demonstrate the performance of the proposed method.

Mining frequent patterns from data streams has drawn increasing attention in recent years. However, previous mining algorithms were all focused on a single data stream. In many emerging applications, it is of critical importance to combine multiple data streams for analysis. For example, in real-time news topic analysis, it is necessary to combine multiple news report streams from dierent media sources to discover collaborative frequent patterns which are reported frequently in all media, and comparative frequent patterns which are reported more frequently in a media than others. To address this problem, we propose a novel frequent pattern mining algorithm Hybrid-Streaming, H-Stream for short. H-Stream builds a new Hybrid-Frequent tree to maintain historical frequent and potential frequent itemsets from all data streams, and incrementally updates these itemsets for efficient collaborative and comparative pattern mining. Theoretical and empirical studies demonstrate the utility of the proposed method.

Cloud computing represents one of the most important research directions for modern computing systems. Existing research efforts on Cloud computing were all focused on designing advanced storage and query techniques for static data. None of them consider the problem that data in a Cloud may appear as continuous and rapid data streams. To address this problem, in this paper we propose a new LCN-Index framework to handle continuous data stream queries in the Cloud. LCN-Index uses the Map-Reduce computing paradigm to process all the queries. In the Mapping stage, it divides all the queries into a batch of predicate sets which are then deployed onto mapping nodes using interval predicate index. In the reducing stage, it merges results from the mapping nodes using multi attribute hash index. In so doing, a data stream can be efficiently evaluated by traversing through the LCN-Index framework. Experiments demonstrate the utility of the proposed method.

Lazy learning, such as k-nearest neighbor learning, has been widely applied to many applications. Known for well capturing data locality, lazy learning can be advantageous for highly dynamic and complex learning environments such as data streams. Yet its high memory consumption and low prediction efficiency have made it less favorable for stream oriented applications. Specifically, traditional lazy learning stores all the training data and the inductive process is deferred until a query appears, whereas in stream applications, data records flow continuously in large volumes and the prediction of class labels needs to be made in a timely manner. In this paper, we provide a systematic solution that overcomes the memory and efficiency limitations and enables fast lazy learning for concept drifting data streams. In particular, we propose a novel Lazy-tree (Ltree for short) indexing structure that dynamically maintains compact high-level summaries of historical stream records. L-trees are M-Tree [5] like, height-balanced, and can help achieve great memory consumption reduction and sub-linear time complexity for prediction. Moreover, L-trees continuously absorb new stream records and discard outdated ones, so they can naturally adapt to the dynamically changing concepts in data streams for accurate prediction. Extensive experiments on real-world and synthetic data streams demonstrate the performance of our approach.

Multimedia stream monitoring refers to removing unwanted and malicious records from multimedia streams. In this application, a large number of filtering queries are registered on time-critical multimedia streams. Each filtering query contains multiple meta filters and a meta filter is shared among multiple filtering queries. The filtering queries and meta filters form a bipartite graph, and the objective is to minimize the overall evaluation time of the queries in the bipartite graph. In order to achieve this goal, some heuristic algorithms were proposed to order the shared meta filters in the graph to reduce the overall evaluation cost. While these methods can achieve near-optimal solutions in ideal stream environments that have stationary probability distributions, in this paper we propose an Adaptive Shared-filter Ordering Model (ASOM) for efficient filtering in dynamic data stream environments. To capture new trends and patterns along dynamic data streams, ASOM uses a time-based exponential smoothing forecasting method to adaptively order the shared meta filters for fast estimation. Experiments demonstrate that ASOM outperforms existing heuristic ordering methods in dynamic stream environments.

Ensemble learning has become a common tool for data stream classification, being able to handle large volumes of stream data and concept drifting. Previous studies focus on building accurate prediction models from stream data. However, a linear scan of a large number of base classifiers in the ensemble during prediction incurs significant costs in response time, preventing ensemble learning from being practical for many real world time-critical data stream applications, such as Web traffic stream monitoring, spam detection, and intrusion detection. In these applications, data streams usually arrive at a speed of GB/second, and it is necessary to classify each stream record in a timely manner. To address this problem, we propose a novel Ensemble-tree (E-tree for short) indexing structure to organize all base classifiers in an ensemble for fast prediction. On one hand, E-trees treat ensembles as spatial databases and employ an R-tree like height-balanced structure to reduce the expected prediction time from linear to sub-linear complexity. On the other hand, E-trees can automatically update themselves by continuously integrating new classifiers and discarding outdated ones, well adapting to new trends and patterns underneath data streams. Experiments on both synthetic and real-world data streams demonstrate the performance of our approach.

Missing data commonly occurs in many applications. While many data imputation methods exist to handle the missing data problem for large scale databases, when applied to concept drifting data streams, these methods face some common difficulties. First, due to large and continuous data volumes, we are unable to maintain all stream records to form a candidate pool and estimate missing values, as most existing methods commonly do. Second, even if we could maintain all complete stream records using a summary structure, the concept drifting problem would make some information obsolete, and thus deteriorate the imputation accuracy. Third, in data streams, it is necessary to develop a fast yet accurate algorithm to find the most similar data for imputation. Fourth, due to the dynamic and sophisticated data collection environments, the missing rate of most stream data may be much higher than that in generic static databases, so the imputation method should be able to accommodate high missing rate in the data. To tackle these challenges, we propose, in this paper, a Streaming k-Nearest-Neighbors Imputation Framework (SKIF) for concept drifting data streams. To handle concept drifting and large volume problems in data streams, SKIF first summarizes historical complete records in some micro-resources (which are high-level statistical data structures), and maintains these micro-resources in a candidate pool as benchmark data. After that, SKIF employs a novel hybrid-kNN imputation procedure, which uses a hybrid similarity search mechanism, to find the most similar micro-resources from the large scale candidate pool efficiently. Experimental results demonstrate the effectiveness of the proposed SKIF framework for data stream imputation tasks.

Ensemble learning is a commonly used tool for building prediction models from data streams, due to its intrinsic merits of handling large volumes stream data. Despite of its extraordinary successes in stream data mining, existing ensemble models, in stream data environments, mainly fall into the ensemble classifiers category, without realizing that building classifiers requires labor intensive labeling process, and it is often the case that we may have a small number of labeled samples to train a few classifiers, but a large number of unlabeled samples are available to build clusters from data streams. Accordingly, in this paper, we propose a new ensemble model which combines both classifiers and clusters together for mining data streams. We argue that the main challenges of this new ensemble model include (1) clusters formulated from data streams only carry cluster IDs, with no genuine class label information, and (2) concept drifting underlying data streams makes it even harder to combine clusters and classifiers into one ensemble framework. To handle challenge (1), we present a label propagation method to infer each cluster's class label by making full use of both class label information from classifiers, and internal structure information from clusters. To handle challenge (2), we present a new weighting schema to weight all base models according to their consistencies with the up-to-date base model. As a result, all classifiers and clusters can be combined together, through a weighted average mechanism, for prediction. Experiments on real-world data streams demonstrate that our method outperforms simple classifier ensemble and cluster ensemble for stream data mining.

Related data streams refer to data streams that can be joined together by matching their join attributes. Existing research on learning from related data streams is based on an assumption that all streams arrive at a central processing unit in a synchronous way, such that in an arbitrary sliding window, all tuples of the streams can be perfectly joined together. This assumption, however, does not hold when related data streams are generated or transferred at different speeds, and thus may arrive in the central processing unit in an asynchronous manner. In this paper, we argue that for asynchronous data streams, there exist a small portion of perfectly joined examples (i.e., complete examples) and a large portion of partially joined examples (i.e., incomplete examples). Accordingly, we present a new Learning from Complete and Fixed Examples (LCFE) framework that can fix incomplete examples to boost the learning. Experiments on both synthetic and real-world data streams demonstrate that LCFE is able to achieve a higher prediction accuracy for learning from related data streams than other simple solutions can offer.

In this paper, we propose a new research problem on active learning from data streams, where data volumes grow continuously, and labeling all data is considered expensive and impractical. The objective is to label a small portion of stream data from which a model is derived to predict future instances as accurately as possible. To tackle the technical challenges raised by the dynamic nature of the stream data, i.e., increasing data volumes and evolving decision concepts, we propose a classifier-ensemble-based active learning framework that selectively labels instances from data streams to build a classifier ensemble. We argue that a classifier ensemble's variance directly corresponds to its error rate, and reducing a classifier ensemble's variance is equivalent to improving its prediction accuracy. Because of this, one should label instances toward the minimization of the variance of the underlying classifier ensemble. Accordingly, we introduce a minimum-variance (MV) principle to guide the instance labeling process for data streams. In addition, we derive an optimal-weight calculation method to determine the weight values for the classifier ensemble. The MV principle and the optimal weighting module are combined to build an active learning framework for data streams. Experimental results on synthetic and real-world data demonstrate the performance of the proposed work in comparison with other approaches.

Excessive lose of customer account is becoming a major headache for VIP E-Mail hosting companies. Analysis of what kind of customer is more prone to lose and finding the appropriate measures to sustain those customers has become urgent needs. Recentl

In this paper, we propose a framework to build prediction models from data streams which contain both labeled and unlabeled examples. We argue that due to the increasing data collection ability but limited resources for labeling, stream data collected at hand may only have a small number of labeled examples, whereas a large portion of data remain unlabeled but can be beneficial for learning. Unleashing the full potential of the unlabeled instances for stream data mining is, however, a significant challenge, consider that even fully labeled data streams may suffer from the concept drifting, and inappropriate uses of the unlabeled samples may only make the problem even worse. To build prediction models, we first categorize the stream data into four different categories, each of which corresponds to the situation where concept drifting may or may not exist in the labeled and unlabeled data. After that, we propose a relational k-means based transfer semi-supervised SVM learning framework (RK-TS3VM), which intends to leverage labeled and unlabeled samples to build prediction models. Experimental results and comparisons on both synthetic and real-world data streams demonstrate that the proposed framework is able to help build prediction models more accurate than other simple approaches can offer.

A new kernel-based learning algorithm called kernel affine subspace nearest point (KASNP) approach is proposed in this paper. Inspired by the geometrical explanation of support vector machines (SVMs) and its nearest point problem in convex hulls, we extend the convex hull of each class to its corresponding affine subspace in high dimensional space induced by kernel. In two class affine subspaces, KASNP finds the nearest points and then constructs a separating hyperplane, which bisects the line segment joining them. The nearest point problem of KASNP is only an unconstrained optimal problem whose solution can be directly computed. Compared with SVM, KASNP avoids solving convex quadratic programming. Experiments on two-spiral dataset, two UCI credit datasets, and face recognition datasets show that our proposed KASNP is effective for data classification.

In this paper, we identify a new research problem on cleansing noisy data streams which contain incorrectly labeled training examples. The objective is to accurately identify and remove mislabeled data, such that the prediction models built from the cleansed streams can be more accurate than the ones trained from the raw noisy streams. For this purpose, we first use bias-variance decomposition to derive a maximum variance margin (MVM) principle for stream data cleansing. Following this principle, we further propose a local and global filtering (LgF) framework to combine the strength of local noise filtering (within one single data chunk) and global noise filtering (across a number of adjacent data chunks) to identify erroneous data. Experimental results on six data streams (including two real-world data streams) demonstrate that LgF significantly outperforms simple methods in identifying noisy examples.

Mining concept drifting data streams is a defining challenge for data mining research. Recent years have seen a large body of work on detecting changes and building prediction models from stream data, with a vague understanding on the types of the concept drifting and the impact of different types of concept drifting on the mining algorithms. In this paper, we first categorize concept drifting into two scenarios: Loose Concept Drifting (LCD) and Rigorous Concept Drifting (RCD), and then propose solutions to handle each of them separately. For LCD data streams, because concepts in adjacent data chunks are sufficiently close to each other, we apply kernel mean matching (KMM) method to minimize the discrepancy of the data chunks in the kernel space. Such a minimization process will produce weighted instances to build classifier ensemble and handle concept drifting data streams. For RCD data streams, because genuine concepts in adjacent data chunks may randomly and rapidly change, we propose a new Optimal Weights Adjustment (OWA) method to determine the optimum weight values for classifiers trained from the most recent (up-to-date) data chunk, such that those classifiers can form an accurate classifier ensemble to predict instances in the yet-to-come data chunk. Experiments on synthetic and real-world datasets will show that weighted instance approach is preferable when the concept drifting is mainly caused by the changing of the class prior probability; whereas the weighted classifier approach is preferable when the concept drifting is mainly triggered by the changing of the conditional probability.

In this paper, we address a new research problem on active learning from data streams where data volumes grow continuously and labeling all data is considered expensive and impractical. The objective is to label a small portion of stream data from which a model is derived to predict newly arrived instances as accurate as possible. In order to tackle the challenges raised by data streams' dynamic nature, we propose a classifier ensembling based active learning framework which selectively labels instances from data streams to build an accurate classifier. A minimal variance principle is introduced to guide instance labeling from data streams. In addition, a weight updating rule is derived to ensure that our instance labeling process can adaptively adjust to dynamic drifting concepts in the data. Experimental results on synthetic and real-world data demonstrate the performances of the proposed efforts in comparison with other simple approaches.