The multivariate fusion transformation is ubiquitous in multivariate time series prediction (MTSP) problems. The previous multivariate fusion transformation fuses the feature of different variates at a time step, then projects them to a new feature space for effective feature representation. However, temporal dependency is the most fundamental property of time series. The previous manner fails to capture the temporal dependency of the feature, which is destroyed in the transformed feature matrix. Multivariate feature extraction based on the feature matrix with missing temporal dependency leads to the loss of predictive performance of MTSP. To address this problem, we propose the Temporal Dependency Priority for Multivariate Time Series Prediction (TemDep) method. Specifically, TemDep extracts feature temporal dependency of multivariate time series first and then considers multivariate feature fusion. Moreover, the low-dimensional and high-dimensional feature fusion manners are designed with the temporal dependency priority to fit different dimensional multivariate time series. The extensive experimental results of different datasets show that our proposed method can outperform all state-of-the-art baseline methods. It proves the significance of temporal dependency priority for MTSP.

To alleviate the over-smoothing problem caused by deep graph neural networks, decoupled graph neural networks (DGNNs) are proposed. DGNNs decouple the graph neural network into two atomic operations, the propagation (P) operation and the transformation (T) operation. Since manually designing the architecture of DGNNs is a time-consuming and expert-dependent process, the DF-GNAS method is designed, which can automatically construct the architecture of DGNNs with fixed propagation operation and deep layers. The propagation operation is a key process for DGNNs to aggregate graph structure information. However, DF-GNAS automatically designs DGNN architecture using fixed propagation operation for different graph structures will cause performance loss. Meanwhile, DF-GNAS designs deep DGNNs for graphs with simple distributions, which may lead to overfitting problems. To solve the above challenges, we propose the Decoupled Graph Neural Architecture Search with Variable Propagation Operation and Appropriate Depth (DGNAS-PD) method. In DGNAS-PD, we design a DGNN operation space with variable efficient propagation operations in order to better aggregate information on different graph structures. We build an effective genetic search strategy to adaptively design appropriate DGNN depths instead of deep DGNNs for the graph with simple distributions in DGNAS-PD. The experiments on five real-world graphs show that DGNAS-PD outperforms state-of-art baseline methods.

In the field of financial prediction, most studies focus on individual stocks or stock indices. Stock sectors are collections of stocks with similar characteristics and the indices of sectors have more stable trends and predictability compared to individual stocks. Additionally, stock sectors are subsets of stock indices, which implies that investment portfolios based on stock sectors have a greater potential to achieve excess returns. In this paper, we propose a new method, Time-aware Graph Structure Learning Network (TGSLN), to address the problem of stock sector ranking recommendation. In this model, we use an indicator called Relative Price Strength (RPS) to describe the ranking change trend of the sectors. To construct the inherent connection between sectors, we construct a multi-variable time series that consists of multi-scale RPS sequences and effective indicators filtered through the factor selector. We also build a stock sector relation graph based on authoritative stock sector classifications. Specially, we design a time-aware graph structure learner, which can mine the sector relations from time series, and enhance the initial graph through graph fusion. Our model outperforms state-of-the-art baselines in both A-share and NASDAQ markets.

In recent years, heterogeneous graph neural networks (HGNNs) have achieved excellent performance. The efficient HGNNs consist of meta-graphs and aggregation operations. Since manually designing meta-graph is an expert-dependent and time-consuming process, the performance of HGNNs is limited. To address this challenge, the differentiable meta-graph search has been proposed to obtain promising meta-graph automatically. However, the previous differentiable meta-graph search constructs the supernet without learnable aggregation operations, which limits the semantics extracting ability of HGNNs with automatically designed meta-graph for downstream tasks. To solve this problem, we propose the Meta-graph Search with Learnable Supernet for Heterogeneous Graph Neural Networks (MSLS). Specifically, to obtain better performance HGNNs, MSLS constructs a supernet with learnable aggregation operations based on the meta-graphs. MSLS adopts decoupling training to train the learnable supernet and obtains the optimal meta-graph with learnable aggregation operations using a constrained evolution strategy. Extensive experiments show that our method (MSLS) achieves the best performance in different tasks.

Graph classification has been widely used for knowledge discovery in numerous practical application scenarios, such as social networks and protein-protein interaction networks. Recently, Graph Neural Networks (GNNs), which generalize deep neural networks to graph-structured data, have drawn considerable attention and achieved state-of-the-art performance in graph classification. However, existing GNN models mainly focus on capturing the information of immediate or first-order neighboring nodes within a single layer. The graph substructure and substructure interaction, that play an important role in learning graph representations, are usually overlooked. In this paper, we propose a Substructure Assembling Graph Attention Network (SA-GAT) to extract graph features and improve the performance of graph classification. SA-GAT is able to fully explore higher-order substructure information hidden in graphs by a core module called Substructure Interaction Attention (SIA). Theoretically, we have also proved that SA-GAT satisfies the graph isomorphism theory of graph neural network design, which is that the network should map isomorphic graphs to the same representation and output the same prediction. Extensive experimental results on multiple real-world graph classification datasets demonstrate that the proposed SA-GAT outperforms the state-of-the-art methods including graph kernels and graph neural networks.

Multivariate time series (MTS) prediction has been widely applied in a diverse range of fields including electricity, economics, finance, and traffic. Many studies have successfully constructed spatial and temporal convolution modules called spatio-temporal block (ST-block) for multivariate time series prediction. However, existing methods need to manually design the architecture topology based on ST-blocks, which is time-consuming and requires extensive expert experience. In this paper, we propose a Spatio-Temporal based Architecture Topology Search (STATS) method for multivariate time series prediction, which can automatically design the ST-block for multivariate time series prediction. In the STATS, we construct static and dynamic graphs topologically to integrate both static and dynamic information to obtain more expressive ST-graphs for the prediction task. Then, STATS explores the architecture topology with the differentiable search algorithm based on ST-blocks automatically. Extensive experiments on four commonly used multivariate time series prediction benchmark datasets demonstrate that our proposed method STATS can outperform the state-of-the-art baseline models.

Stock prediction is a challenging task due to the uncertainty of stock markets. Despite the success of previous works, most of them rely on the assumption that stock data follow the identically identical distribution while the existence of multiple trading patterns in stock market violates it, ignoring multiple patterns in stock market will inevitably lead to the performance decline, and the lack of pattern prior knowledge further hinders the learning of patterns. In this paper, we propose a novel training process Pattern Adaptive Training based on Optimal Transport (OT) to train a set of predictors specializing in diverse patterns while without any prior pattern knowledge and inconsistent assumption. Based on this process, we further mine the potential fitness rank among specialists and design the Pattern Adaptive Specialist Network (PASN) with proposed ranking based selector to choose appropriate specialist predictor for samples. Extensive experimental results show that our method achieves best IC and other metrics on real-world stock datasets.

Graph Neural Networks (GNNs) have drawn attention due to their excellent performance in fraud detection tasks, which reveal fraudsters by aggregating the features of their neighbors. However, some fraudsters typically tend to alleviate their suspiciousness by connecting with many benign ones. Besides, label-imbalanced neighborhood also deteriorates fraud detection accuracy. Such behaviors violate the homophily assumption and worsen the performance of GNN-based fraud detectors. In this paper, we propose a Dual-Augment Graph Neural Network (DAGNN) for fraud detection tasks. In DAGNN, we design a two-pathway framework including disparity augment (DA) pathway and similarity augment (SA) pathway. Accordingly, we devise two novel information aggregation strategies. One is to augment the disparity between target node and its heterogenous neighbors in original topology. The other is to augment its similarity to homogenous neighbors in a relatively label-balanced neighborhood. The experimental results compared with the state-of-the-art models on two real-world datasets demonstrate the superiority of the proposed DAGNN.

Multivariate time series (MTS) forecasting is a challenging task. In MTS forecasting, We need to consider both intra-series temporal correlations and inter-series spatial correlations simultaneously. However, existing methods capture spatial correlations from the local data of the time series, without taking the global historical information of time series into account. In addition, most methods base on graph neural network mining for the temporal correlations tend to the redundancy of information at adjacent time points in the time-series data, which introduces noise. In this paper, we propose a memory augmented graph learning network (MAGL), which captures the spatial correlations in terms of the global historical features of MTS. Specifically, we use a memory unit to learn from the local data of MTS. The memory unit records the global historical features of the time series, which is used to mine the spatial correlations. We also design a temporal feature distiller to reduce the noise in extracting temporal features. We extensively evaluate our model on four real-world datasets, comparing with several state-of-the-art methods. The experimental results show MAGL outperforms the state-of-the-art baseline methods on several datasets.

Stock prediction is a hot topic of research in the field of Fintech. Stocks are not independent of each other. But, existing studies ignore the relations between stocks or simply utilize stock spatial dependencies based on predefined graphs. The predefined graphs may miss some potential relations and are not suitable for depicting the dynamic relations between stocks. To address both problems for stock recommendation, we propose the static-dynamic graph neural network (SDGNN). In SDGNN, a graph learning module is designed to learn the static and dynamic graphs. This module employs a data-driven approach which makes the model uncover potential relations between stocks. Furthermore, to enable each stock node to obtain more information from more important neighbor nodes, we develop a graph interaction module. It implements interactions between the static graph and the dynamic graph. Experiments demonstrate that our proposed model significantly outperforms the current state-of-the-art methods on two real-world stock datasets.

The stock market investors aim at maximizing their investment returns. Stock recommendation task is to recommend stocks with higher return ratios for the investors. Most stock prediction methods study the historical sequence patterns to predict stock trend or price in the near future. In fact, the future price of a stock is correlated not only with its historical price, but also with other stocks. In this article, we take into account the relationships between stocks (corporations) by stock relation graph. Furthermore, we propose a Time-aware Relational Attention Network (TRAN) for graph-based stock recommendation according to return ratio ranking. In TRAN, the time-aware relational attention mechanism is designed to capture time-varying correlation strengths between stocks by the interaction of historical sequences and stock description documents. With the dynamic strengths, the nodes of the stock relation graph aggregate the features of neighbor stock nodes by graph convolution operation. For a given group of stocks, the proposed TRAN model can output the ranking results of stocks according to their return ratios. The experimental results on several real-world datasets demonstrate the effectiveness of our TRAN for stock recommendation.

With the development of e-commerce, fraud behaviors have been becoming one of the biggest threats to the e-commerce business. Fraud behaviors seriously damage the ranking system of e-commerce platforms and adversely influence the shopping experience of users. It is of great practical value to detect fraud behaviors on e-commerce platforms. However, the task is non-trivial, since the adversarial action taken by fraudsters. Existing fraud detection systems used in the e-commerce industry easily suffer from performance decay and can not adapt to the upgrade of fraud patterns, as they take already known fraud behaviors as supervision information to detect other suspicious behaviors. In this article, we propose a competitive graph neural networks (CGNN)-based fraud detection system (eFraudCom) to detect fraud behaviors at one of the largest e-commerce platforms, “Taobao” 1 . In the eFraudCom system, (1) the competitive graph neural networks (CGNN) as the core part of eFraudCom can classify behaviors of users directly by modeling the distributions of normal and fraud behaviors separately; (2) some normal behaviors will be utilized as weak supervision information to guide the CGNN to build the profile for normal behaviors that are more stable than fraud behaviors. The algorithm dependency on fraud behaviors will be eliminated, which enables eFraudCom to detect fraud behaviors in presence of the new fraud patterns; (3) the mutual information regularization term can maximize the separability between normal and fraud behaviors to further improve CGNN. eFraudCom is implemented into a prototype system and the performance of the system is evaluated by extensive experiments. The experiments on two Taobao and two public datasets demonstrate that the proposed deep framework CGNN is superior to other baselines in detecting fraud behaviors. A case study on Taobao datasets verifies that CGNN is still robust when the fraud patterns have been upgraded.

Graph neural architecture search has received a lot of attention as Graph Neural Networks (GNNs) has been successfully applied on the non-Euclidean data recently. However, exploring all possible GNNs architectures in the huge search space is too time-consuming or impossible for big graph data. In this paper, we propose a parallel graph architecture search (GraphPAS) framework for graph neural networks. In GraphPAS, we explore the search space in parallel by designing a sharing-based evolution learning, which can improve the search efficiency without losing the accuracy. Additionally, architecture information entropy is adopted dynamically for mutation selection probability, which can reduce space exploration. The experimental result shows that GraphPAS outperforms state-of-art models with efficiency and accuracy simultaneously.

Knowledge Graphs (KGs) are widely used in various applications of information retrieval. Despite the large scale of KGs, they are still facing incomplete problems. Conventional approaches on Knowledge Graph Completion (KGC) require a large number of training instances for each relation. However, long-tail relations which only have a few related triples are ubiquitous in KGs. Therefore, it is very difficult to complete the long-tail relations. In this paper, we propose a meta pattern learning framework (MetaP) to predict new facts of relations under a challenging setting where there is only one reference for each relation. Patterns in data are representative regularities to classify data. Triples in KGs also conform to relation-specific patterns which can be used to measure the validity of triples. Our model extracts the patterns effectively through a convolutional pattern learner and measures the validity of triples accurately by matching query patterns with reference patterns. Extensive experiments demonstrate the effectiveness of our method. Besides, we build a few-shot KGC dataset of COVID-19 to assist the research process of the new coronavirus.

In distributed storage, dynamic loads usually cause a huge burden such as hot-spot and unblance region problems. In this paper, we focus on the problems of dynamic unbalance loads in distributed database HBase. Firstly, we propose a new intelligent load balance framework to support the collection and analysis of dynamic load information. Then, two efficient load balance strategies are proposed to solve hot-spot problem and unbalance region problem. Finally, we evaluate the strategies on HBase using real-world big data on distributed clusters. Compared to the existing balance strategies, the proposed load balance strategies can make more effective use of the resources and reduce the response time for large dynamic loads.

Recommending stock with the highest return ratio is always a challenging problem in the field of financial technology. In this paper, we propose a time-aware graph relational attention network (TRAN) for stock recommendation based on return ratio ranking. In TRAN, time-aware relational attention mechanism is the key unit to capture time-varying correlation strength between stocks by the interaction of historical sequences and stock description documents. With the dynamic strength, the nodes of the stock relation graph aggregate the features of neighbor stock nodes by graph convolution operation. For a given group of stocks, our model can output the ranking results of stocks according to their return ratios. The experimental results on several real-world datasets demonstrate the effectiveness of our TRAN for stock recommendation.

Predicting the survival of cancer patients holds significant meaning for public health, and has attracted increasing attention in medical information communities. In this study, we propose a novel framework for cancer survival prediction named Multimodal Graph Neural Network (MGNN), which explores the features of real-world multimodual data such as gene expression, copy number alteration and clinical data in a unified framework. In order to explore the inherent relation, we first construct the bipartite graphs between patients and multimodal data. Subsequently, graph neural network is adopted to obtain the embedding of each patient on different bipartite graphs. Finally, a multimodal fusion neural layer is designed to fuse the features from different modal data. The output of our method is the classification of short term survival or long term survival for each patient. Experimental results on one breast cancer dataset demonstrate that MGNN outperforms all baselines. Furthermore, we test the trained model on lung cancer dataset, and the experimental results verify the strong robust by comparing with state-of-the-art methods.

Subgraph matching query is to find out the sub-graphs of data graph G which match a given query graph Q. Traditional methods can not deal with big data graphs due to their high computational complex. In this paper, we propose a distributed top-k subgraph search method over big graphs. The proposed method is designed at the level of single vertex and all vertices obtain their matching state separately without requiring global graph information. Therefore, it can be easily deployed in distributed platform like Hadoop. The evaluations of running time, number of messages and supersteps show the efficiency and scalability of the proposed method.

Aspect-level opinion mining aims to find and aggregate opinions on opinion targets. Previous work has demonstrated that precise modeling of opinion targets within the surrounding context can improve performances. However, how to effectively and efficiently learn hidden word semantics and better represent targets and the context still needs to be further studied. In this paper, we propose and compare two interactive attention neural networks for aspect-level opinion mining, one employs two bi-directional Long-Short-Term-Memory (BLSTM) and the other employs two Convolutional Neural Networks (CNN). Both frameworks learn opinion targets and the context respectively, followed by an attention mechanism that integrates hidden states learned from both the targets and context. We compare our model with state-of-the-art baselines on two SemEval 2014 datasets1. Experiment results show that our models obtain competitive performances against the baselines on both datasets. Our work contributes to the improvement of state-of-the-art aspect-level opinion mining methods and offers a new approach to support human decision-making process based on opinion mining results. The quantitative and qualitative comparisons in our work aim to give basic guidance for neural network selection in similar tasks.

Botnet, a group of coordinated bots, is becoming the main platform of malicious Internet activities like DDOS, click fraud, web scraping, spam/rumor distribution, etc. This paper focuses on design and experiment of a new approach for botnet detection from streaming web server logs, motivated by its wide applicability, real-time protection capability, ease of use and better security of sensitive data. Our algorithm is inspired by a Principal Component Analysis (PCA) to capture correlation in data, and we are first to recognize and adapt Lanczos method to improve the time complexity of PCA-based botnet detection from cubic to sub-cubic, which enables us to more accurately and sensitively detect botnets with sliding time windows rather than fixed time windows. We contribute a generalized online correlation matrix update formula, and a new termination condition for Lanczos iteration for our purpose based on error bound and non-decreasing eigenvalues of symmetric matrices. On our dataset of an ecommerce website logs, experiments show the time cost of Lanczos method with different time windows are consistently only 20% to 25% of PCA.

Network alignment is becoming an active topic in network data analysis. Despite extensive research, we realize that efficient use of topological and attribute information for large attributed network alignment has not been sufficiently addressed in previous studies. In this paper, based on Stochastic Block Model (SBM) and Dirichlet-multinomial, we propose "divide-and-conquer" models CAlign that jointly consider network alignment, community discovery and community alignment in one framework for large networks with node attributes, in an effort to reduce both the computation time and memory usage while achieving better or competitive performance. It is provable that the algorithms derived from our model have sub-quadratic time complexity and linear space complexity on a network with small densification power, which is true for most real-world networks. Experiments show CAlign is superior to two recent state-of-art models in terms of accuracy, time and memory on large networks, and CAlign is capable of handling millions of nodes on a modern desktop machine.

In this paper, we propose a novel k-anonymization scheme to counter deanonymization queries on social networks. With this scheme, all entities are protected by k-anonymization, which means the attackers cannot re-identify a target with confidence higher than 1/k. The proposed scheme minimizes the modification on original networks, and accordingly maximizes the utility preservation of published data while achieving k-anonymization privacy protection. Extensive experiments on real data sets demonstrate the effectiveness of the proposed scheme, where the efficacy of the k-anonymized networks is verified with the distributions of pagerank, betweenness, and their Kolmogorov-Smirnov (K-S) test.

In the real world, many problems on massive graphs can be mapped to an underlying critical problem of discovering top-k subgraphs. For massive graphs, subgraph queries may have enormous number of matches, and so it is inefficient to compute all matches when only top-k matches are desired. Meanwhile, parallel algorithm is urgent for the scalability of massive graph computing. In this paper, we address the challenges of top-k subgraph query in massive graph. Firstly, we present a new graph matching notion: “approximate graph simulation”. With approximate graph simulation, top-k subgraph query can be customized by appointing a weighted query graph, which provides good flexibility for different application scenarios. Secondly, we propose a parallel top-k subgraph query algorithm at the level of vertex. With such algorithm, each vertex in massive graph obtains its matching state separately without requiring global graph information. In the algorithm, we also design a filter mechanism to speed up the the computation and a aggregation mechanism to obtain top-k vertices for query focus. Using real-life datasets, we experimentally verify that our approach of parallel top-k subgraph query are efficient.