Recent years have seen rapid progress in network representation learning, which removes the need for burdensome feature engineering and facilitates downstream network-based tasks. In reality, networks often exhibit heterogeneity, which means there may exist multiple types of nodes and interactions. Heterogeneous networks raise new challenges to representation learning, as the awareness of node and edge types is required. In this article, we study a basic building block of general heterogeneous networks, the heterogeneous networks with two types of nodes. Many problems can be solved by decomposing general heterogeneous networks into multiple bipartite ones. Recently, to overcome the demerits of non-metric measures used in the embedding space, metric learning-based approaches have been leveraged to tackle heterogeneous network representation learning. These approaches first generate triplets of samples, in which an anchor node, a positive counterpart, and a negative one co-exist, and then try to pull closer positive samples and push away negative ones. However, when dealing with heterogeneous networks, even the simplest two-typed ones, triplets cannot simultaneously involve both positive and negative samples from different parts of networks. To address this incompatibility of triplet-based metric learning, in this article, we propose a novel quintuple-based method for learning node representations in bipartite heterogeneous networks. Specifically, we generate quintuples that contain positive and negative samples from two different parts of networks. And we formulate two learning objectives that accommodate quintuple-based learning samples, a proximity-based loss that models the relations in quintuples by sigmoid probabilities and an angular loss that more robustly maintains similarity structures. In addition, we also parameterize feature learning by using one-dimensional convolution operators around nodes’ neighborhoods. Compared with eight methods, extensive experiments on two downstream tasks manifest the effectiveness of our approach.

In recent years, the field of unsupervised graph representation learning has witnessed the emergence of graph contrastive learning (GCL) as a highly successful approach. GCL excels in learning graph representations by effectively bringing positive sample pairs into proximity while simultaneously pushing negative sample pairs apart in the representation space, without any manual labels. Graph-structured data to be learned inherently exhibits a critical hierarchical structure, which is crucial for organizing and managing graphs. Leveraging this attribute enhances the accuracy of graph representation outcomes. However, current GCL methods tend to overlook the hierarchical structure, which can result in sampling bias during node selection. Nodes of the same semantics can potentially be sampled as negative pairs. To overcome these limitations, we present a novel framework, Hierarchical Graph Contrastive Learning via Debiasing Noise Samples with Adaptive Repelling Ratio (HGClear). Our framework enables the simultaneous learning of node representations and the graph hierarchy in an end-to-end manner. During the process of method design, we discovered that the accuracy of node category prediction significantly affects representation results. To remove the bias caused by noise samples, we introduced a module to handle boundary nodes (i.e., noise samples) that are vulnerable to mislabeling. Specifically, we introduce a hierarchy detection module that captures both coarse-grained views and category attributes of nodes. Leveraging these results, we can identify boundary nodes and establish varying repelling ratios based on category labels, replacing the conventional temperature coefficient in the contrastive loss. Simultaneously incorporating an intra-view node contrast module not only eliminates the bias resulting from noise samples but also enhances the uniqueness of node representations. Numerous experiments on node classification datasets show that HGClear produces encouraging results and outperforms some state-of-the-art methods.

Resolving community structure in networks is of significant benefit for both scientific inquiries and practical applications. Recently, deep neural networks have demonstrated excellent performance on various graph mining tasks, including community detection. However, there are still some challenges that are urgent to be addressed. First, being frequently formulated in an unsupervised setting, community detection has been proved to be more resistant to the advantages of end-to-end learning. Many deep methods carry out clustering algorithms after the acquisition of node representations. Second, very few studies consider the heterogeneity of a large number of real-world networks in end-to-end community detection. For instance, the building blocks of general heterogeneous networks are the bipartite model, which is a ubiquitous structure where two types of nodes co-exist. In view of these challenges, we study the end-to-end community co-partition of two types of nodes in bipartite networks. Specifically, we extend both spectral and spatial graph convolution operators to bipartite structures for node feature encoding. Then we formulate a novel loss function with a modularity-based objective, as well as two collapsed regularizations for producing more informative community assignment matrices. Co-partitions of nodes can be directly achieved by optimization with stochastic gradient descent under the proposed framework. Comprehensive empirical analysis, compared with various types of classic and deep methods, demonstrates the efficacy and the scalability of the proposed method.

Real-world networks often show heterogeneity. A frequently encountered type is the bipartite heterogeneous structure, in which two types of nodes and three types of edges exist. Recently, much attention has been devoted to representation learning in these networks. One of the essential differences between heterogeneous and homogeneous learning is that the former structure requires methods to possess awareness to node and edge types. Most existing methods, including metapath-based, proximity-based and graph neural network-based, adopt inner product or vector norms to evaluate the similarities in embedding space. However, these measures either violates the triangle inequality, or show severe sensitivity to scaling transformation. The limitations often hinder the applicability to real-world problems. In view of this, in this paper, we propose a novel angle-based method for bipartite heterogeneous network representation. Specifically, we first construct training sets by generating quintuples, which contain both positive and negative samples from two different parts of networks. Then we analyze the quintuple-based problem from a geometry perspective, and transform the comparisons between preferred and non-preferred samples to the comparisons of angles. In addition, we utilize convolution modules to extract node features. A hinge loss, as the final objective, is proposed to relax the angular constraint for learning. Extensive experiments for two typical tasks show the efficacy of the proposed method, comparing with eight competitive methods.

Keyphrase annotation task aims to retrieve the most representative phrases that express the essential gist of documents. In reality, some phrases that best summarize documents are often absent from the original text, which motivates researchers to develop generation methods, being able to create phrases. Existing generation approaches usually adopt the encoder-decoder framework for sequence generation. However, the widely-used recurrent neural network might fail to capture long-range dependencies among items. In addition, intuitively, as keyphrases are likely to correlate with topical words, some methods propose to introduce topic models into keyphrase generation. But they hardly leverage the global information of topics. In view of this, we employ the Transformer architecture with the pre-trained BERT model as the encoder-decoder framework for keyphrase generation. BERT and Transformer are demonstrated to be effective for many text mining tasks. But they have not been extensively studied for keyphrase generation. Furthermore, we propose a topic attention mechanism to utilize the corpus-level topic information globally for keyphrase generation. Specifically, we propose BertTKG, a keyphrase generation method that uses a contextualized neural topic model for corpus-level topic representation learning, and then enhances the document representations learned by pre-trained language model for better keyphrase decoding. Extensive experiments conducted on three public datasets manifest the superiority of BertTKG.

Given the high cost of large-scale data centers, an important design goal is to fully utilize available power resources to maximize the computing capacity. In this paper we present Ampere, a novel power management system for data centers to increase the computing capacity by over-provisioning the number of servers. Instead of doing power capping that degrades the performance of running jobs, we use a statistical control approach to implement dynamic power management by indirectly affecting the workload scheduling, which can enormously reduce the risk of power violations. Instead of being a part of the already over-complicated scheduler, Ampere only interacts with the scheduler with two basic APIs. Instead of power control on the rack level, we impose power constraint on the row level, which leads to more room for over provisioning.

With the ever increasing number of large scale Internet applications, inter data center (inter-DC) data transfers are becoming more and more common. Traditional inter-DC transfers suffers from both low-utilization and congestion, and traffic prediction is an important method to optimize these transfers. Inter-DC traffic is harder to predict than many other types of network traffic, because it is dominated by a few large applications. We propose a model that significantly reduces the prediction errors. In our model, we combine wavelet transform with artificial neural network (ANN) to improve prediction accuracy. Specifically, we explicitly add information of elephant flows, the least predictable yet dominating traffic in inter-DC network, into our prediction model. To reduce the amount of monitoring overhead for the elephant flow information, we added interpolation to fill in the unknown values in the elephant flows. We demonstrate that we can reduce prediction errors over existing methods by 5%~10%. Our prediction is already in production at Baidu, one of the largest Internet companies in China, helping reducing the peak network bandwidth.

With the ever increasing number of large scale Internet applications, inter-data-center (inter-DC) data transfers are becoming more and more common. Traditional inter-DC transfers suffer from both low utilization and congestion, and traffic prediction is an important method to optimize these transfers. Inter-DC traffic is harder to predict than many other types of network traffic because it is dominated by a few large applications. We propose a model that significantly reduces the prediction errors. In our model, we combine wavelet transform with artificial neural network to improve prediction accuracy. Specifically, we explicitly add information of sublink traffic and elephant flows, the least predictable yet dominating traffic in inter-DC network, into our prediction model. To reduce the amount of monitoring overhead for the elephant flow information, we add interpolation to fill in the unknown values in the elephant flows. We demonstrate that we can reduce prediction errors over existing methods by 5%~30%. Our prediction is in production as part of the traffic scheduling system at Baidu, one of the largest Internet companies in China, helping to reduce the peak network bandwidth.

We present Fatman, an enterprise-scale archival storage based on volunteer contribution resources from underutilized web servers, usually deployed on thousands of nodes with spare storage capacity. Fatman is specifically designed for enhancing the utilization of existing storage resources and cutting down the hardware purchase cost. Two major concerned issues of the system design are maximizing the resource utilization of volunteer nodes without violating Service Level Objectives (SLOs) and minimizing the cost without reducing the availability of archival system. Fatman has been widely deployed on tens of thousands of server nodes across several datacenters, provided more than 100PB storage capacity and served dozens of internal mass-data applications. The system realizes an efficient storage quota consolidation by strong isolation and budget limitation, to maximally support resources contribution without any degradation on host-level SLOs. It firstly improves data reliability by applying disk failure prediction to minish failure recovery cost, named fault-aware data management, dramatically reduces the MTTR by 76.3% and decreases file crash ratio by 35% on real-life product workload.

Most of the modern hard disk drives support Self-Monitoring, Analysis and Reporting Technology (SMART), which can monitor internal attributes of individual drives and predict impending drive failures by a thresholding method. As the prediction performance of the thresholding algorithm is disappointing, some researchers explored various statistical and machine learning methods for predicting drive failures based on SMART attributes. However, the failure detection rates of these methods are only up to 50% ~ 60% with low false alarm rates (FARs). We explore the ability of Backpropagation (BP) neural network model to predict drive failures based on SMART attributes. We also develop an improved Support Vector Machine (SVM) model. A real-world dataset concerning 23,395 drives is used to verify these models. Experimental results show that the prediction accuracy of both models is far higher than previous works. Although the SVM model achieves the lowest FAR (0.03%), the BP neural network model is considerably better in failure detection rate which is up to 95% while keeping a reasonable low FAR.