Drug recommendation based on electronic health record (EHR) is fundamental to effective disease treatment. Similar to commercial sequence-based recommendation systems, the accuracy of drug recommendation largely depends on precise patient modeling. However, patient modeling is more complex, as it not only requires sequence modeling of patient's disease course, but also needs to refer to the information of patients with similar medical medication. In EHR data, many patients have only one visit record, and the similarity between patients is often vague and unclear, which may cause noise and ambiguity. This leads to significant challenges for the drug recommendation field, especially when patient records are sparse or when patient similarity is vague. To address the above challenges, we propose RaVSNet (Relevance aware Visit Similarity Network), which improves drug recommendation by leveraging both longitudinal and transversal visit similarity and integrating medical relevance knowledge. RaVSNet utilizes multi-dimensional visit information similar to the patient's current visit as a reference, and employs a relevance-aware network to explicitly model the matching relationships between medical conditions and medications. Additionally, RaVSNet designs a general pretraining framework specifically for drug recommendation, including two tasks, Medication Sequence Reconstruction (MSR) and Causal Effect Inference (CEI), to discover the deep connections between medical information and medications. Experimental results on two public EHR datasets, MIMIC-III and MIMIC-IV demonstrate that the proposed algorithm outperforms state-of-the-art methods, yielding more accurate drug recommendation combinations, and the proposed general pretraining framework can be seamlessly integrated into most drug recommendation methods to achieve performance improvements.

Deep graph learning models have recently been developed to learn from various graphs that are prevalent in describing and modeling complex systems, including those in bioinformatics. However, a versatile explanation method for uncovering the general graph patterns that guide deep graph models in making predictions remains elusive. In this paper, we propose DGX, a novel deep graph model explainer that generates explanatory graphs to explain trained, opaque-box deep graph models. Its effectiveness is demonstrated by producing multiple graphs that collectively encode the structural knowledge captured by the graph neural network on both synthetic and real graph data. Importantly, DGX can produce diverse explanations by generating a set of distinguishable graphs and can provide customized explanations based on prior knowledge or constraints specified by users. We apply DGX to explain a mutagenicity prediction model by exploring the underlying groups of mutagenic compounds, and we explain the model on brain functional networks by revealing the structural patterns that enable the model to differentiate autism spectrum disorder from healthy controls. These findings offer an effective, diverse, and customized approach to explaining the underlying mechanisms and enhancing the understanding of models learned from real graph data, particularly in fields such as biomedicine and bioinformatics.

The performance of biomedical information retrieval greatly depends on biomedical knowledge; however the knowledge of available medical knowledge base is often incomplete and out-of-dated. To solve the problem that incomplete knowledge bases cannot provide the medical knowledge required for biomedical information retrieval, the paper proposes an Entity Relation Aware Graph Neural Ranking model (ERAGNR), aiming to fully leverage the internal knowledge of the document to alleviate the problem caused by incomplete external knowledge bases. ERAGNR mines the relationships between biomedical entities in the document through entity relation extraction and combines them with external knowledge. It increases the semantic association and reduces the semantic gap between the query and the document. The method first constructs a knowledge-query graph and a document-entity graph, and then fuses the two graphs to obtain a knowledge-query-document-entity graph. In a multi-task learning framework that combines text retrieval and relation extraction tasks, ERAGNR employs a shared text encoder and a graph neural network. This enables ERAGNR to learn semantic matching patterns between queries and documents and recognize relationships between entities in the documents. As a result, the model can capture semantic matching signals between entity relationships in the context and queries. The experimental results show that ERAGNR outperforms the state-of-the-art models. Through biomedical relation extraction task, the model can learn the ability to capture the context of the entity relations in the document, so that the model can more accurately match the semantics between the query and the document.

Advanced deep learning methods, especially graph neural networks (GNNs), are increasingly expected to learn from brain functional network data and predict brain disorders. In this paper, we proposed a novel Transformer and snowball encoding networks (TSEN) for brain functional network classification, which introduced Transformer architecture with graph snowball connection into GNNs for learning whole-graph representation. TSEN combined graph snowball connection with graph Transformer by snowball encoding layers, which enhanced the power to capture multi-scale information and global patterns of brain functional networks. TSEN also introduced snowball graph convolution as position embedding in Transformer structure, which was a simple yet effective method for capturing local patterns naturally. We evaluated the proposed model by two large-scale brain functional network datasets from autism spectrum disorder and major depressive disorder respectively, and the results demonstrated that TSEN outperformed the state-of-the-art GNN models and the graph-transformer based GNN models.

Deep learning methods have been applied for dynamic brain functional network analysis recently. However, they are usually restricted by the complex spatio-temporal dynamics and the limited labeled data. In this paper, we proposed a stacked Transformer neural network, namely BrainPST, to capture spatio-temporal patterns for dynamic brain functional network classification. BrainPST model integrated spatial and temporal information by stacking two Transformers: one for learning snapshot networks and the other for learning sequence of functional connections. Unlike recent models, BrainPST was designed to pre-train the stacked Transformer network by leveraging unlabeled existing brain imaging data. A pre-training framework with designed pre-training strategies was proposed to learn general spatio-temporal representations from the unlabeled brain networks, and to fine-tune the pre-trained model in downstream tasks. BrainPST is a conceptually simple and effective model. The results of experiments showed the BrainPST model without pre-training achieved comparative performance with the recent models, and the pre-trained BrainPST obtained new state-of-the-art performance. The pre-trained BrainPST improved 3.92% of AUC compared with the model without pre-training.

Video denoising aims at removing noise from noisy video frames and meanwhile preserving their structures and details. It is a challenging task, as both noise and video structures/details correspond to high-frequency components of a noisy video which are hard to distinguish. This paper proposes a deep video denoiser using a progressive decomposition process with conditional invertibility. Noisy video frames are first decomposed into two latent codes via a forward process of conditional invertible coupling layers, where one latent code carries the maximal information regarding the noise-free reference frame while the other encodes the information regarding noise, misalignment and content difference. The clean video is then reconstructed from the latent codes of noise-free frames using the reverse pass of the coupling layers. To improve the robustness to variant noise levels, the coupling layers are conditioned on noise level. In addition, memory units are introduced to the conditioned coupling layers to better exploit temporal correlation among frames for feature disentanglement. Experiments on two benchmark datasets have demonstrated the effectiveness of our method.

The development of omics data and biomedical images has greatly advanced the progress of precision medicine in diagnosis, treatment, and prognosis. The fusion of omics and imaging data, i.e., omics-imaging fusion, offers a new strategy for understanding complex diseases. However, due to a variety of issues such as the limited number of samples, high dimensionality of features, and heterogeneity of different data types, efficiently learning complementary or associated discriminative fusion information from omics and imaging data remains a challenge. Recently, numerous machine learning methods have been proposed to alleviate these problems. In this review, from the perspective of fusion levels and fusion methods, we first provide an overview of preprocessing and feature extraction methods for omics and imaging data, and comprehensively analyze and summarize the basic forms and variations of commonly used and newly emerging fusion methods, along with their advantages, disadvantages and the applicable scope. We then describe public datasets and compare experimental results of various fusion methods on the ADNI and TCGA datasets. Finally, we discuss future prospects and highlight remaining challenges in the field.

BACKGROUND: Autism spectrum disorders (ASD) imply a spectrum of symptoms rather than a single phenotype. ASD could affect brain connectivity at different degree based on the severity of the symptom. Given their excellent learning capability, graph neural networks (GNN) methods have recently been used to uncover functional connectivity patterns and biological mechanisms in neuropsychiatric disorders, such as ASD. However, there remain challenges to develop an accurate GNN learning model and understand how specific decisions of these graph models are made in brain network analysis. RESULTS: In this paper, we propose a graph attention network based learning and interpreting method, namely GAT-LI, which learns to classify functional brain networks of ASD individuals versus healthy controls (HC), and interprets the learned graph model with feature importance. Specifically, GAT-LI includes a graph learning stage and an interpreting stage. First, in the graph learning stage, a new graph attention network model, namely GAT2, uses graph attention layers to learn the node representation, and a novel attention pooling layer to obtain the graph representation for functional brain network classification. We experimentally compared GAT2 model's performance on the ABIDE I database from 1035 subjects against the classification performances of other well-known models, and the results showed that the GAT2 model achieved the best classification performance. We experimentally compared the influence of different construction methods of brain networks in GAT2 model. We also used a larger synthetic graph dataset with 4000 samples to validate the utility and power of GAT2 model. Second, in the interpreting stage, we used GNNExplainer to interpret learned GAT2 model with feature importance. We experimentally compared GNNExplainer with two well-known interpretation methods including Saliency Map and DeepLIFT to interpret the learned model, and the results showed GNNExplainer achieved the best interpretation performance. We further used the interpretation method to identify the features that contributed most in classifying ASD versus HC. CONCLUSION: We propose a two-stage learning and interpreting method GAT-LI to classify functional brain networks and interpret the feature importance in the graph model. The method should also be useful in the classification and interpretation tasks for graph data from other biomedical scenarios.

The curse of dimensionality, which is caused by high-dimensionality and low-sample-size, is a major challenge in gene expression data analysis. However, the real situation is even worse: labelling data is laborious and time-consuming, so only a small part of the limited samples will be labelled. Having such few labelled samples further increases the difficulty of training deep learning models. Interpretability is an important requirement in biomedicine. Many existing deep learning methods are trying to provide interpretability, but rarely apply to gene expression data. Recent semi-supervised graph convolution network methods try to address these problems by smoothing the label information over a graph. However, to the best of our knowledge, these methods only utilize graphs in either the feature space or sample space, which restrict their performance. We propose a transductive semi-supervised representation learning method called a hierarchical graph convolution network (HiGCN) to aggregate the information of gene expression data in both feature and sample spaces. HiGCN first utilizes external knowledge to construct a feature graph and a similarity kernel to construct a sample graph. Then, two spatial-based GCNs are used to aggregate information on these graphs. To validate the model's performance, synthetic and real datasets are provided to lend empirical support. Compared with two recent models and three traditional models, HiGCN learns better representations of gene expression data, and these representations improve the performance of downstream tasks, especially when the model is trained on a few labelled samples. Important features can be extracted from our model to provide reliable interpretability.

Graph neural networks have been developed for various node classification tasks in graph data. However, the noise in graph data would affect the effectiveness of training and prediction of these graph learning models. In this paper, we propose a graph learning approach, named GCN_CL, which introduces a confident learning method into the graph convolutional networks model to classify nodes in graph data with label noise. The proposed approach includes node classification module and confident learning module, where the confident learning module selects clean nodes with high confident labels for node classification module to train an accurate model in the graphs with label noise. Pseudo label method is further applied on unlabeled data to increase samples for confident learning and improve the classification performance of subsequent node classification module. We evaluated classification performance of GCN_CL and compared our approach against other models to classify autism spectrum disorders patients versus health controls in population graph, which is constructed from ABIDE I dataset. The experimental results show GCN_CL achieves achieves the best performance in the population graph with different artificial noise levels.

Graph convolutional networks (GCNs) have been successfully applied to many graph data on various learning tasks such as node classification. However, there is limited understanding of the internal logic and decision patterns of GCNs. In this paper, we propose a layer-wise relevance propagation based explanation method for GCNs, namely GCN-LRP, to explore the latent pattern of GCNs. Then, we use three well-known citation network data sets and synthetic graph data sets for node classification tasks with GCN-LRP explanation, and experimentally identify latent attentions when GCNs aggregates information from the node and its neighboring nodes: (i) GCNs pay more attention to the classified node when comparing with its neighbors; (ii) GCNs do not pay attention to all the neighboring nodes equally, and a few neighboring nodes received more attention than others. Moreover, we further theoretically analyze and find that: (i) the latent attentions come from the recursively aggregating of GCNs; (ii) the neighboring nodes, which share enough neighbors with classified node, would receive more attention than other neighbors; (iii) the latent attention could hardly be changed by model training. We also discuss the advantage and limitations of GCNs introduced by the latent attentions, and implications of our findings for graph data learning with GCNs.

Online mobile advertising plays a vital role in the mobile app ecosystem. The mobile advertising frauds caused by fraudulent clicks or other actions on advertisements are considered one of the most critical issues in mobile advertising systems. To combat the evolving mobile advertising frauds, machine learning methods have been successfully applied to identify advertising frauds in tabular data, distinguishing suspicious advertising fraud operation from normal one. However, such approaches may suffer from labor-intensive feature engineering and robustness of the detection algorithms, since the online advertising big data and complex fraudulent advertising actions generated by malicious codes, botnets, and click-firms are constantly changing. In this paper, we propose a novel weighted heterogeneous graph embedding and deep learning-based fraud detection approach, namely, GFD, to identify fraudulent apps for mobile advertising. In the proposed GFD approach, (i) we construct a weighted heterogeneous graph to represent behavior patterns between users, mobile apps, and mobile ads and design a weighted metapath to vector algorithm to learn node representations (graph-based features) from the graph; (ii) we use a time window based statistical analysis method to extract intrinsic features (attribute-based features) from the tabular sample data; (iii) we propose a hybrid neural network to fuse graph-based features and attribute-based features for classifying the fraudulent apps from normal apps. The GFD approach was applied on a large real-world mobile advertising dataset, and experiment results demonstrate that the approach significantly outperforms well-known learning methods.

Predicting churning customers in advance allows marketers to retain existing and valuable customers, and to develop a customer churn predicting model is a key issue of customer relationship management in modern marketing. In this paper, a product-based Recurrent Neural Network (pRNN) approach is proposed for customer churn prediction in telecommunication sector. In the proposed model, RNN with long short-term memory units is used to learn sequential patterns from customer data changing over time, and the product operation is introduced before recurrent layer to learn high-order interaction between features. pRNN is applied on a real-world telecommunication dataset; experiment results demonstrate that pRNN significantly outperforms other comparison models.

We propose Top-N-Rank, a novel family of list-wise Learning-to-Rank models for reliably recommending the N top-ranked items. The proposed models optimize a variant of the widely used cumulative discounted gain (DCG) objective function which differs from DCG in two important aspects: (i) It limits the evaluation of DCG only on the top N items in the ranked lists, thereby eliminating the impact of low-ranked items on the learned ranking function; and (ii) it incorporates weights that allow the model to leverage multiple types of implicit feedback with differing levels of reliability or trustworthiness. Because the resulting objective function is non-smooth and hence challenging to optimize, we consider two smooth approximations of the objective function, using the traditional sigmoid function and the rectified linear unit (ReLU). We propose a family of learning-to-rank algorithms (Top-N-Rank) that work with any smooth objective function. Then, a more efficient variant, Top-N-Rank.ReLU, is introduced, which effectively exploits the properties of ReLU function to reduce the computational complexity of Top-N-Rank from quadratic to linear in the average number of items rated by users. The results of our experiments using two widely used benchmarks, namely, the MovieLens data set and the Amazon Video Games data set demonstrate that: (i) The "top-N truncation" of the objective function substantially improves the ranking quality of the top N recommendations; (ii) using the ReLU for smoothing the objective function yields significant improvement in both ranking quality as well as runtime as compared to using the sigmoid; and (iii) Top-N-Rank.ReLU substantially outperforms the well-performing list-wise ranking methods in terms of ranking quality.

Drug-Drug Interaction (DDI) relation extraction is a multi-class classification problem that aims to predict the interaction between drugs in a sentence. The configuration of Convolutional Neural Network (CNN) in relation extraction usually applied shallow architecture layers, which may make the information in given input text is not fully captured, thus fail to capture a long sentence containing the detected drug relation or some irrelevant word captured during the feature extraction process. This paper proposed an extending depth of the CNN layer called DeepCNN for DDI relation extraction. The DeepCNN learns the high quality of the learning representation so that it is able to cover long input sentences as the typical of DDIExtraction dataset. We use multi-channel word-embedding to enlarge the vocabulary and decrease the number of unknown words, and Adam update rule to automatically learn the network parameters of DeepCNN for DDI relation extraction. The experiments show that the architecture of 10 layers DeepCNN successfully obtained the significant improvement compared to the previous CNN method in DDI relation extraction. The result proves that CNN is a robust and well-deserved for DDI relation extraction.

With the rapid development of the mobile games, mobile game recommendation has become a core technique to mobile game marketplaces. This paper proposes a bipartite graph based recommendation algorithm PKBBR (Prior Knowledge Based for Bipartite Graph Rank). We model the user's interest in mobile game based on bipartite graph structure and use the users' mobile game behavior to describe the edge weights of the graph, then incorporate users' prior knowledge into the projection procedure of the bipartite graph to enrich the information among the nodes. Because the popular games have a great influence on mobile game marketplace, we design a hybrid recommendation algorithm to incorporate popularity recommendation based on users' behaviors. The experiment results show that this hybrid method could achieve a better performance than other approaches.

To well support the real-time requirement from applications, we enhance the real-time ability in Linux kernel through two new kernel mechanisms. Firstly, we present a new microsecond-level timer mechanism based on UTIME provided by Kurt-Linux, but different from it. The new timing implementation provides more flexible mechanism to support the different-grained timing requirement, as well as more flexible and preferential management of microsecond-level timers. Secondly, we present a new interrupt handler to reduce the kernel delay, which makes it possible scheduling interrupt handler with other processes under the system scheduler. Therefore, making interrupt routine under control is achieved. The experiments show that the improvements are distinct and significant.