As circular non-coding RNA (circRNA) is closely associated with various human diseases, identifying disease-related circRNAs can provide a deeper understanding of the mechanisms underlying disease pathogenesis. Advanced circRNA-disease association prediction methods mainly focus on graph learning techniques such as graph convolutional networks. However, these methods do not fully encode the multiscale neighbor topologies of each node, and the dependencies among the pairwise attributes. We propose a multi-scale neighbor topology-guided transformer with Kolmogorov-Arnold network (KAN) enhanced feature learning for circRNA and disease association prediction, termed MKCD. First, MKCD incorporates an adaptive multiscale neighbor topology embedding construction strategy (AMNE), which generates neighbor topologies covering varying scopes of neighbors by random walks. Second, we design a dynamic multi-scale neighbor topology-guided transformer (DMTT) that leverages the multi-scale neighbor topologies to guide the learning of relationships among circRNA, miRNA, and disease nodes. The multi-scale neighbor topology is dynamically evolved, providing adaptive guidance to the transformer's learning process. Third, we establish a feature-gated network (FGN) to evaluate the importance of topological features and the original node attributes. Finally, we propose an adaptive joint convolutional neural networks and KAN learning strategy (ACK) to learn the global and local dependencies of pairwise features. Comprehensive comparison experiments show that MKCD outperforms six state-of-the-art methods, improving AUC and AUPR by at least 14.1% and 7.6%, respectively. Ablation experiments further validate the effectiveness of AMNE, DMTT, FGN and ACK innovations. Case studies on three diseases further validate the application value of our method in discovering reliable circRNA candidates for the diseases.

This study explores the design space of two-handed input (i.e., clicking or tapping with the thumb) on the touchpads of controllers for virtual reality (VR) interaction. Four experiments were conducted to fulfill this purpose. Experiment 1 investigated how users employed two VR controllers to perform four representative interaction tasks in VR and identified 14 potentially usable two-handed operations that involved tapping or clicking. Experiments 2 and 3 analyzed user performance of the 14 operations, providing insights into their interaction characteristics in terms of completion time, accuracy, and subjective feedback. In Experiment 4, we designed a command-input technique based on the proposed operations. We verified its effectiveness compared to context menus and marking menus in a VR text entry scenario. Our technique generally had shorter times and similar accuracy to the two menu types. Our work contributes to the design of VR interactions using two-handed controllers.

Remote sensing images (RSIs) are frequently characterized by multi-scale inter-class objects and inconsistently distributed objects due to scene limitations, which would cause a significant data imbalance challenging the corresponding semantic segmentation. Recent methods have leveraged various deep learning techniques to capture high-quality representations for RSI semantic segmentation, but are hardly capable of addressing the afore-mentioned challenge given their limited explorations towards the mechanisms behind the representations. The recently discovered Neural Collapse (NC) phenomenon in computer vision models suggests the simplex equiangular tight frame (ETF) as the optimal representation structure, which has motivated us to observe that the optimal structure of last-layer representations is disrupted and inter-class representations for minor classes tend to become closer to each other beacuse of data imbalance. To address these issues, we propose Inter-class and Intra-class Neural Collapse Tuning (In2NeCT) to optimize the representations that satisfy the simplex ETF, which facilitates the discrimination of inter-class representations and the coherence of intra-class representations. Extensive experiments on three datasets demonstrate that our In2NeCT consistently leads to significant improvements in performance and outperforms the state-of-the-art methods.

Atopic dermatitis (AD) is a chronic inflammatory skin disorder characterized by recurrent itching, erythema, dryness, and eczematous lesions. Automated AD severity assessment is crucial for cost-effective and precision clinical decision-making but remains challenging. This is due to the subtle contrast variations between key dermatological signs and significant variations in lesion sizes across patients and disease stages. To address these issues, we propose ADSA-Net, which is designed to handle both intra- and inter-class variabilities. ADSA-Net first extracts multi-scale texture-aware features to effectively model variations in lesion size and texture. It then leverages contrastive learning to enhance intra- and inter-class differentiation, strengthening model's discriminatory ability for samples that are difficult to distinguish. Finally, ADSA-Net refines the learning process by leveraging a dynamic feature pool of correctly classified samples to guide the calibration of misclassified instances, enhancing overall accuracy. We further establish a dataset for AD severity assessment. Comprehensive experiments on this dataset show that ADSA-Net significantly outperforms existing state-of-the-art methods.

Radar imaging provides non-contact, privacy-preserving, and environmentally robust monitoring for continuous human motion recognition (HMR) by leveraging diverse information embedded in various radar signal domains. However, current research has not effectively integrated multi-radar and multi-domain imaging to fully exploit the benefits of distributed radar systems. To bridge this gap, we propose a multi-radar, multi-domain parallel cross-attention model with four key components: intra-domain cross-radar weight sharing encoders specific to each domain for consistent feature extraction and parameter reduction, domain-level parallel cross-attention (DLPCAN) modules to fuse domain-specific features and enhance feature representation robustness in each radar, a source-level attention fusion (SLAF) module to highlight significant features from multiple radar inputs, and two bi-directional gated recurrent unit (BiGRU) modules to capture temporal information. The model is trained using connectionist temporal classification (CTC) loss for effective sequence prediction. By integrating data from multiple radar nodes and domains, our approach significantly improves continuous HMR performance compared to single radar systems and single domain data. Comparative evaluations demonstrate that our model outperforms state-of-the-art radar imaging-based HMR solutions.

BACKGROUND: The human microbiome plays a crucial role in regulating the efficacy and toxicity of drugs as well as in developing the drugs. Therefore, predicting the drug-related microbes is beneficial for analyzing the functional mechanisms of drugs. Recently, the graph learning based methods demonstrated their advantages in extracting the node features from the biological heterogeneous graphs. However, the previous methods failed to completely preserve the intrinsic structures of biological data and did not fully utilize the topological and positional information for predicting the drug-microbe associations. RESULTS: We propose a new prediction model, structure-sensitive transformer and multi-view graph contrastive learning for microbe-drug association prediction (SMMDA), to encode and integrate the topological structures, semantics, and multiple-view embedding features of the drugs and microbes. Considering the sparsity of the original features of drugs and microbes, the learnable data augmentation strategy is designed to learn their global representations. Since similar drugs are more likely to associate with the similar microbes, a structure-sensitive transformer is proposed to integrate the topology structures composed of drugs (microbes) to form the multi-view embedding features. We design two contrastive learning strategies to exploit the complementary semantics across multiple views. As the embedding features from multiple views have various semantics, we design view-level attention to adaptively integrate these features. CONCLUSIONS: The extensive experimental results show that SMMDA outperforms several state-of-the-art methods for predicting the drug-related candidate microbes. The ablation studies show the effectiveness of the major innovations which include the learnable data augmentation, structure-sensitive transformer-based node feature learning, and multi-view contrastive learning. The case studies on three drugs also demonstrate SMMDA's capability in retrieving the potential microbe candidates for the drugs.

Transformers have shown great potential in vision tasks such as semantic segmentation. However, most of the existing transformer-based segmentation models neglect the cross-attention between pixel features and class features which impedes the application of transformers. Inspired by the concept of object queries in k-means Mask Transformer, we develop cluster learning and contrastive cluster assignment (CCA) for medical image segmentation in this paper. The cluster learning leverages the object queries to fit the feature-level cluster centers. The contrastive cluster assignment is introduced to guide the pixel class prediction using the cluster centers. Our method is a plug-in and can be integrated into any model. We design two networks for supervised segmentation tasks and semi-supervised segmentation tasks respectively. We equip the decoder with our proposed modules for the supervised segmentation to improve the pixel-level predictions. For the semi-supervised segmentation, we enhance the feature extraction capability of the encoder by using our proposed modules. We conduct comprehensive comparison and ablation experiments on public medical image datasets (ACDC, LA, Synapse, and ISIC2018), the results demonstrate that our proposed models outperform state-of-the-art models consistently, validating the effectiveness of our proposed method. The source code is accessible at https://github.com/zhujinghua1234/CCA-Seg.

NeRF registration has gained increasing attention recently. While existing research demonstrates considerable potential for this task, most methods primarily focus on either global geometric or rendering photometric information during feature learning, overlooking the rich cross-modal information inherent in the NeRF embedding feature space. In this paper, we propose DeSC, a novel NeRF registration approach that leverages the rich cross-modal features from NeRF to learn robust semantic descriptors. In particular, we propose a Deep Semantic Aggregation module, which employs a weighted graph convolution network to capture high-frequency texture details in NeRF patches. This approach reveals the underlying semantics shared across different NeRFs of the same scene, thereby yielding more robust global feature descriptors that lead to better alignment accuracy and robustness. In addition, we design a density-aware photometric consistency loss that facilitates the learning of robust features. Extensive experimental results on Objaverse datasets demonstrate that our approach produces superior registration performance to state-of-the-art techniques.

Medical image segmentation is essential for diagnosis but requires expensive and time-consuming labeled data. Semi-supervised learning (SSL) mitigates this issue by using unlabeled data to improve generalization. However, current SSL methods encounter issues with inadaptive perturbations and low-quality pseudo-labels. Vision foundation models, such as SAM, have shown promise in segmentation. We propose SESAME, an SSL method integrating SAM and U-Net to improve labeling accuracy through a foundation model-assisted pipeline. In particular, we introduce a reliability score to address low-quality pseudo-labels and employ strategies for utilizing both reliable and unreliable images. Reliable images are associated with refined pseudo-labels via a conflict resolving strategy, whereas unreliable ones undergo a mutual region swapping strategy. Extensive experiments demonstrate that our SESAME outperforms representative methods for medical image segmentation.

As a powerful microstructural imaging technique, neurite orientation dispersion and density imaging (NODDI) provides detailed insights into brain microstructures. Its clinical application is often restricted by the necessity for high-quality scanning, which can be challenging to achieve in practical settings. To overcome this limitation, we propose an innovative 3D conditional latent diffusion model (3D-CLDM) to generate high-quality NODDI index maps from low-resolution diffusion magnetic resonance imaging data. The 3D-CLDM is a two-stage super-resolved microstructure estimation model that includes training a vector quantized generative adversarial network and a diffusion model. It leverages the sophisticated high-dimensional data modeling capabilities of the conditional latent diffusion model to effectively capture and represent intricate microstructural features that are difficult to detect with conventional techniques. We conducted comprehensive experiments using data from the human connectome project to rigorously assess our model’s performance. The results reveal that our approach not only significantly improves the quality of super-resolved microstructural estimation but also surpasses current state-of-the-art models in both qualitative and quantitative evaluations. This highlights the potential of 3D-CLDM to advance brain microstructure imaging, making it more feasible and effective for clinical applications.

Molecular subtyping of cancer is recognized as a critical and challenging upstream task for personalized therapy. Existing deep learning methods have achieved significant performance in this domain when abundant data samples are available. However, the acquisition of densely labeled samples for cancer molecular subtypes remains a significant challenge for conventional data-intensive deep learning approaches. In this work, we focus on the few-shot molecular subtype prediction problem in heterogeneous and small cancer datasets, aiming to enhance precise diagnosis and personalized treatment. We first construct a new few-shot dataset for cancer molecular subtype classification and auxiliary cancer classification, named TCGA Few-Shot, from existing publicly available datasets. To effectively leverage the relevant knowledge from both tasks, we introduce a task-specific embedding-based meta-learning framework (TSEML). TSEML leverages the synergistic strengths of a model-agnostic meta-learning (MAML) approach and a prototypical network (ProtoNet) to capture diverse and fine-grained features. Comparative experiments conducted on the TCGA FewShot dataset demonstrate that our TSEML framework achieves superior performance in addressing the problem of few-shot molecular subtype classification.

Gliomas hold a prominent position in neurooncology due to their high malignancy and poor survival rates. Accurately predicting the prognosis and survival risk of glioma patients is crucial for clinical treatment. Recent advances in survival prediction methods emphasize the importance of integrating complementary information from diverse modalities while neglecting the significant modality gap between pathological images and genomic data. To address this issue, we propose a modality-specific knowledge interaction network (MSKI-Net), which integrates whole slide images (WSI), RNA-Seq gene expression data, and copy number variation (CNV) data for glioma survival analysis. The MSKI-Net consists of a modality-specific feature enhancement (MSFE) module, a modality-interactive cross-attention (MICA) module, and a modality-specific knowledge-guided representation learning (MSKR) module. The three modules collaborate by complementing modality-specific features with modality-agnostic knowledge to improve the learning capability of MSKI-Net. Furthermore, we construct a dataset named TCGAmm, which combines WSI, RNA-Seq, and CNV data from The Cancer Genome Atlas (TCGA) to address the issue of data scarcity. Extensive experiments demonstrate that MSKI-Net achieves superior performance in predicting the survival risk of glioma cancer.

MRI images contain multi-modal information, introducing complexity to brain tumor segmentation. Recent studies have incorporated the Transformer model, given its exceptional capability to model long-range dependence, into convolutional neural networks (CNNs) to address limited receptive fields. However, such a hybrid strategy often neglects the inherent multimodal characteristics of MRI images and lacks the capacity to capture modality-specific features. In this paper, we propose a multi-modal and multi-scale feature fusion network (MuMoSNet) for brain tumor segmentation from 3D MRI images. Specifically, our MuMoSNet introduces a parallel ME-Transformer encoder alongside the CNN-based encoder in 3D U-Net to separately extract modality-specific features. Besides, we devise a multi-feature fusion (MuFF) module to learn affinity relationships between cross-modality shared features and modality-specific features, maximizing the exploration of multi-modal information. Extensive experiments on both BraTS21 and BraTS20 datasets suggest that our MuMoSNet outperforms current representative methods for brain tumor segmentation.

MOTIVATION: The human microbiome may impact the effectiveness of drugs by modulating their activities and toxicities. Predicting candidate microbes for drugs can facilitate the exploration of the therapeutic effects of drugs. Most recent methods concentrate on constructing of the prediction models based on graph reasoning. They fail to sufficiently exploit the topology and position information, the heterogeneity of multiple types of nodes and connections, and the long-distance correlations among nodes in microbe-drug heterogeneous graph. RESULTS: We propose a new microbe-drug association prediction model, NGMDA, to encode the position and topological features of microbe (drug) nodes, and fuse the different types of features from neighbors and the whole heterogeneous graph. First, we formulate the position and topology features of microbe (drug) nodes by t-step random walks, and the features reveal the topological neighborhoods at multiple scales and the position of each node. Second, as the features of nodes are high-dimensional and sparse, we designed an embedding enhancement strategy based on supervised fully connected autoencoders to form the embeddings with representative features and the more discriminative node distributions. Third, we propose an adaptive neighbor feature fusion module, which fuses features of neighbors by the constructed position- and topology-sensitive heterogeneous graph neural networks. A novel self-attention mechanism is developed to estimate the importance of the position and topology of each neighbor to a target node. Finally, a heterogeneous graph feature fusion module is constructed to learn the long-distance correlations among the nodes in the whole heterogeneous graph by a relationship-aware graph transformer. Relationship-aware graph transformer contains the strategy for encoding the connection relationship types among the nodes, which is helpful for integrating the diverse semantics of these connections. The extensive comparison experimental results demonstrate NGMDA's superior performance over five state-of-the-art prediction methods. The ablation experiment shows the contributions of the multi-scale topology and position feature learning, the embedding enhancement strategy, the neighbor feature fusion, and the heterogeneous graph feature fusion. Case studies over three drugs further indicate that NGMDA has ability in discovering the potential drug-related microbes. AVAILABILITY AND IMPLEMENTATION: Source codes and Supplementary Material are available at https://github.com/pingxuan-hlju/NGMDA.

MOTIVATION: The microbes in human body play a crucial role in influencing the functions of drugs, as they can regulate the activities and toxicities of drugs. Most recent methods for predicting drug-microbe associations are based on graph learning. However, the relationships among multiple drugs and microbes are complex, diverse, and heterogeneous. Existing methods often fail to fully model the relationships. In addition, the attributes of drug-microbe pairs exhibit long-distance spatial correlations, which previous methods have not integrated effectively. RESULTS: We propose a new prediction method named DHDMP which is designed to encode the relationships among multiple drugs and microbes and integrate the attributes of various neighbor nodes along with the pairwise long-distance correlations. First, we construct a hypergraph with dynamic topology, where each hyperedge represents a specific relationship among multiple drug nodes and microbe nodes. Considering the heterogeneity of node attributes across different categories, we developed a node category-sensitive hypergraph convolution network to encode these diverse relationships. Second, we construct homogeneous graphs for drugs and microbes respectively, as well as drug-microbe heterogeneous graph, facilitating the integration of features from both homogeneous and heterogeneous neighbors of each target node. Third, we introduce a graph convolutional network with cross-graph feature propagation ability to transfer node features from homogeneous to heterogeneous graphs for enhanced neighbor feature representation learning. The propagation strategy aids in the deep fusion of features from both types of neighbors. Finally, we design spatial cross-attention to encode the attributes of drug-microbe pairs, revealing long-distance correlations among multiple pairwise attribute patches. The comprehensive comparison experiments showed our method outperformed state-of-the-art methods for drug-microbe association prediction. The ablation studies demonstrated the effectiveness of node category-sensitive hypergraph convolution network, graph convolutional network with cross-graph feature propagation, and spatial cross-attention. Case studies on three drugs further showed DHDMP's potential application in discovering the reliable candidate microbes for the interested drugs. AVAILABILITY AND IMPLEMENTATION: Source codes and supplementary materials are available at https://github.com/pingxuan-hlju/DHDMP.

Dysregulation of miRNAs is closely related to the progression of various diseases, so identifying disease-related miRNAs is crucial. Most recently proposed methods are based on graph reasoning, while they did not completely exploit the topological structure composed of the higher-order neighbor nodes and the global and local features of miRNA and disease nodes. We proposed a prediction method, MDAP, to learn semantic features of miRNA and disease nodes based on various meta-paths, as well as node features from the entire heterogeneous network perspective, and node pair attributes. Firstly, for both the miRNA and disease nodes, node category-wise meta-paths were constructed to integrate the similarity and association connection relationships. Each target node has its specific neighbor nodes for each meta-path, and the neighbors of longer meta-paths constitute its higher-order neighbor topological structure. Secondly, we constructed a meta-path specific graph convolutional network module to integrate the features of higher-order neighbors and their topology, and then learned the semantic representations of nodes. Thirdly, for the entire miRNA-disease heterogeneous network, a global-aware graph convolutional autoencoder was built to learn the network-view feature representations of nodes. We also designed semantic-level and representation-level attentions to obtain informative semantic features and node representations. Finally, the strategy based on the parallel convolutional-deconvolutional neural networks were designed to enhance the local feature learning for a pair of miRNA and disease nodes. The experiment results showed that MDAP outperformed other state-of-the-art methods, and the ablation experiments demonstrated the effectiveness of MDAP's major innovations. MDAP's ability in discovering potential disease-related miRNAs was further analyzed by the case studies over three diseases.

Cancer is widely recognized as the primary cause of mortality worldwide, and pathology analysis plays a pivotal role in achieving accurate cancer diagnosis. The intricate representation of features in histopathological images encompasses abundant information crucial for disease diagnosis, regarding cell appearance, tumor microenvironment, and geometric characteristics. However, recent deep learning methods have not adequately exploited geometric features for pathological image classification due to the absence of effective descriptors that can capture both cell distribution and gathering patterns, which often serve as potent indicators. In this paper, inspired by clinical practice, a Hierarchical Graph Pyramid Transformer (HGPT) is proposed to guide pathological image classification by effectively exploiting a geometric representation of tissue distribution which was ignored by existing state-of-the-art methods. First, a graph representation is constructed according to morphological feature of input pathological image and learn geometric representation through the proposed multi-head graph aggregator. Then, the image and its graph representation are feed into the transformer encoder layer to model long-range dependency. Finally, a locality feature enhancement block is designed to enhance the 2D local representation of feature embedding, which is not well explored in the existing vision transformers. An extensive experimental study is conducted on Kather-5K, MHIST, NCT-CRC-HE, and GasHisSDB for binary or multi-category classification of multiple cancer types. Results demonstrated that our method is capable of consistently reaching superior classification outcomes for histopathological images, which provide an effective diagnostic tool for malignant tumors in clinical practice.

Accurate tumor segmentation is crucial for esophageal cancer radiotherapy treatment planning. The low contrast among the esophagus, tumors, and surrounding tissues, and irregular tumor shapes limit the performance of automatic segmentation methods. In this paper, we aim to exploit the irregular shapes of tumors to facilitate accurate segmentation. We propose a simple and pluggable shape-aware contrastive deep supervision network (SCDSNet) with shape-aware regularization and voxel-to-voxel contrastive deep supervision. Specifically, the shape-aware regularization with an uncertainty minimization strategy encourages the precise predictions of an additional shape-aware head. The voxel-to-voxel contrastive deep supervision enhances the multi-scale shape-tumor contrast for better voxel-to-voxel prediction of shapes. The proposed method is simple and highly pluggable, which can easily be extended to other frameworks. Further, we establish a large in-house dataset on esophageal cancer to validate the effectiveness of our proposed method. The quantitative and qualitative experimental results demonstrate the effectiveness of SCDSNet on the esophageal cancer dataset.

The rapidly emerging field of deep learning-based computational pathology has shown promising results in utilizing whole slide images (WSIs) to objectively prognosticate cancer patients. However, most prognostic methods are currently limited to either histopathology or genomics alone, which inevitably reduces their potential to accurately predict patient prognosis. Whereas integrating WSIs and genomic features presents three main challenges: (1) the enormous heterogeneity of gigapixel WSIs which can reach sizes as large as 150,000×150,000 pixels; (2) the absence of a spatially corresponding relationship between histopathology images and genomic molecular data; and (3) the existing early, late, and intermediate multimodal feature fusion strategies struggle to capture the explicit interactions between WSIs and genomics. To ameliorate these issues, we propose the Mutual-Guided Cross-Modality Transformer (MGCT), a weakly-supervised, attention-based multimodal learning framework that can combine histology features and genomic features to model the genotype-phenotype interactions within the tumor microenvironment. To validate the effectiveness of MGCT, we conduct experiments using nearly 3,600 gigapixel WSIs across five different cancer types sourced from The Cancer Genome Atlas (TCGA). Extensive experimental results consistently emphasize that MGCT outperforms the state-of-the-art (SOTA) methods.

Identifying enhancers is a critical task in bioinformatics due to their primary role in regulating gene expression. For this reason, various computational algorithms devoted to enhancer identification have been put forward over the years. More features are extracted from the single DNA sequences to boost the performance. Nevertheless, DNA structural information is neglected, which is an essential factor affecting the binding preferences of transcription factors to regulatory elements like enhancers. Here, we propose SENIES, a DNA shape enhanced deep learning predictor, to identify enhancers and their strength. The predictor consists of two layers where the first layer is for enhancer and non-enhancer identification, and the second layer is for predicting the strength of enhancers. Apart from two common sequence-derived features (i.e., one-hot and k-mer), DNA shape is introduced to describe the 3D structures of DNA sequences. Performance comparison with state-of-the-art methods conducted on public datasets demonstrates the effectiveness and robustness of our predictor. The code implementation of SENIES is publicly available at https://github.com/hlju-liye/SENIES.

Since abnormal expression of long non-coding RNAs (lncRNAs) is associated with various human diseases, identifying disease-related lncRNAs helps reveal the pathogenesis of diseases. Existing methods for lncRNA-disease association prediction mainly focus on multi-sourced data related to lncRNAs and diseases. The rich semantic information of meta-paths, composed of multiple kinds of connections between lncRNA and disease nodes, is neglected. We propose a new prediction method, MGLDA, to encode and integrate the semantics of multiple meta-paths, the global topology of heterogeneous graph, and pairwise attributes of lncRNA and disease nodes. First, a tri-layer heterogeneous graph is constructed to associate multi-sourced data across the lncRNA, disease, and miRNA nodes. Afterwards, we establish multiple meta-paths connecting the lncRNA and disease nodes to derive and denote various semantics. Each meta-path contains its specific semantics formulated by an embedding strategy, and each embedding covers local topology formed by the diverse semantic connections among the lncRNA, disease, and miRNA nodes. We construct multiple graph convolutional autoencoders (GCA) with topology-level attention to learn global and multiple local topologies from the tri-layer graph and each meta-path, respectively. The topology-level attention mechanism can learn the importance of various global and local topologies for adaptive pairwise topology fusion. Finally, a convolutional autoencoder learns the attribute representations of lncRNA-disease pairs, which integrates the learnt detailed and representative pairwise features. Experimental results show that MGLDA outperforms other state-of-the-art prediction methods in comparison and retrieves more real lncRNA-disease associations in the top-ranked candidates. The ablation study also demonstrates the important contributions of the local and global topology learning, and pairwise attribute learning. Case studies on three diseases further demonstrate MGLDA's ability to identify potential disease-related lncRNAs.

Simulator sickness induced by 360° stereoscopic video contents is a prolonged challenging issue in Virtual Reality (VR) system. Current machine learning models for simulator sickness prediction ignore the underlying interdependencies and correlations across multiple visual features which may lead to simulator sickness. We propose a model for sickness prediction by automatic learning and adaptive integrating multi-level mappings from stereoscopic video features to simulator sickness scores. Firstly, saliency, optical flow and disparity features are extracted from videos to reflect the factors causing simulator sickness, including human attention area, motion velocity and depth information. Then, these features are embedded and fed into a 3-dimensional convolutional neural network (3D CNN) to extract the underlying multi-level knowledge which includes low-level and higher-order visual concepts, and global image descriptor. Finally, an attentional mechanism is exploited to adaptively fuse multi-level information with attentional weights for sickness score estimation. The proposed model is trained by an end-to-end approach and validated over a public dataset. Comparison results with state-of-the-art models and ablation studies demonstrated improved performance in terms of Root Mean Square Error (RMSE) and Pearson Linear Correlation Coefficient.

MOTIVATION: The development process of a new drug is time-consuming and costly. Thus, identifying new uses for approved drugs, named drug repositioning, is helpful for speeding up the drug development process and reducing development costs. Existing drug-related disease prediction methods mainly focus on single or multiple drug-disease heterogeneous networks. However, heterogeneous networks, and drug subnets and disease subnet contained in heterogeneous networks cover the common topology information between drug and disease nodes, the specific information between drug nodes and the specific information between disease nodes, respectively. RESULTS: We design a novel model, CTST, to extract and integrate common and specific topologies in multiple heterogeneous networks and subnets. Multiple heterogeneous networks composed of drug and disease nodes are established to integrate multiple kinds of similarities and associations among drug and disease nodes. These heterogeneous networks contain multiple drug subnets and a disease subnet. For multiple heterogeneous networks and subnets, we then define the common and specific representations of drug and disease nodes. The common representations of drug and disease nodes are encoded by a graph convolutional autoencoder with sharing parameters and they integrate the topological relationships of all nodes in heterogeneous networks. The specific representations of nodes are learned by specific graph convolutional autoencoders, respectively, and they fuse the topology and attributes of the nodes in each subnet. We then propose attention mechanisms at common representation level and specific representation level to learn more informative common and specific representations, respectively. Finally, an integration module with representation feature level attention is built to adaptively integrate these two representations for final association prediction. Extensive experimental results confirm the effectiveness of CTST. Comparison with six latest methods and case studies on five drugs further verify CTST has the ability to discover potential candidate diseases.

MOTIVATION: Effective computational methods to predict drug-protein interactions (DPIs) are vital for drug discovery in reducing the time and cost of drug development. Recent DPI prediction methods mainly exploit graph data composed of multiple kinds of connections among drugs and proteins. Each node in the graph usually has topological structures with multiple scales formed by its first-order neighbors and multi-order neighbors. However, most of the previous methods do not consider the topological structures of multi-order neighbors. In addition, deep integration of the multi-modality similarities of drugs and proteins is also a challenging task. RESULTS: We propose a model called ALDPI to adaptively learn the multi-scale topologies and multi-modality similarities with various significance levels. We first construct a drug-protein heterogeneous graph, which is composed of the interactions and the similarities with multiple modalities among drugs and proteins. An adaptive graph learning module is then designed to learn important kinds of connections in heterogeneous graph and generate new topology graphs. A module based on graph convolutional autoencoders is established to learn multiple representations, which imply the node attributes and multiple-scale topologies composed of one-order and multi-order neighbors, respectively. We also design an attention mechanism at neighbor topology level to distinguish the importance of these representations. Finally, since each similarity modality has its specific features, we construct a multi-layer convolutional neural network-based module to learn and fuse multi-modality features to obtain the attribute representation of each drug-protein node pair. Comprehensive experimental results show ALDPI's superior performance over six state-of-the-art methods. The results of recall rates of top-ranked candidates and case studies on five drugs further demonstrate the ability of ALDPI to discover potential drug-related protein candidates. CONTACT: [email protected].

MOTIVATION: Identifying proteins that interact with drugs plays an important role in the initial period of developing drugs, which helps to reduce the development cost and time. Recent methods for predicting drug-protein interactions mainly focus on exploiting various data about drugs and proteins. These methods failed to completely learn and integrate the attribute information of a pair of drug and protein nodes and their attribute distribution. RESULTS: We present a new prediction method, GVDTI, to encode multiple pairwise representations, including attention-enhanced topological representation, attribute representation and attribute distribution. First, a framework based on graph convolutional autoencoder is constructed to learn attention-enhanced topological embedding that integrates the topology structure of a drug-protein network for each drug and protein nodes. The topological embeddings of each drug and each protein are then combined and fused by multi-layer convolution neural networks to obtain the pairwise topological representation, which reveals the hidden topological relationships between drug and protein nodes. The proposed attribute-wise attention mechanism learns and adjusts the importance of individual attribute in each topological embedding of drug and protein nodes. Secondly, a tri-layer heterogeneous network composed of drug, protein and disease nodes is created to associate the similarities, interactions and associations across the heterogeneous nodes. The attribute distribution of the drug-protein node pair is encoded by a variational autoencoder. The pairwise attribute representation is learned via a multi-layer convolutional neural network to deeply integrate the attributes of drug and protein nodes. Finally, the three pairwise representations are fused by convolutional and fully connected neural networks for drug-protein interaction prediction. The experimental results show that GVDTI outperformed other seven state-of-the-art methods in comparison. The improved recall rates indicate that GVDTI retrieved more actual drug-protein interactions in the top ranked candidates than conventional methods. Case studies on five drugs further confirm GVDTI's ability in discovering the potential candidate drug-related proteins. CONTACT: [email protected] Supplementary information: Supplementary data are available at Briefings in Bioinformatics online.

Since abnormal expression of long noncoding RNAs (lncRNAs) is often closely related to various human diseases, identification of disease-associated lncRNAs is helpful for exploring the complex pathogenesis. Most of recent methods concentrate on exploiting multiple kinds of data related to lncRNAs and diseases for predicting candidate disease-related lncRNAs. These methods, however, failed to deeply integrate the topology information from the meta-paths that are composed of lncRNA, disease and microRNA (miRNA) nodes. We proposed a new method based on fully connected autoencoders and convolutional neural networks, called ACLDA, for inferring potential disease-related lncRNA candidates. A heterogeneous graph that consists of lncRNA, disease and miRNA nodes were firstly constructed to integrate similarities, associations and interactions among them. Fully connected autoencoder-based module was established to extract the low-dimensional features of lncRNA, disease and miRNA nodes in the heterogeneous graph. We designed the attention mechanisms at the node feature level and at the meta-path level to learn more informative features and meta-paths. A module based on convolutional neural networks was constructed to encode the local topologies of lncRNA and disease nodes from multiple meta-path perspectives. The comprehensive experimental results demonstrated ACLDA achieves superior performance than several state-of-the-art prediction methods. Case studies on breast, lung and colon cancers demonstrated that ACLDA is able to discover the potential disease-related lncRNAs.

Identifying disease-related microRNAs (miRNAs) assists the understanding of disease pathogenesis. Existing research methods integrate multiple kinds of data related to miRNAs and diseases to infer candidate disease-related miRNAs. The attributes of miRNA nodes including their family and cluster belonging information, however, have not been deeply integrated. Besides, the learning of neighbor topology representation of a pair of miRNA and disease is a challenging issue. We present a disease-related miRNA prediction method by encoding and integrating multiple representations of miRNA and disease nodes learnt from the generative and adversarial perspective. We firstly construct a bilayer heterogeneous network of miRNA and disease nodes, and it contains multiple types of connections among these nodes, which reflect neighbor topology of miRNA-disease pairs, and the attributes of miRNA nodes, especially miRNA-related families and clusters. To learn enhanced pairwise neighbor topology, we propose a generative and adversarial model with a convolutional autoencoder-based generator to encode the low-dimensional topological representation of the miRNA-disease pair and multi-layer convolutional neural network-based discriminator to discriminate between the true and false neighbor topology embeddings. Besides, we design a novel feature category-level attention mechanism to learn the various importance of different features for final adaptive fusion and prediction. Comparison results with five miRNA-disease association methods demonstrated the superior performance of our model and technical contributions in terms of area under the receiver operating characteristic curve and area under the precision-recall curve. The results of recall rates confirmed that our model can find more actual miRNA-disease associations among top-ranked candidates. Case studies on three cancers further proved the ability to detect potential candidate miRNAs.

MOTIVATION: Long noncoding RNAs (lncRNAs) play an important role in the occurrence and development of diseases. Predicting disease-related lncRNAs can help to understand the pathogenesis of diseases deeply. The existing methods mainly rely on multi-source data related to lncRNAs and diseases when predicting the associations between lncRNAs and diseases. There are interdependencies among node attributes in a heterogeneous graph composed of all lncRNAs, diseases and micro RNAs. The meta-paths composed of various connections between them also contain rich semantic information. However, the existing methods neglect to integrate attribute information of intermediate nodes in meta-paths. RESULTS: We propose a novel association prediction model, GSMV, to learn and deeply integrate the global dependencies, semantic information of meta-paths and node-pair multi-view features related to lncRNAs and diseases. We firstly formulate the global representations of the lncRNA and disease nodes by establishing a self-attention mechanism to capture and learn the global dependencies among node attributes. Second, starting from the lncRNA and disease nodes, respectively, multiple meta-pathways are established to reveal different semantic information. Considering that each meta-path contains specific semantics and has multiple meta-path instances which have different contributions to revealing meta-path semantics, we design a graph neural network based module which consists of a meta-path instance encoding strategy and two novel attention mechanisms. The proposed meta-path instance encoding strategy is used to learn the contextual connections between nodes within a meta-path instance. One of the two new attention mechanisms is at the meta-path instance level, which learns rich and informative meta-path instances. The other attention mechanism integrates various semantic information from multiple meta-paths to learn the semantic representation of lncRNA and disease nodes. Finally, a dilated convolution-based learning module with adjustable receptive fields is proposed to learn multi-view features of lncRNA-disease node pairs. The experimental results prove that our method outperforms seven state-of-the-art comparing methods for lncRNA-disease association prediction. Ablation experiments demonstrate the contributions of the proposed global representation learning, semantic information learning, pairwise multi-view feature learning and the meta-path instance encoding strategy. Case studies on three cancers further demonstrate our method's ability to discover potential disease-related lncRNA candidates. CONTACT: [email protected] or [email protected]. SUPPLEMENTARY INFORMATION: Supplementary data are available at Briefings in Bioinformatics online.

MOTIVATION: Identifying new therapeutic effects for the approved drugs is beneficial for effectively reducing the drug development cost and time. Most of the recent computational methods concentrate on exploiting multiple kinds of information about drugs and disease to predict the candidate associations between drugs and diseases. However, the drug and disease nodes have neighboring topologies with multiple scales, and the previous methods did not fully exploit and deeply integrate these topologies. RESULTS: We present a prediction method, multi-scale topology learning for drug-disease (MTRD), to integrate and learn multi-scale neighboring topologies and the attributes of a pair of drug and disease nodes. First, for multiple kinds of drug similarities, multiple drug-disease heterogenous networks are constructed respectively to integrate the similarities and associations related to drugs and diseases. Moreover, each heterogenous network has its specific topology structure, which is helpful for learning the corresponding specific topology representation. We formulate the topology embeddings for each drug node and disease node by random walking on each heterogeneous network, and the embeddings cover the neighboring topologies with different scopes. Because the multi-scale topology embeddings have context relationships, we construct Bi-directional long short-term memory-based module to encode these embeddings and their relationships and learn the neighboring topology representation. We also design the attention mechanisms at feature level and at scale level to obtain the more informative pairwise features and topology embeddings. A module based on multi-layer convolutional networks is constructed to learn the representative attributes of the drug-disease node pair according to their related similarity and association information. Comprehensive experimental results indicate that MTRD achieves the superior performance than several state-of-the-art methods for predicting drug-disease associations. MTRD also retrieves more actual drug-disease associations in the top-ranked candidates of the prediction result. Case studies on five drugs further demonstrate MTRD's ability in discovering the potential candidate diseases for the interested drugs.

Identifying new disease indications for the approved drugs can help reduce the cost and time of drug development. Most of the recent methods focus on exploiting the various information related to drugs and diseases for predicting the candidate drug-disease associations. However, the previous methods failed to deeply integrate the neighborhood topological structure and the node attributes of an interested drug-disease node pair. We propose a new prediction method, ANPred, to learn and integrate pairwise attribute information and neighbor topology information from the similarities and associations related to drugs and diseases. First, a bi-layer heterogeneous network with intra-layer and inter-layer connections is established to combine the drug similarities, the disease similarities, and the drug-disease associations. Second, the embedding of a pair of drug and disease is constructed based on integrating multiple biological premises about drugs and diseases. The learning framework based on multi-layer convolutional neural networks is designed to learn the attribute representation of the pair of drug and disease nodes from its embedding. The sequences composed of neighbor nodes are formed based on random walk on the heterogeneous network. A framework based on fully-connected autoencoder and skip-gram module is constructed to learn the neighbor topological representations of nodes. The cross-validation results indicate the performance of ANPred is superior to several state-of-the-art methods. The case studies on 5 drugs further confirm the ability of ANPred in discovering the potential drug-disease association candidates.

Identification of interactions between drugs and target proteins plays a critical role not only in drug discovery but also in drug repositioning. Deep integration of inter-connections and intra-similarities between heterogeneous multi-source data about drugs and targets, however, is a challenging issue. We propose a drug-target interaction (DTI) prediction model by learning from drug and protein related multi-scale attributes and global topology formed by heterogeneous connections. A drug-protein-disease heterogeneous network (RPD-Net) is firstly constructed to associate diverse similarities, interactions and associations across nodes. Secondly, we propose a multi-scale pairwise deep representation learning module consisting of a new embedding strategy to integrate diverse inter-relations and intra-relations, and dilation convolutions for multi-scale deep representation extraction. A global topology learning module is proposed which is composed of strategy based on non-negative matrix factorization (NMF) to extract topology from RPD-Net, and a new relational-level attention mechanism for discriminative topology embedding. Experimental results using public dataset demonstrate improved performance over state-of-the-art methods and contributions of our major innovations. Evaluation results by top k recall rates and case studies on five drugs further show the effectiveness of our method in retrieving potential target candidates for drugs.

Abnormal expressions of long non-coding RNAs (lncRNAs) are associated with various human diseases. Identifying disease-related lncRNAs can help clarify complex disease pathogeneses. The latest methods for lncRNA-disease association prediction rely on diverse data about lncRNAs and diseases. These methods, however, cannot adequately integrate the neighbour topological information of lncRNA and disease nodes. Moreover, more intrinsic features of lncRNA-disease node pairs can be explored to better predict their latent associations. We developed a novel method, named GTAN, to predict the association propensities between lncRNAs and diseases. GTAN integrates various information about lncRNAs and diseases, and exploits neighbour topology and attribute representations of a pair of lncRNA-disease nodes. We adopted in GTAN a graph neural network architecture with three attention mechanisms and multi-layer convolutional neural networks. First, a neighbour-level self-attention mechanism is constructed to learn the importance of each neighbour for an interested lncRNA or disease node. Second, topology-level attention is proposed to enhance contextual dependencies among multiple local topology representations. An attention-enhanced graph neural network framework is then established to learn a topology representation of top-ranked neighbours. GTAN also has attribute-level attention to distinguish various contributions of attributes of the lncRNA-disease pair. Finally, attribute representation is learned by multi-layer CNN to integrate detailed features and representative features of the pair. Extensive experimental results demonstrated that GTAN outperformed state-of-the-art methods. The ablation studies confirmed the important contributions of three attention mechanisms. Case studies on three cancers further showed GTAN's ability in discovering potential lncRNA candidates related to diseases.

How to exploit useful features to enhance the quality of blurred images is a long-standing topic in single image deblurring. Existing learning-based approaches show exciting performance by increasing the receptive fields depending on multi-scale and scale recurrent strategy. However, it is still a challenging task for deblurring to enlarge the receptive field only relying on increasing the number of layers of a neural network. To tackle this challenge, we propose a multi-scale spatial and edge attention enhanced model (MSEA) for image deblurring. Firstly, edge features are extracted to guide the network's attention to the recovery of fine details and texture information. Then we introduce spatial attention fusion mechanism for the adaptive fusion of features derived from edge maps and blurry images, and those representing shallow fine-grained details and in-depth abstract features. Qualitative and quantitative evaluation results over GoPro and VideoDeblurring datasets demonstrated the improved performance, especially when there are sharp edges and rich textures.

As the abnormalities of long non-coding RNAs (lncRNAs) are closely related to various human diseases, identifying disease-related lncRNAs is important for understanding the pathogenesis of complex diseases. Most of current data-driven methods for disease-related lncRNA candidate prediction are based on diseases and lncRNAs. Those methods, however, fail to consider the deeply embedded node attributes of lncRNA-disease pairs, which contain multiple relations and representations across lncRNAs, diseases and miRNAs. Moreover, the low-dimensional feature distribution at the pairwise level has not been taken into account. We propose a prediction model, VADLP, to extract, encode and adaptively integrate multi-level representations. Firstly, a triple-layer heterogeneous graph is constructed with weighted inter-layer and intra-layer edges to integrate the similarities and correlations among lncRNAs, diseases and miRNAs. We then define three representations including node attributes, pairwise topology and feature distribution. Node attributes are derived from the graph by an embedding strategy to represent the lncRNA-disease associations, which are inferred via their common lncRNAs, diseases and miRNAs. Pairwise topology is formulated by random walk algorithm and encoded by a convolutional autoencoder to represent the hidden topological structural relations between a pair of lncRNA and disease. The new feature distribution is modeled by a variance autoencoder to reveal the underlying lncRNA-disease relationship. Finally, an attentional representation-level integration module is constructed to adaptively fuse the three representations for lncRNA-disease association prediction. The proposed model is tested over a public dataset with a comprehensive list of evaluations. Our model outperforms six state-of-the-art lncRNA-disease prediction models with statistical significance. The ablation study showed the important contributions of three representations. In particular, the improved recall rates under different top $k$ values demonstrate that our model is powerful in discovering true disease-related lncRNAs in the top-ranked candidates. Case studies of three cancers further proved the capacity of our model to discover potential disease-related lncRNAs.

MOTIVATION: Identifying the proteins that interact with drugs can reduce the cost and time of drug development. Existing computerized methods focus on integrating drug-related and protein-related data from multiple sources to predict candidate drug-target interactions (DTIs). However, multi-scale neighboring node sequences and various kinds of drug and protein similarities are neither fully explored nor considered in decision making. RESULTS: We propose a drug-target interaction prediction method, DTIP, to encode and integrate multi-scale neighbouring topologies, multiple kinds of similarities, associations, interactions related to drugs and proteins. We firstly construct a three-layer heterogeneous network to represent interactions and associations across drug, protein, and disease nodes. Then a learning framework based on fully-connected autoencoder is proposed to learn the nodes' low-dimensional feature representations within the heterogeneous network. Secondly, multi-scale neighbouring sequences of drug and protein nodes are formulated by random walks. A module based on bidirectional gated recurrent unit is designed to learn the neighbouring sequential information and integrate the low-dimensional features of nodes. Finally, we propose attention mechanisms at feature level, neighbouring topological level and similarity level to learn more informative features, topologies and similarities. The prediction results are obtained by integrating neighbouring topologies, similarities and feature attributes using a multiple layer CNN. Comprehensive experimental results over public dataset demonstrated the effectiveness of our innovative features and modules. Comparison with other state-of-the-art methods and case studies of five drugs further validated DTIP's ability in discovering the potential candidate drug-related proteins.

Imbalance data classification is a challenging task in automatic seizure detection from electroencephalogram (EEG) recordings when the durations of non-seizure periods are much longer than those of seizure activities. An imbalanced learning model is proposed in this paper to improve the identification of seizure events in long-term EEG signals. To better represent the underlying microstructure distributions of EEG signals while preserving the non-stationary nature, discrete wavelet transform (DWT) and uniform 1D-LBP feature extraction procedure are introduced. A learning framework is then designed by the ensemble of weakly trained support vector machines (SVMs). Under-sampling is employed to split the imbalanced seizure and non-seizure samples into multiple balanced subsets where each of them is utilized to train an individual SVM classifier. The weak SVMs are incorporated to build a strong classifier which emphasizes seizure samples and in the meantime analyzing the imbalanced class distribution of EEG data. Final seizure detection results are obtained in a multi-level decision fusion process by considering temporal and frequency factors. The model was validated over two long-term and one short-term public EEG databases. The model achieved a [Formula: see text]-mean of 97.14% with respect to epoch-level assessment, an event-level sensitivity of 96.67%, and a false detection rate of 0.86/h on the long-term intracranial database. An epoch-level [Formula: see text]-mean of 95.28% and event-level false detection rate of 0.81/h were yielded over the long-term scalp database. The comparisons with 14 published methods demonstrated the improved detection performance for imbalanced EEG signals and the generalizability of the proposed model.

Accurate recognition of modern and traditional porcelain styles is a challenging issue in Cantonese porcelain management due to the large variety and complex elements and patterns. We propose a hybrid system with porcelain style identification and image recreation modules. In the identification module, prediction of an unknown porcelain sample is obtained by logistic regression of ensembled neural networks of top-ranked design signatures, which are obtained by discriminative analysis and transformed features in principal components. The synthesis module is developed based on a conditional generative adversarial network, which enables users to provide a designed mask with porcelain elements to generate synthesized images of Cantonese porcelain. Experimental results of 603 Cantonese porcelain images demonstrate that the proposed model outperforms other methods relative to precision, recall, area under curve of receiver operating characteristic, and confusion matrix. Case studies on image creation indicate that the proposed system has the potential to engage the community in understanding Cantonese porcelain and promote this intangible cultural heritage.

Clustering is increasingly important for multiview data analytics and current algorithms are either based on the collaborative learning of local partitions or directly derived global clustering from multikernel learning. In this paper, we innovate a clustering model that unifies the local partitions and global clustering in a collaborative learning framework. We first construct a common multikernel space from a set of basis kernels to better reflect clustering information of each individual view. Then, considering that joint local partitions would conform to the global clustering, we fuse the local partitions and global clustering guidance as a single objective function in accordance with fuzzy clustering form. The collaborative learning strategy enables the mutual and interactive clustering from local partitions and global clustering. The validation was performed over two synthetic and four public databases and the clustering accuracy was measured by normalized mutual information and rand index. The experimental results demonstrated that the proposed algorithm outperformed the related state-of-the-art algorithms in comparison, which included multitask, multikernel, and multiview clustering approaches.

BACKGROUND: Direct volume rendering is one of flexible and effective approaches to inspect large volumetric data such as medical and biological images. In conventional volume rendering, it is often time consuming to set up a meaningful illumination environment. Moreover, conventional illumination approaches usually assign same values of variables of an illumination model to different structures manually and thus neglect the important illumination variations due to structure differences. RESULTS: We introduce a novel illumination design paradigm for volume rendering on the basis of topology to automate illumination parameter definitions meaningfully. The topological features are extracted from the contour tree of an input volumetric data. The automation of illumination design is achieved based on four aspects of attenuation, distance, saliency, and contrast perception. To better distinguish structures and maximize illuminance perception differences of structures, a two-phase topology-aware illuminance perception contrast model is proposed based on the psychological concept of Just-Noticeable-Difference. CONCLUSIONS: The proposed approach allows meaningful and efficient automatic generations of illumination in volume rendering. Our results showed that our approach is more effective in depth and shape depiction, as well as providing higher perceptual differences between structures.