Molecular representation learning is of great importance for drug molecular analysis. The development in molecular representation learning has demonstrated great promise through self-supervised pre-training strategy to overcome the scarcity of labeled molecular property data. Recent studies concentrate on pre-training molecular representation encoders by integrating both 2D topological and 3D geometric structures. However, existing methods rely on molecule-level or atom-level alignment for different views, while overlooking hierarchical self-supervised learning to capture both inter-molecule and intra-molecule correlation. Additionally, most methods employ 2D or 3D encoders to individually extract molecular characteristics locally or globally for molecular property prediction. The potential for effectively fusing these two molecular representations remains to be explored. In this work, we propose a Multi-View Molecular Representation Learning method (MVMRL) for molecular property prediction. First, hierarchical pre-training pretext tasks are designed, including fine-grained atom-level tasks for 2D molecular graphs as well as coarse-grained molecule-level tasks for 3D molecular graphs to provide complementary information to each other. Subsequently, a motif-level fusion pattern of multi-view molecular representations is presented during fine-tuning to enhance the performance of molecular property prediction. We evaluate the effectiveness of the proposed MVMRL by comparing with state-of-the-art baselines on molecular property prediction tasks, and the experimental results demonstrate the superiority of MVMRL.

Drug recommendation that aims to provide a prescription for a patient is an essential task in healthcare. Drug molecular graphs provide valuable support for drug recommendation. Existing methods tend to overlook drugs' molecular graphs or use the core substructures of molecular graphs with a rule-based segmentation strategy. However, such methods have several limitations: (1) The rule-based segmentation strategy is inflexible and sub-optimal for extremely complex scenarios. (2) The core substructures derived only consider the drug's chemical characteristics and ignore the patient's health condition. (3) The spurious correlation brought by trivial substructures is disregarded. To address these limitations, we design a novel drug recommendation method from a causal perspective, where a conditional causal representation learner for drug recommendation is proposed. Specifically, we first separate the drug molecular representation into causal and spurious parts depending on various patients' health conditions. Then, we eliminate the spurious correlation caused by the spurious part with causal intervention. Extensive experiments on the MIMIC-III and MIMIC-IV datasets demonstrate that our approach achieves new state-of-the-art performance (e.g., 6.68% Jaccard improvements on MIMIC-III with p-value 0.05).

With the increasing customization of spectrometers, spectral unmixing has become a widely used technique in fields such as remote sensing, textiles, and environmental protection. However, endmember variability is a common issue for unmixing, where changes in lighting, atmospheric, temporal conditions, or the intrinsic spectral characteristics of materials, can all result in variations in the measured spectrum. Recent studies have employed deep neural networks to tackle endmember variability. However, these approaches rely on generic networks to implicitly resolve the issue, which struggles with the ill-posed nature and lack of effective convergence constraints for endmember variability. This paper proposes a streamlined multi-task learning model to rectify this problem, incorporating abundance regression and multi-label classification with Unmixing as a Bayesian Inverse Problem, denoted as BIPU. To address the issue of the ill-posed nature, the uncertainty of unmixing is quantified and minimized through the Laplace approximation in a Bayesian inverse solver. In addition, to improve convergence under the influence of endmember variability, the paper introduces two types of constraints. The first separates background factors of variants from the initial factors for each endmember, while the second identifies and eliminates the influence of non-existent endmembers via multi-label classification during convergence. The effectiveness of this model is demonstrated not only on a self-collected near-infrared spectral textile dataset (FENIR), but also on three commonly used remote sensing hyperspectral image datasets, where it achieves state-of-the-art unmixing performance and exhibits strong generalization capabilities.

Molecular representation learning has remarkably accelerated the development of drug analysis and discovery. It implements machine learning methods to encode molecule embeddings for diverse downstream drug-related tasks. Due to the scarcity of labeled molecular data, self-supervised molecular pre-training is promising as it can handle large-scale unlabeled molecular data to prompt representation learning. Although many universal graph pre-training methods have been successfully introduced into molecular learning, there still exist some limitations. Many graph augmentation methods, such as atom deletion and bond perturbation, tend to destroy the intrinsic properties and connections of molecules. In addition, identifying subgraphs that are important to specific chemical properties is also challenging for molecular learning. To address these limitations, we propose the self-supervised Molecular Graph Information Bottleneck (MGIB) model for molecular pre-training. MGIB observes molecular graphs from the atom view and the motif view, deploys a learnable graph compression process to extract the core subgraphs, and extends the graph information bottleneck into the self-supervised molecular pre-training framework. Model analysis validates the contribution of the self-supervised graph information bottleneck and illustrates the interpretability of MGIB through the extracted subgraphs. Extensive experiments involving molecular property prediction, including 7 binary classification tasks and 6 regression tasks demonstrate the effectiveness and superiority of our proposed MGIB.

Combinatorial drug recommendation involves recommending appropriate drug combinations for patients based on their complex health conditions, which is an essential task for AI in healthcare. However, existing approaches have several limitations. Firstly, they fail to fully utilize important information such as the hierarchical structure of drug molecules, patient visit history, and prior medical knowledge. Secondly, they ignore the inherent associations between these pieces of information and only encode one or two of them in isolation, leading to sub-optimal results. To address these issues, we propose KE-HMFNet, which leverages patient visit history, hierarchical molecular representation of drugs, and prior medical knowledge, and explicitly models their inherent association to make medication recommendations that are both effective and safe. Specifically, we develop a patient-guided fusion mechanism to make the hierarchical molecular representation disease-relevant and substructure-aware. Additionally, we design a knowledge-enhanced medication relation representation module to capture the inherent relation between drugs based on the patient’s condition. Extensive experiments on the MIMIC-III dataset demonstrate that our approach achieves new state-of-the-art performance1.

Bipartite graphs have been widely used in many applications such as recommender systems, search engines and so on. Recent works consider bipartite graphs as homogeneous graphs and apply graph convolution networks for link prediction or node classification. However, in bipartite graphs, there are two types of nodes which are from different domains such as users and items in recommender systems, and cannot be in the same embedding space. In this paper, we proposed a novel graph convolution operation to propagate in bipartite graph with less spatial and temporal complexities, and two mapping functions with adversarial constraints to transfer features between two domains. Experimental results show that the proposed model achieves the improved performance on the tasks of link prediction and recommendation in real-world scenarios.

Existing methods for Multimodal Sentiment Analysis (MSA) mainly focus on integrating multimodal data effectively on limited multimodal data. Learning more informative multimodal representation often relies on large-scale labeled datasets, which are difficult and unrealistic to obtain. To learn informative multimodal representation on limited labeled datasets as more as possible, we proposed TMMDA for MSA, a new Token Mixup Multimodal Data Augmentation, which first generates new virtual modalities from the mixed token-level representation of raw modalities, and then enhances the representation of raw modalities by utilizing the representation of the generated virtual modalities. To preserve semantics during virtual modality generation, we propose a novel cross-modal token mixup strategy based on the generative adversarial network. Extensive experiments on two benchmark datasets, i.e., CMU-MOSI and CMU-MOSEI, verify the superiority of our model compared with several state-of-the-art baselines. The code is available at https://github.com/xiaobaicaihhh/TMMDA.

It is believed that nonverbal visual information including facial expressions, facial micro-actions and head movements plays a significant role in fundamental social communication. Unfortunately it is regretful that the blind can not achieve such necessary information. Therefore, we propose a social assistant system, Lookine, to help them to go beyond this limitation. For Lookine, we apply the novel techniques including facial expression recognition, facial action recognition and head pose estimation, and obey barrier-free principles in our design. In experiments, the algorithm evaluation and user study prove that our system has promising accuracy, good real-time performance, and great user experience.