Meta-reinforcement learning requires utilizing prior task distribution information obtained during exploration to rapidly adapt to unknown tasks. The efficiency of an agent's exploration hinges on accurately identifying the current task. Recent Bayes-Adaptive Deep RL approaches often rely on reconstructing the environment's reward signal, which is challenging in sparse reward settings, leading to suboptimal exploitation. Inspired by bisimulation metrics, which robustly extracts behavioral similarity in continuous MDPs, we propose SimBelief—a novel meta-RL framework via measuring similarity of task belief in Bayes-Adaptive MDP (BAMDP). SimBelief effectively extracts common features of similar task distributions, enabling efficient task identification and exploration in sparse reward environments. We introduce latent task belief metric to learn the common structure of similar tasks and incorporate it into the real task belief. By learning the latent dynamics across task distributions, we connect shared latent task belief features with specific task features, facilitating rapid task identification and adaptation. Our method outperforms state-of-the-art baselines on sparse reward MuJoCo and panda-gym tasks.

Combination therapy has been widely applied in the treatment of complex diseases due to its ability to improve efficacy and reduce drug resistance. In recent years, deep learning techniques have significantly enhanced the accuracy of computational models for predicting drug interactions. However, these methods either focus solely on the molecular structure of drugs and the gene expression of cell lines or consider only the interaction networks related to drugs and cell lines. To effectively explore the combined impact of the structural information of drugs and cell lines and their interaction information in networks on drug synergy prediction, we propose a novel multimodal representation fusion model using hypergraph neural networks, called MoHyperSynergy. MoHyperSynergy constructs a hypergraph based on multimodal representations using the K-nearest neighbors approach. It interacts multimodal representations at the local feature level through an outer product embedding layer and fuses multimodal representations at the global semantic level using a hypergraph neural network, effectively utilizing information from different modalities of drugs and cell lines. MoHyperSynergy simultaneously extracts initial representations from drug molecular structures, cell line gene expressions, and the embeddings of drugs and cell lines in biomedical knowledge graphs as different modalities, and pretrains the model’s representation extractors in a self-supervised manner. We conducted experiments on real-world public datasets and evaluated our model under various scenarios that may arise in practical applications. The experimental results show that MoHyperSynergy achieves excellent performance in predicting drug synergy and significantly outperforms the current state-of-the-art models.

Herbs applicability in disease treatment has been verified through experiences over thousands of years. The understanding of herb-disease associations (HDAs) is yet far from complete due to the complicated mechanism inherent in multi-target and multi-component (MTMC) botanical therapeutics. Most of the existing prediction models fail to incorporate the MTMC mechanism. To overcome this problem, we propose a novel dual-channel hypergraph convolutional network, namely HGHDA, for HDA prediction. Technically, HGHDA first adopts an autoencoder to project components and target protein onto a low-dimensional latent space so as to obtain their embeddings by preserving similarity characteristics in their original feature spaces. To model the high-order relations between herbs and their components, we design a channel in HGHDA to encode a hypergraph that describes the high-order patterns of herb-component relations via hypergraph convolution. The other channel in HGHDA is also established in the same way to model the high-order relations between diseases and target proteins. The embeddings of drugs and diseases are then aggregated through our dual-channel network to obtain the prediction results with a scoring function. To evaluate the performance of HGHDA, a series of extensive experiments have been conducted on two benchmark datasets, and the results demonstrate the superiority of HGHDA over the state-of-the-art algorithms proposed for HDA prediction. Besides, our case study on Chuan Xiong and Astragalus membranaceus is a strong indicator to verify the effectiveness of HGHDA, as seven and eight out of the top 10 diseases predicted by HGHDA for Chuan-Xiong and Astragalus-membranaceus, respectively, have been reported in literature.

MOTIVATION: In the field of drug discovery, accurately and effectively predicting the binding affinity between proteins and ligands is crucial for drug screening and optimization. However, current research primarily utilizes representations based on sequence or structure to predict protein-ligand binding affinity, with relatively less study on protein surface information, which is crucial for protein-ligand interactions. Moreover, when dealing with multimodal information of proteins, traditional approaches typically concatenate features from different modalities in a straightforward manner without considering the heterogeneity among them, which results in an inability to effectively exploit the complementary between modalities. RESULTS: We introduce a novel multimodal feature extraction (MFE) framework that, for the first time, incorporates information from protein surfaces, 3D structures, and sequences, and uses cross-attention mechanism for feature alignment between different modalities. Experimental results show that our method achieves state-of-the-art performance in predicting protein-ligand binding affinity. Furthermore, we conduct ablation studies that demonstrate the effectiveness and necessity of protein surface information and multimodal feature alignment within the framework. AVAILABILITY AND IMPLEMENTATION: The source code and data are available at https://github.com/Sultans0fSwing/MFE.

BACKGROUND: Drug repositioning is a very important task that provides critical information for exploring the potential efficacy of drugs. Yet developing computational models that can effectively predict drug-disease associations (DDAs) is still a challenging task. Previous studies suggest that the accuracy of DDA prediction can be improved by integrating different types of biological features. But how to conduct an effective integration remains a challenging problem for accurately discovering new indications for approved drugs. METHODS: In this paper, we propose a novel meta-path based graph representation learning model, namely RLFDDA, to predict potential DDAs on heterogeneous biological networks. RLFDDA first calculates drug-drug similarities and disease-disease similarities as the intrinsic biological features of drugs and diseases. A heterogeneous network is then constructed by integrating DDAs, disease-protein associations and drug-protein associations. With such a network, RLFDDA adopts a meta-path random walk model to learn the latent representations of drugs and diseases, which are concatenated to construct joint representations of drug-disease associations. As the last step, we employ the random forest classifier to predict potential DDAs with their joint representations. RESULTS: To demonstrate the effectiveness of RLFDDA, we have conducted a series of experiments on two benchmark datasets by following a ten-fold cross-validation scheme. The results show that RLFDDA yields the best performance in terms of AUC and F1-score when compared with several state-of-the-art DDAs prediction models. We have also conducted a case study on two common diseases, i.e., paclitaxel and lung tumors, and found that 7 out of top-10 diseases and 8 out of top-10 drugs have already been validated for paclitaxel and lung tumors respectively with literature evidence. Hence, the promising performance of RLFDDA may provide a new perspective for novel DDAs discovery over heterogeneous networks.

The accuracy of state of charge estimation results will directly affect the performance of battery management system. Due to such, we focus in this article on the SOC estimation of Lithium-Ion batteries based on a fractional second-order RC model with free noninteger differentiation orders. For such an estimation, three Kalman filters are employed: the adaptive extended Kalman filter (AEKF), extended Kalman filter (EKF), and Unscented Kalman Filter (UKF). The Fractional-Order Model (FOM) parameters and differentiation orders are identified by the Particle Swarm Optimization (PSO) algorithm, and a pulsed-discharge test is implemented to verify the accuracy of parameter identification. The output voltage error of the FOM model is much less than that of the Integer-Order Model (IOM). The FOM model has lower root-mean square error (RMSE), the mean absolute error (MAE), and the maximum absolute error (MAXAE) of SOC estimation than the IOM model during the SOC estimation regardless of AEKF, EKF or UKF. Experimental results show that the FOM can simulate the polarisation on effect and charge–discharge characteristics of the battery more realistically, demonstrating that the SOC estimation based on FOM is more accurate and promising than the one based on the IOM when using the same Kalman filters.