Abstract Research in scientific domains now generates more than a million articles annually, overwhelming researchers and hindering discovery. This surge has sparked interest in biomedical hypothesis generation (HG), which aims to uncover implicit patterns among biomedical concepts. Most existing methods focus on pairwise link prediction, overlooking the complex, multi-concept relationships underlying many breakthroughs. We introduce HyHG , a temporal Hy pergraph contrastive learning framework for biomedical H ypothesis G eneration, which redefines hypotheses as hyperedges—sets of co-mentioned concepts in an article. By representing articles as hyperedges and organizing them into a temporal hypergraph, HyHG captures the evolution of scientific ideas over time. A transformer-based architecture learns from historical hyperedge sequences to predict future hyperedges—sets of concepts likely to co-occur in the future literature. To distinguish genuine hypotheses from misleading ones, HyHG employs a time-anchored contrastive loss and hard negative sampling based on minimal edits to real hyperedges. We demonstrate that HyHG achieves state-of-the-art performance on three biomedical datasets. Our code and data are available at: https://github.com/amir-hassan25/Temporal-Hypergraph-Contrastive-Learning.

Human intelligence can continuously and adaptively build multimodal cognition from a series of diverse modalities in the external world. Modality-varying Continual Learning (MVCL) aims to imitate such human intelligence, which trains models on a stream of non-stationary and modality-fluctuating data distributions while sequentially transferring and protecting past knowledge. When an MVCL learner cannot anticipate the complexity of future new modalities and inter-modal interactions, the challenge of dealing with knowledge saturation (KS) with satisfactory parameter efficiency (PE) increases. Existing works focused mainly on overcoming the forgetting of past knowledge but overlooked the critical tradeoff between KS and PEC To address this gap, we propose a novel continual learning frame-work, namely LifelongSkill, that explicitly optimizes this tradeoff. Our key idea is to capture the interpretable inter-task diversity underlying the task stream, and then use this information to guide the parameter-efficient knowledge transfer and necessary network expansion. Specifically, we learn a Latent Knowledge Hypergraph (LKGraph), comprising a variety of semantically-distinct functional capabilities (namely skills) learned from tasks, to represent task diversity through skill co-occurrences. Then, we propose a Skill-wise Node Decoder (SND) to facilitate parameter-efficient network expansion and knowledge transfer guided by LKGraph. Experiment results demonstrate the proposed approach achieves the best tradeoffs between performance and parameter efficiency compared with baselines.

Accurate prediction of patient outcomes from electronic health records (EHRs) is a fundamental task in data mining with practical benefits to clinical decision support and healthcare resource allocation. Over the past few years, with the advent of large language models (LLMs), there has been increasing interest in training LLMs on EHR clinical notes to improve outcome predictions. Despite significant advances, existing approaches have a certain limitation. Specifically, the existing approaches largely model clinical notes as flat token sequences and overlook their intrinsic semi-structured organization into sections (e.g., History of Present Illness and Physical Exam). Moreover, most of the existing approaches ignore the issue of missing data prevalent in real-world EHR clinical notes. To address these challenges, we propose a novel approach that leverages the inherent structure of clinical notes to impute missing sections and learns robust feature representations needed for outcome prediction. In particular, we propose a context-aware section imputation strategy that utilizes multi-head attention to infer missing section representations based on inter-section dependencies within the clinical note. Moreover, to learn disentangled feature representations, we propose orthogonality constraints across the section embeddings. Extensive experiments on multiple benchmark datasets for clinical outcome prediction show that the proposed approach achieves consistent improvements over strong baseline algorithms. The code has been released on github at https://github.com/LeiGong0125Carrot/Strucure-Awared-Clinical-Note-Processing/tree/ICDM-2025

Zero-shot biomedical text classification requires accurate assignment of biomedical text (e.g., scientific abstract) to previously unseen labels (or concepts). Existing methods often struggle to generalize to novel concepts such as new diseases, drugs, and genes. To address these unique challenges, we propose a framework that combines feature disentanglement with contrastive learning to address this limitation. It separates each abstract into a content representation relevant for classification and a variance representation. This disentanglement ensures that the content features are invariant to the writing style, improving generalization to unseen labels. A contrastive learning strategy further structures the latent space by encouraging semantic clustering and separation of categories. Moreover, we model intra-class variance as a shared distribution across labels to enable variational data augmentation, enhancing robustness. The framework uses a domain-specific biomedical language model for feature extraction and fixed label anchors for semantic alignment. Extensive experiments conducted on the largest available biomedical corpus achieve superior performance on zero-shot multi-label classification tasks by learning discriminative and style-invariant representations.

Biomedical research now generates more than a million articles annually, overwhelming researchers and hindering discovery. This surge has sparked interest in biomedical hypothesis generation (HG), which aims to uncover implicit patterns among biomedical concepts. Most existing methods focus on pairwise link prediction, overlooking the complex, multi-concept relationships underlying many breakthroughs. We introduce HyHG, a temporal Hypergraph contrastive learning framework for biomedical Hypothesis Generation, which redefines hypotheses as hyperedges–sets of co-mentioned concepts in an article. By representing articles as hyperedges and organizing them into a temporal hypergraph, HyHG captures the evolution of scientific ideas over time. A transformer-based architecture learns from historical hyperedge sequences to predict future hyperedges–sets of concepts likely to co-occur in future literature. To distinguish genuine hypotheses from misleading ones, HyHG employs a timeanchored contrastive loss and hard negative sampling based on minimal edits to real hyperedges. We demonstrate state-of-the-art performance on three biomedical datasets. Our code and data are available at: https://github.com/amirhassan25/Temporal-Hypergraph-Contrastive-Learning.

Accurate health risk prediction is crucial for making informed clinical decisions and assessing the appropriate allocation of medical resources. While recent deep learning based approaches have shown great promise in risk prediction, they primarily focus on modeling the sequential information in Electronic Health Records (EHRs) and fail to leverage the rich mobility interactions among health entities. As a result, the existing approaches yield unsatisfactory performance in downstream risk prediction tasks, especially tasks such as Clostridioides difficile Infection (CDI) incidence prediction that are primarily spread through mobility interactions. To address this issue, we propose a new approach that leverages Hypergraphs to explicitly model mobility interactions to improve predictive performance in health risk prediction tasks. Unlike regular graphs that are limited to modeling pairwise relationships, hypergraphs can effectively characterize the complex high-order semantic relationships between patients. Moreover, we introduce a new contrastive learning strategy that exploits the domain knowledge to generate semantically meaningful positive (homologous) and negative (heterologous) pairs needed for contrastive learning. This unique contrastive pair augmentation strategy boosts the power of contrastive learning by generating feature representations that are both robust and well-aligned with the domain knowledge. Experiments on two real-world datasets demonstrate the advantage of our approach in both short-term and long-term risk prediction tasks, such as CDI incidence prediction and MICU transfer prediction. Our framework obtains gains in performance up to 29.49 % for PHOP, 30.64 % for MIMIC-IV for MICU transfer prediction, 13.17 % for PHOP, and 4.45 % for MIMIC-IV for CDI Incidence Prediction.

Social determinants of health (SDoH) factors such as poverty, social environment, and unemployment are known to profoundly impact health outcomes. However, extracting SDoH from the electronic health records (EHR) is a challenge due to the unstructured nature of clinical narratives that encode them. To address this, several approaches ranging from rule-based natural language processing to large language models have been proposed in the literature. Despite significant advances, the existing SDoH extraction approaches are not robust to the noise present in clinical notes or discharge summaries and thus yield unsatisfactory performance. In other words, the noisy information in clinical notes leads to the generation of low-quality feature representations of medical concepts that severely impacts the performance of SDoH extraction.In this paper, we propose a novel approach that augments EHR discharge summaries with context-specific semantic knowledge from biomedical literature to generate robust feature representations needed for accurate SDoH extraction. Specifically, our approach identifies key contextual information (e.g., symptoms, diseases, and medications) from EHR discharge summaries and retrieves relevant scientific articles to generate additional semantic context for SDoH classifier. Moreover, to effectively fuse complementary information from both EHR discharge summaries and biomedical literature, we propose a new feature infusion strategy that adaptively fuses feature representations based on their contextual relevance. Experimental results on the benchmark MIMIC-SDoH dataset demonstrate that the proposed approach significantly outperforms baseline algorithms and highlight the role of context-specific feature augmentation in enhancing the accuracy of SDoH extraction.

Continual learning has emerged as a powerful approach to address the challenges of non-stationary environments, allowing machine learning models to adapt to new data while retaining the previously acquired knowledge. In time-sensitive healthcare applications, where entities such as physicians, hospital rooms, and medications exhibit continuous changes over time, continual learning holds great promise, yet its application remains relatively unexplored. This paper aims to bridge this gap by proposing a novel framework, i.e., Continually-Adaptive Representation Learning, designed to adapt representations in response to changing data distributions in evolving healthcare applications. Specifically, the proposed approach develops a continual learning strategy wherein the context information (e.g., interactions) of healthcare entities is exploited to continually identify and retrain the representations of those entities whose context evolved over time. Moreover, different from existing approaches, the proposed approach leverages the valuable patient information present in clinical notes to generate accurate and robust healthcare embeddings. Notably, the proposed continually-adaptive representations are have practical benefits in low-resource clinical settings where it is difficult to training machine learning models from scratch to accommodate the newly available data streams. Experimental evaluations on real-world healthcare datasets demonstrate the effectiveness of our approach in time-sensitive healthcare applications such as Clostridioides difficile (C.diff) Infection (CDI) incidence prediction task and medical intensive care unit transfer prediction task.

This paper studies the convergence and generalization of a large class of Stochastic Gradient Descent (SGD) momentum schemes, in both learning from scratch and transferring representations with fine-tuning. Momentum-based acceleration of SGD is the default optimizer for many deep learning models. However, there is a lack of general convergence guarantees for many existing momentum variants in conjunction withstochastic gradient. It is also unclear how the momentum methods may affect thegeneralization error. In this paper, we give a unified analysis of several popular optimizers, e.g., Polyak's heavy ball momentum and Nesterov's accelerated gradient. Our contribution is threefold. First, we give a unified convergence guarantee for a large class of momentum variants in thestochastic setting. Notably, our results cover both convex and nonconvex objectives. Second, we prove a generalization bound for neural networks trained by momentum variants. We analyze how hyperparameters affect the generalization bound and consequently propose guidelines on how to tune these hyperparameters in various momentum schemes to generalize well. We provide extensive empirical evidence to our proposed guidelines. Third, this study fills the vacancy of a formal analysis of fine-tuning in literature. To our best knowledge, our work is the first systematic generalizability analysis on momentum methods that cover both learning from scratch and fine-tuning. Our codes are available https://github.com/jsycsjh/Demystify-Hyperparameters-for-Stochastic-Optimization-with-Transferable-Representations .

Pre-trained concept representations are essential to many biomedical text mining and natural language processing tasks. As such, various representation learning approaches have been proposed in the literature. More recently, contextualized embedding approaches (i.e., BERT based models) that capture the implicit semantics of concepts at a granular level have significantly outperformed the conventional word embedding approaches (i.e., Word2Vec/GLoVE based models). Despite significant accuracy gains achieved, these approaches are often computationally expensive and memory inefficient. To address this issue, we propose a new representation learning approach that efficiently adapts the concept representations to the newly available data. Specifically, the proposed approach develops a knowledge-guided continual learning strategy wherein the accurate/stable context-information present in human-curated knowledge-bases is exploited to continually identify and retrain the representations of those concepts whose corpus-based context evolved coherently over time. Different from previous studies that mainly leverage the curated knowledge to improve the accuracy of embedding models, the proposed research explores the usefulness of semantic knowledge from the perspective of accelerating the training efficiency of embedding models. Comprehensive experiments under various efficiency constraints demonstrate that the proposed approach significantly improves the computational performance of biomedical word embedding models.

Fake news travels at unprecedented speeds, reaches global audiences and puts users and communities at great risk via social media platforms. Deep learning based models show good performance when trained on large amounts of labeled data on events of interest, whereas the performance of models tends to degrade on other events due to domain shift. Therefore, significant challenges are posed for existing detection approaches to detect fake news on emergent events, where large-scale labeled datasets are difficult to obtain. Moreover, adding the knowledge from newly emergent events requires to build a new model from scratch or continue to fine-tune the model, which can be challenging, expensive, and unrealistic for real-world settings. In order to address those challenges, we propose an end-to-end fake news detection framework named MetaFEND, which is able to learn quickly to detect fake news on emergent events with a few verified posts. Specifically, the proposed model integrates meta-learning and neural process methods together to enjoy the benefits of these approaches. In particular, a label embedding module and a hard attention mechanism are proposed to enhance the effectiveness by handling categorical information and trimming irrelevant posts. Extensive experiments are conducted on multimedia datasets collected from Twitter and Weibo. The experimental results show our proposed MetaFEND model can detect fake news on never-seen events effectively and outperform the state-of-the-art methods.

Indexing biomedical research articles with Medical Subject Headings (MeSH) can greatly facilitate biomedical research and information retrieval. Currently MeSH indexing is performed by human experts. To alleviate the time consumption and monetary cost caused by manual indexing, many automatic MeSH indexing models have been developed, such as MeSHProbeNet, DeepMeSH, and NLM’s official model Medical Text Indexer. In this article, we propose an end-to-end framework, MeSHProbeNet-P, which extends MeSHProbeNet with personalizable MeSH probes. In MeSHProbeNet-P, each MeSH probe carries certain aspects of biomedical knowledge and extracts related information from input articles. MeSHProbeNet-P is able to automatically personalize its MeSH probes for different input articles to ensure that the current MeSH probes best fit the current input article and the most informative features can be extracted from the article. We demonstrate the effectiveness of MeSHProbeNet-P in a real-world large-scale MeSH indexing challenge. MeSHProbeNet-P won the first place in the first batch of Task A in the 2019 BioASQ challenge. The result on the first test set of the challenge is reported in this article. We also provide ablation studies to show the advantages of personalizable MeSH probes.

This paper develops the Correlation Networks (CorNet) architecture for the extreme multi-label text classification (XMTC) task, where the objective is to tag an input text sequence with the most relevant subset of labels from an extremely large label set. XMTC can be found in many real-world applications, such as document tagging and product annotation. Recently, deep learning models have achieved outstanding performances in XMTC tasks. However, these deep XMTC models ignore the useful correlation information among different labels. CorNet addresses this limitation by adding an extra CorNet module at the prediction layer of a deep model, which is able to learn label correlations, enhance raw label predictions with correlation knowledge and output augmented label predictions. We show that CorNet can be easily integrated with deep XMTC models and generalize effectively across different datasets. We further demonstrate that CorNet can bring significant improvements over the existing deep XMTC models in terms of both performance and convergence rate. The models and datasets are available at: https://github.com/XunGuangxu/CorNet.

Hypothesis generation (HG) refers to the task of mining meaningful implicit association between unlinked biomedical concepts. The majority of prior studies have focused on uncovering these implicit linkages from static snapshots of the corpus, thereby largely ignoring the temporal dynamics of medical concepts. More recently, a few initial studies attempted to overcome this issue by modelling the temporal change of concepts from natural language text. However, they still fail to leverage the evolutionary features of concepts from contemporary knowledge-bases (KB's) such as semantic lexicons and ontologies. In practice such KB's contain up-to-date information that is important to incorporate, especially, in highly evolving domains such as biomedicine. Furthermore, considering the complementary strength of these sources of information - corpus and ontology - a few natural questions arise: Can joint modelling of (co)-evolutionary dynamics from these resources aid in encoding the temporal features at a granular level? Can the mutual evolution between these intertwined resources lead to better predictive effects? To answer these questions, in this study, we present a novel HG framework that unearths the latent associations between concepts by modeling their co-evolution across complementary sources of information. More specifically, the proposed approach adopts a shared temporal matrix factorization framework that models the co-evolution of concepts across both corpus and KB. Extensive experiments on the largest available biomedical corpus validates the effectiveness of the proposed approach.

Recent advances in unsupervised language processing methods have created an opportunity to exploit massive text corpora for developing high-quality vector space representation (also known as word embeddings) of words. Towards this direction, practitioners have developed and applied several data driven embedding models with quite good rate of success. However, a drawback of these models lies in their premise of static context; wherein, the meaning of a word is assumed to remain the same over the period of time. This is limiting because it is known that the semantic meaning of a concept evolves over time. While such semantic drifts are routinely observed in almost all the domains; their effect is acute in domain such as biomedicine, where the semantic meaning of a concept changes relatively fast. To address this, in this study, we aim to learn temporally aware vector representation of medical concepts from the timestamped text data, and in doing so provide a systematic approach to formalize the problem. More specifically, a dynamic word embedding based model that jointly learns the temporal characteristics of medical concepts and performs across time-alignment is proposed. Apart from capturing the evolutionary characteristics in an optimal manner, the model also factors in the implicit medical properties useful for a variety of bio-medical applications. Empirical studies conducted on two important bio-medical use cases validates the effectiveness of the proposed approach and suggests that the model not only learns quality embeddings but also facilitates intuitive trajectory visualizations.

Word embeddings are finding their increasing application in a variety of biomedical Natural Language Processing (bioNLP) tasks, ranging from drug discovery to automated disease diagnosis. While these word embeddings in their entirety have shown meaningful syntactic and semantic regularities, however, the meaning of individual dimensions remains elusive. This becomes problematic both in general and particularly in sensitive domains such as bio-medicine, wherein, the interpretability of results is crucial to its widespread adoption. To address this issue, in this study, we aim to improve the interpretability of pre-trained word embeddings generated from a text corpora, and in doing so provide a systematic approach to formalize the problem. More specifically, we exploit the rich categorical knowledge present in the biomedical domain, and propose to learn a transformation matrix that transforms the input embeddings to a new space where they are both interpretable and retain their original expressive features. Experiments conducted on the largest available biomedical corpus suggests that the model is capable of performing interpretability that resembles closely to the human-level intuition.

Given two topics of interest (A, C) that are otherwise disconnected - for instance two concepts: a disease ("Migraine") and a therapeutic substance ("Magnesium") - this paper attempts to find the conceptual bridges (e.g., serotonin (B)) that connects them in a novel way. This problem of mining implicit linkage is known as hypotheses generation and its potential to accelerate scientific progress is widely recognized. Almost all of the prior studies to tackle this problem ignore the temporal dynamics of concepts. This is limiting because it is known that the semantic meaning of a concept evolves over time. To overcome this issue, in this study, we define this problem as mining time-aware Top-k conceptual bridges, and in doing so provide a systematic approach to formalize the problem. Specifically, the proposed model first extracts relevant entities from the corpus, represents them in time-specific latent spaces, and then further reasons upon it to generate novel and experimentally testable hypotheses. The key challenge in this approach is to learn a mapping function that encodes the temporal characteristics of concepts and aligns the across-time latent spaces. To solve this, we propose an effective algorithm that learns precise mapping sensitive to both global and local semantics of the input query. Both qualitative and quantitative evaluations performed on the largest available biomedical corpus substantiate the importance of leveraging the evolutionary semantics of medical concepts and suggests that the generated hypotheses are novel and worthy of clinical trials.

As news reading on social media becomes more and more popular, fake news becomes a major issue concerning the public and government. The fake news can take advantage of multimedia content to mislead readers and get dissemination, which can cause negative effects or even manipulate the public events. One of the unique challenges for fake news detection on social media is how to identify fake news on newly emerged events. Unfortunately, most of the existing approaches can hardly handle this challenge, since they tend to learn event-specific features that can not be transferred to unseen events. In order to address this issue, we propose an end-to-end framework named Event Adversarial Neural Network (EANN), which can derive event-invariant features and thus benefit the detection of fake news on newly arrived events. It consists of three main components: the multi-modal feature extractor, the fake news detector, and the event discriminator. The multi-modal feature extractor is responsible for extracting the textual and visual features from posts. It cooperates with the fake news detector to learn the discriminable representation for the detection of fake news. The role of event discriminator is to remove the event-specific features and keep shared features among events. Extensive experiments are conducted on multimedia datasets collected from Weibo and Twitter. The experimental results show our proposed EANN model can outperform the state-of-the-art methods, and learn transferable feature representations.

The technological advancements in biomedical domain has led to a tremendous growth of unstructured data; primarily a result of increased publication of findings. At the same time, a corresponding interest in the Natural Language Processing (NLP) community to develop scalable methodologies to exploit such massive unlabeled corpora for unsupervised language processing has resulted in new opportunities towards developing semantic sensitive models. Amongst them, the field of word embeddings has garnered significant attention due to its capability to understand implicit semantics. However such data driven models are largely agnostic of the rich explicit semantic knowledge available in the biomedical domain in the form of vocabularies and ontologies. This is problematic because it leads to a poor representation of words with little local context and its effect is acute in biomedical domain. In this paper, we propose a novel model (MeSH2Vec) that jointly exploits both contextual information and available explicit semantic knowledge to learn externally augmented word embeddings. Unlike existing approaches, the proposed methodology is more dexterous in its ability to handle relationships between indirectly related concepts. The 13% improvement in the correlation to experts, shown on experiments involving biomedical concept similarity and relatedness task validates the effectiveness of the proposed approach and demonstrates the importance of incorporating human curated knowledge in the process of generating word embeddings.

Literature based discovery (LBD) is a task that aims to uncover hidden associations between non-interacting scientific concepts by rationally connecting independent nuggets of information. Broadly, prior approaches to LBD include use of: a) distributional statistics and explicit representation, b) graph-theoretic measures, and c) supervised machine learning methods to find associations. However, purely distributional approaches may not necessarily entail semantically meaningful association and graph-theoretic approaches suffer from scalability issues. While supervised machine learning based approaches have the potential to elucidate associations, the training data required is too expensive to generate. In this paper we propose a novel dynamic Medical Subject Heading (MeSH) embedding model which is able to model the evolutionary behavior of medical concepts to uncover latent associations between them. The proposed model allows us to learn the evolutionary trajectories of MeSH embeddings and detect informative terms. Hence, based on the dynamic MeSH embeddings, meaningful medical hypotheses can be efficiently generated. To evaluate the efficacy of the proposed model, we perform both qualitative and quantitative evaluation. The results demonstrate that leveraging the evolutionary features of MeSH concepts is an effective way for predicting novel associations.