Single-cell annotation is a fundamental task in the analysis of single-cell data, and one promising research direction relies on the marker gene information accumulated in biology. Recently, self-training strategies have been introduced into the field, which significantly improve the annotation accuracy by iteratively optimizing the model. However, existing methods have not yet systematically explored how to construct self-training frameworks that are more applicable to single-cell data. To this end, we propose a context-aware self-training model named CSSTA. Firstly, the contextual information of marker genes is introduced to enhance the compatibility of marker genes with different single-cell datasets to generate high-quality pseudo-labels. Secondly, high- and low-confidence pseudo-labels recognition and supervision strategies more applicable to single-cell data are designed that can better guide the optimization of the model. Finally, the insight of the single-cell foundation model on cell-cell association information is introduced. Experiments demonstrate that the introduction of marker gene contextual information significantly improves the ability to recognize cell-cell type associations with heuristic-based strategies. Benchmark experiments show that CSSTA significantly outperforms state-of-the-art baselines. Notably, we demonstrate the potential of CSSTA for hierarchical cellular annotation by extending it to hierarchies.

Abstract Neural topic modeling is a scalable automated technique for text data mining. In various downstream tasks of topic modeling, it is preferred that the discovered topics well align with labels. However, due to the lack of guidance from labels, unsupervised neural topic models are less powerful in this situation. Existing supervised neural topic models often adopt a label-free prior to generate the latent document-topic distributions and use them to predict the labels and thus achieve label-topic alignment indirectly. Such a mechanism faces the following issues: 1) The label-free prior leads to topics blending the latent patterns of multiple labels; and 2) One is unable to intuitively identify the explicit relationships between labels and the discovered topics. To tackle these problems, we develop a novel supervised neural topic model which utilizes a chain-structured graphical model with a label-conditioned prior. Soft indicators are introduced to explicitly construct the label-topic relationships. To obtain well-organized label-topic relationships, we formalize an entropy-regularized optimal transport problem on the embedding space and model them as the transport plan. Moreover, our proposed method can be flexibly integrated with most existing unsupervised neural topic models. Experimental results on multiple datasets demonstrate that our model can greatly enhance the alignment between labels and topics while maintaining good topic quality.

Complex Query Answering (CQA) on knowledge graphs is a fundamental yet challenging task, which can be formalized as answering a subset of first-order logic queries containing logical conjunction, disjunction, negation, and existential quantifiers. Recent research reveals that Link Predictors (LPs) trained on 1-hop queries can generalize to various types of complex queries. However, existing methods neglect crucial characteristics of LPs' outputs, including the effects of highly relevant entities and uncertainty. What's worse, as they model logical operations by fuzzy set operations, these methods suffer from problems like inflexibility, sensitivity to noise, and inconsistency with priority in human cognition, which limits their performance, especially on queries with negation. To address these challenges, we propose an efficient fuzzy system for CQA that requires no extra training overheads and is plug-and-play with existing LP-based methods. Firstly, we expand the output of LPs by two complementary membership functions of weak and strong relevance, which help to distinguish the target entities from highly relevant and irrelevant entities. Subsequently, we model logical operations through fuzzy rule bases and infer the final predictions via defuzzification, providing a flexible and tractable scheme for modeling logical operations. Finally, the effectiveness of the proposed fuzzy system is validated by its outstanding performance on benchmark datasets when compared to state-of-theart methods. The source code of our proposed method is available at https://anonymous.4open.science/r/FuzzSys-CQA-E285.

Hierarchical topic modeling, which can mine implicit semantics in the corpus and automatically construct topic hierarchical relationships, has received considerable attention recently. However, the current hierarchical topic models are mainly based on Euclidean space, which cannot well retain the implicit hierarchical semantic information in the corpus, leading to irrational structure of the generated topics. On the other hand, the existing Generative Adversarial Network (GAN) based neural topic models perform satisfactorily, but they remain constrained by pattern collapse due to the discontinuity of latent space. To solve the above problems, with the hypothesis of hyperbolic space, we propose a novel GAN-based hierarchical topic model to mine high-quality topics by introducing contrastive learning to capture information from documents. Furthermore, the distinct tree-like property of hyperbolic space preserves the implicit hierarchical semantics of documents in topic embeddings, which are projected into the hyperbolic space. Finally, we use a multi-head self-attention mechanism to learn implicit hierarchical semantics of topics and mine topic structure information. Experiments on real-world corpora demonstrate the remarkable performance of our model on topic coherence and topic diversity, as well as the rationality of the topic hierarchy.

Recent advances in microfluidics and sequencing technologies allow researchers to explore cellular heterogeneity at single-cell resolution. In recent years, deep learning frameworks, such as generative models, have brought great changes to the analysis of transcriptomic data. Nevertheless, relying on the potential space of these generative models alone is insufficient to generate biological explanations. In addition, most of the previous work based on generative models is limited to shallow neural networks with one to three layers of latent variables, which may limit the capabilities of the models. Here, we propose a deep interpretable generative model called d-scIGM for single-cell data analysis. d-scIGM combines sawtooth connectivity techniques and residual networks, thereby constructing a deep generative framework. In addition, d-scIGM incorporates hierarchical prior knowledge of biological domains to enhance the interpretability of the model. We show that d-scIGM achieves excellent performance in a variety of fundamental tasks, including clustering, visualization, and pseudo-temporal inference. Through topic pathway studies, we found that d-scIGM-learned topics are better enriched for biologically meaningful pathways compared to the baseline models. Furthermore, the analysis of drug response data shows that d-scIGM can capture drug response patterns in large-scale experiments, which provides a promising way to elucidate the underlying biological mechanisms. Lastly, in the melanoma dataset, d-scIGM accurately identified different cell types and revealed multiple melanin-related driver genes and key pathways, which are critical for understanding disease mechanisms and drug development.

MOTIVATION: Recent advances in sequencing technology provide opportunities to study biological processes at a higher resolution. Cell type annotation is an important step in scRNA-seq analysis, which often relies on established marker genes. However, most of the previous methods divide the identification of cell types into two stages, clustering and assignment, whose performances are susceptible to the clustering algorithm, and the marker information cannot effectively guide the clustering process. Furthermore, their linear heuristic-based cell assignment process is often insufficient to capture potential dependencies between cells and types. RESULTS: Here, we present Interpretable Cell Type Annotation based on self-training (sICTA), a marker-based cell type annotation method that combines the self-training strategy with pseudo-labeling and the nonlinear association capturing capability of Transformer. In addition, we incorporate biological priori knowledge of genes and pathways into the classifier through an attention mechanism to enhance the transparency of the model. A benchmark analysis on 11 publicly available single-cell datasets demonstrates the superiority of sICTA compared to state-of-the-art methods. The robustness of our method is further validated by evaluating the prediction accuracy of the model on different cell types for each single-cell data. Moreover, ablation studies show that self-training and the ability to capture potential dependencies between cells and cell types, both of which are mutually reinforcing, work together to improve model performance. Finally, we apply sICTA to the pancreatic dataset, exemplifying the interpretable attention matrix captured by sICTA. AVAILABILITY AND IMPLEMENTATION: The source code of sICTA is available in public at https://github.com/nbnbhwyy/sICTA. The processed datasets can be found at https://drive.google.com/drive/folders/1jbqSxacL_IDIZ4uPjq220C9Kv024m9eL. The final version of the model will be permanently available at https://doi.org/10.5281/zenodo.13474010.

Abstract Topic taxonomy discovery aims at uncovering topics of different abstraction levels and constructing hierarchical relations between them. Unfortunately, most prior work can hardly model semantic scopes of words and topics by holding the Euclidean embedding space assumption. What’s worse, they infer asymmetric hierarchical relations by symmetric distances between topic embeddings. As a result, existing methods suffer from problems of low-quality topics at high abstraction levels and inaccurate hierarchical relations. To alleviate these problems, this paper develops a Box embedding-based Topic Model (BoxTM) that maps words and topics into the box embedding space, where the asymmetric metric is defined to properly infer hierarchical relations among topics. Additionally, our BoxTM explicitly infers upper-level topics based on correlation between specific topics through recursive clustering on topic boxes. Finally, extensive experiments validate high-quality of the topic taxonomy learned by BoxTM.

Hierarchical topic models, which can extract semantically meaningful topics from a text corpus in an unsupervised manner and automatically organise them into a topic hierarchy, have been widely used to discover the underlying semantic structure of documents.However, the existing models often assume in the prior that the topic hierarchy is a tree structure, ignoring symmetrical dependencies between topics at the same level.Moreover, the sparsity of text data often complicate the analysis.To address these issues, we propose NSEM-GMHTM as a deep topic model, with a Gaussian mixture prior distribution to improve the model's ability to adapt to sparse data, which explicitly models hierarchical and symmetric relations between topics through the dependency matrices and nonlinear structural equations.Experiments on widely used datasets show that our NSEM-GMHTM generates more coherent topics and a more rational topic structure when compared to state-of-theart baselines.Our code is available at https: //github.com/nbnbhwyy/NSEM-GMHTM. W3 W2 W1

The out-of-vocabulary (OOV) words are difficult to represent while critical to the performance of embedding-based downstream models.Prior OOV word embedding learning methods failed to model complex word formation well.In this paper, we propose a novel graph-based relation mining method, namely GRM, for OOV word embedding learning.We first build a Word Relationship Graph (WRG) based on word formation and associate OOV words with their semantically relevant words, which can mine the relational information inside word structures.Subsequently, our GRM can infer high-quality embeddings for OOV words through passing and aggregating semantic attributes and relational information in the WRG, regardless of contextual richness.Extensive experiments demonstrate that our model significantly outperforms state-of-theart baselines on both intrinsic and downstream tasks when faced with OOV words.

Combination therapy, which can improve therapeutic efficacy and reduce side effects, plays an important role in the treatment of complex diseases. Yet, a large number of possible combinations among candidate compounds limits our ability to identify effective combinations. Though many studies have focused on predicting potential drug combinations, the existing methods are not entirely satisfactory in terms of performance and scalability. In this study, we propose a new computational pipeline, called DCMGCN, which integrates diverse drug-related information, to predict novel drug combinations. Specifically, DCMGCN first learns low-dimensional representations of drugs from the drug attributes and similarity networks. Then, by quantifying the degree of the nodes in the known drug-drug network and the similarity between connected nodes, we found the drug-drug network has heterophily and sparseness, which may limit the effectiveness of the graph convolutional network (GCN). Therefore, we introduce two designs to modify GCN. Finally, the drug representations are optimized using modified GCN (MGCN) and used to predict drug combinations. The tests on multiple drug combination datasets show that DCMGCN achieved substantial improvements over state-of-the-art methods. Importantly, our model may embed the mechanism of ground-truth drug pairs into the low-dimensional representation of each drug, which may help to further clarify the understanding of mechanisms of drug action.

As time goes by, language evolves with word semantics changing. Unfortunately, traditional word embedding methods neglect the evolution of language and assume that word representations are static. Although contextualized word embedding models can capture the diverse representations of polysemous words, they ignore temporal information as well. To tackle the aforementioned challenges, we propose a graph-based dynamic word embedding (GDWE) model, which focuses on capturing the semantic drift of words continually. We introduce word-level knowledge graphs (WKGs) to store short-term and long-term knowledge. WKGs can provide rich structural information as supplement of lexical information, which help enhance the word embedding quality and capture semantic drift quickly. Theoretical analysis and extensive experiments validate the effectiveness of our GDWE on dynamic word embedding learning.