Topic modeling plays a critical role in organizing and understanding large-scale web content. While neural topic models (NTMs) based on variational autoencoders (VAEs) have achieved notable success in analyzing textual data, they remain limited in addressing the multimodal nature of modern web content. Existing unimodal or multimodal extensions often suffer from posterior collapse and fail to capture the directional semantics inherent in both text and images, resulting in incoherent topics and limited interpretability. To address these challenges, we propose MM-vNTM (MultiModal Neural Topic Model with von Mises-Fisher Mixtures), a framework for web-scale topic discovery over multimodal data. MM-vNTM leverages pre-aligned cross-modal embeddings as inputs and jointly models document-level representations of text and image modalities in a shared hyperspherical latent space. Furthermore, it defines topics as mixtures of von Mises-Fisher (vMF) distributions in the L2-normalized word embedding space, explicitly capturing directional similarity. Experiments on multimedia web datasets demonstrate that MM-vNTM consistently outperforms state-of-the-art unimodal and multimodal baselines in terms of overall topic quality, highlighting its effectiveness for real-world web scenarios.

Topic modeling plays a vital role in uncovering latent semantic structures from large-scale biomedical corpora. While classical probabilistic models such as Latent Dirichlet Allocation (LDA) have been widely used, they often struggle with scalability and capturing complex semantic relationships in domain-specific contexts. Neural topic models (NTMs) based on variational autoencoders (VAEs) provide a more flexible and scalable alternative. However, existing NTMs face two key challenges: the neglect of the underlying geometric structure of semantic spaces and the lack of domain-specific adaptation, both of which contribute to suboptimal topic coherence and weak document clustering. To address these limitations, we pro-pose BiovMNVTM, a geometry-aware variational topic modeling framework tailored for biomedical text analysis. BiovMNVTM leverages the von Mises-Fisher (vMF) mixture distribution to model directional relationships on the hypersphere and integrates contextualized biomedical embeddings from BioBERT to capture rich, domain-specific semantics. Comprehensive experiments on multiple biomedical datasets show that BiovMNVTM achieves superior performance in terms of topic quality and clustering ability, demonstrating the effectiveness of incorporating geomet-ric and domain-aware modeling in biomedical topic discovery.

Nowadays, a large number of Knowledge Graph Completion (KGC) methods have been proposed by using embedding based manners, to overcome the incompleteness problem faced with knowledge graph (KG). One important recent innovation in Natural Language Processing (NLP) domain is the employ of deep neural models that make the most of pre-training, culminating in BERT, the most popular example of this line of approaches today. Recently, a series of new KGC methods introducing a pre-trained language model, such as KG-BERT, have been developed and released compelling performance. However, previous pre-training based KGC methods usually train the model by using simple training task and only utilize one-hop relational signals in KG, which leads that they cannot model high-order semantic contexts and multi-hop complex relatedness. To overcome this problem, this article presents a novel pre-training framework for KGC task, which especially consists of both one-hop relation level task (low-order) and multi-hop meta-graph level task (high-order). Hence, the proposed method can capture not only the elaborate sub-graph structure but also the subtle semantic information on the given KG. The empirical results show the efficiency of the proposed method on the widely used real-world datasets.