Visual hallucination remains a major obstacle to the reliability of Large Vision-Language Models (LVLMs). We argue that this issue originates from a fundamental statistical misspecification: the conventional softmax attention implicitly assumes i.i.d. noise, yet real LVLM attention patterns exhibit structured and competitive biases (e.g., attention sinks) that violate this assumption. To address this mismatch, we introduce Latent Attention Denoising (LAD), a principled and training-free framework that recasts attention calibration as a one-step score-based denoising process. LAD employs an interpretable energy function to derive an analytic score and applies a single Langevin-inspired update to actively steer corrupted attention logits toward more faithful configurations. This intervention imposes negligible computational overhead and operates at a speed comparable to standard greedy decoding. Extensive evaluations across diverse architectures confirm that LAD achieves superior performance on both generative and discriminative tasks, effectively mitigating hallucinations while maintaining efficiency comparable to standard decoding.

Topic modeling plays a critical role in organizing and understanding large-scale web content. While neural topic models (NTMs) based on variational autoencoders (VAEs) have achieved notable success in analyzing textual data, they remain limited in addressing the multimodal nature of modern web content. Existing unimodal or multimodal extensions often suffer from posterior collapse and fail to capture the directional semantics inherent in both text and images, resulting in incoherent topics and limited interpretability. To address these challenges, we propose MM-vNTM (MultiModal Neural Topic Model with von Mises-Fisher Mixtures), a framework for web-scale topic discovery over multimodal data. MM-vNTM leverages pre-aligned cross-modal embeddings as inputs and jointly models document-level representations of text and image modalities in a shared hyperspherical latent space. Furthermore, it defines topics as mixtures of von Mises-Fisher (vMF) distributions in the L2-normalized word embedding space, explicitly capturing directional similarity. Experiments on multimedia web datasets demonstrate that MM-vNTM consistently outperforms state-of-the-art unimodal and multimodal baselines in terms of overall topic quality, highlighting its effectiveness for real-world web scenarios.

In recent years, axial data, where observations are treated as axes of direction, has gained prominence in a range of complex tasks, including gene expression data clustering, blind speech separation, and depth image analysis. However, prevailing methods for axial data modeling mainly rely on shallow probabilistic models, which often overlook the hidden and hierarchical dependencies in the latent space. These methods also require a separate, human-engineered feature extractor to obtain features from raw axial data for downstream tasks. This work introduces a novel framework, Axial Variational Autoencoders (AVAEs), for modeling and representation learning of axial data by leveraging a deep generative model, the Variational Autoencoder (VAE). Unlike existing approaches, our method can autonomously learn more expressive representations from axial data by designing a VAE that uses the Watson distribution as the latent prior. Furthermore, we introduce a tailored reparameterization technique to support stable training. We validate the effectiveness of our model through experiments on simulated axial datasets and a real-world application.

The success of prompt learning when adapted to the fully qualified type name (FQN) inference has been demonstrated in the literature. However, the understanding of its success is limited in model outputs and model structures. In this article, we conduct a thorough study on the behaviors of prompt learning in FQN inference from the perspectives of sensitivity and usefulness. Rather than simply masking some knowledge, we first perform sensitivity analysis on five aspects to reveal how much FQN knowledge to include, how much to mask, and where to mask, and then yield an efficient configuration strategy. We further conduct a usefulness analysis in three aspects to demonstrate the superiority of the proposed configuration strategy. This suggests that the strong performance of our model is attributable to the homogeneity among large code pre-training, FQN prompt learning, and type inference as a fill-in-blank task. Finally, we summarize a practical guideline on best practices and pitfalls to avoid when applying prompt learning to FQN inference and other software engineering (SE) tasks.

Large language models (LLMs) have demonstrated impressive performance in code generation, particularly when augmented with chain-of-thought (CoT) prompting techniques. They break down requirements into intermediate reasoning steps, which act as design rationales to guide LLMs in writing code like human programmers. Thus, the quality of these steps is crucial for ensuring the correctness and reliability of the generated code. However, the specific factors influencing the quality of CoT generated by LLMs remain largely unexplored. To what extent can we trust the thoughts generated by LLMs? How good are they? This paper empirically explores the external and internal factors of why LLMs generate unsatisfactory CoTs by analyzing 1,023 failed code samples on two widely used code generation benchmarks. We also evaluate their impact on code generation performance by analyzing 210 CoT-code pairs and refining the unsatisfied CoTs by prompting LLMs. Our study yields the following findings: 1) Among the factors affecting CoT quality, external factors account for 53.60%, primarily including unclear requirements and lack of contextual information. Internal factors make up 40.10%, mainly due to inconsistencies between CoT and prompts caused by LLMs’ misunderstanding of the instructions. 2) Despite CoT being correct, 18.5% of the generated code still contains errors. This is primarily due to LLMs failing to follow instructions, leading to inconsistencies between CoT and the code. Additionally, we found that even when the code is correct, there is an 11.90% chance that the CoT contains errors. 3) Our further research on refining the low-quality CoTs reveals that LLMs can improve CoT, especially when providing detailed CoT problem information. Our findings shed light on the underlying issues that hinder the effectiveness of CoT in LLM-based code generation, offering valuable insights for enhancing both the reasoning process and the overall reliability of code generation.

Topic modeling plays a vital role in uncovering latent semantic structures from large-scale biomedical corpora. While classical probabilistic models such as Latent Dirichlet Allocation (LDA) have been widely used, they often struggle with scalability and capturing complex semantic relationships in domain-specific contexts. Neural topic models (NTMs) based on variational autoencoders (VAEs) provide a more flexible and scalable alternative. However, existing NTMs face two key challenges: the neglect of the underlying geometric structure of semantic spaces and the lack of domain-specific adaptation, both of which contribute to suboptimal topic coherence and weak document clustering. To address these limitations, we pro-pose BiovMNVTM, a geometry-aware variational topic modeling framework tailored for biomedical text analysis. BiovMNVTM leverages the von Mises-Fisher (vMF) mixture distribution to model directional relationships on the hypersphere and integrates contextualized biomedical embeddings from BioBERT to capture rich, domain-specific semantics. Comprehensive experiments on multiple biomedical datasets show that BiovMNVTM achieves superior performance in terms of topic quality and clustering ability, demonstrating the effectiveness of incorporating geomet-ric and domain-aware modeling in biomedical topic discovery.

Clustering high-dimensional directional data (i.e., L2 normalized vectors) presents significant challenges due to the intricate spherical representations of latent embeddings and the limitations of classical (non-deep) clustering techniques. Moreover, dynamically inferring the number of clusters remains a fundamental issue in existing deep clustering methods, especially those involving complex model-selection criteria. This paper addresses these challenges by introducing a novel deep nonparametric clustering framework that employs hyperspherical latent embeddings within a Variational Autoencoder architecture, enhanced by an infinite Von Mises-Fisher Mixture Model as a dynamic prior. This approach enables automatic adaptation of cluster numbers during training, eliminating the need for predefined clusters and traditional model selection processes. Our scalable architecture effectively integrates In-vMFMM with hyperspherical embeddings to tackle the complexities of directional data. Utilizing a joint training strategy, our method alternates between updating neural network parameters and adjusting mixture model priors via nonparametric variational Bayes. Empirical evaluations on benchmark datasets, including complex ImageNet-50, demonstrate that our approach significantly outperforms state-of-the-art deep nonparametric clustering methods. It also robustly estimates the number of clusters, showcasing its effectiveness and versatility in handling high-dimensional directional data.

Dataflow graphs (DFGs) capture definitions (defs) and uses across program blocks, which is a fundamental program representation for program analysis, testing and maintenance. However, dynamically typed programming languages like Python present implicit dataflow issues that make it challenging to determine def-use flow information at compile time. Static analysis methods like Soot and WALA are inadequate for handling these issues, and manually enumerating comprehensive heuristic rules is impractical. Large pre-trained language models (LLMs) offer a potential solution, as they have powerful language understanding and pattern matching abilities, allowing them to predict implicit dataflow by analyzing code context and relationships between variables, functions, and statements in code. We propose leveraging LLMs’ in-context learning ability to learn implicit rules and patterns from code representation and contextual information to solve implicit dataflow problems. To further enhance the accuracy of LLMs, we design a five-step chain of thought (CoT) and break it down into an Artificial Intelligence (AI) chain, with each step corresponding to a separate AI unit to generate accurate DFGs for Python code. Our approach’s performance is thoroughly assessed, demonstrating the effectiveness of each AI unit in the AI Chain. Compared to static analysis, our method achieves 82% higher def coverage and 58% higher use coverage in DFG generation on implicit dataflow. We also prove the indispensability of each unit in the AI Chain. Overall, our approach offers a promising direction for building software engineering tools by utilizing foundation models, eliminating significant engineering and maintenance effort, but focusing on identifying problems for AI to solve.