Multimodal Large Language Models (MLLMs) hallucinate, resulting in an emerging topic of visual hallucination evaluation (VHE). This paper contributes a ChatGPT-Prompted visual hallucination evaluation Dataset (PhD) for objective VHE at a large scale. The essence of VHE is to ask an MLLM questions about specific images to assess its susceptibility to hallucination. Depending on what to ask (objects, attributes, sentiment, etc.) and how the questions are asked, we structure PhD along two dimensions, i.e. task and mode. Five visual recognition tasks, ranging from low-level (object/attribute recognition) to middle-level (sentiment/position recognition and counting), are considered. Besides a normal visual QA mode, which we term PhD-base, PhD also asks questions with specious context (PhD-sec) or with incorrect context (PhD-icc), or with AI-generated counter common sense images (PhD-ccs). We construct PhD by a ChatGPT-assisted semi-automated pipeline, encompassing four pivotal modules: task-specific hallucinatory item (hitem) selection, hitem-embedded question generation, specious/incorrect context generation, and counter-common-sense (CCS) image generation. With over 14k daily images, 750 CCS images and 102k VQA triplets in total, PhD reveals considerable variability in MLLMs’ performance across various modes and tasks, offering valuable insights into the nature of hallucination. As such, PhD stands as a potent tool not only for VHE but may also play a significant role in the refinement of MLLMs.

Sketch animation, which brings static sketches to life by generating dynamic video sequences, has found widespread applications in GIF design, cartoon production, and daily entertainment. While current methods for sketch animation perform well in single-object sketch animation, they struggle in multi-object scenarios. By analyzing their failures, we identify two major challenges of transitioning from single-object to multi-object sketch animation: object-aware motion modeling and complex motion optimization. For multi-object sketch animation, we propose MoSketch based on iterative optimization through Score Distillation Sampling (SDS) and thus animating a multi-object sketch in a training-data free manner. To tackle the two challenges in a divide-and-conquer strategy, MoSketch has four novel modules, i.e., LLM-based scene decomposition, LLM-based motion planning, multi-grained motion refinement, and compositional SDS. Extensive qualitative and quantitative experiments demonstrate the superiority of our method over existing sketch animation approaches. MoSketch takes a pioneering step towards multi-object sketch animation, opening new avenues for future research and applications.

MOTIVATION: Spatial transcriptomics (ST) addresses the loss of spatial context in single-cell RNA-sequencing by simultaneously capturing gene expression and spatial location information. A critical task of ST is the identification of spatial domains. However, challenges such as high noise levels and data sparsity make the identification process more difficult. RESULTS: To tackle these challenges, STMGAMF, a multi-view graph convolutional network model that employs an adaptive adjacency matrix and a multi-strategy fusion mechanism is proposed. STMGAMF dynamically adjusts the edge weights to capture complex spatial structures during training by implementing the adaptive adjacency matrix and optimizes the embedded features through the multi-strategy fusion mechanism. STMGAMF is evaluated on multiple ST datasets and outperforms existing algorithms in tasks like spatial domain identification, visualization, and spatial trajectory inference. Its robust performance in spatial domain identification and strong generalization capability position STMGAMF as a valuable tool for unraveling the complexity of tissue structures and underlying biological processes. AVAILABILITY AND IMPLEMENTATION: Source code is available at Github (https://github.com/Fuyh0628/STMGAMF) and Zenodo (https://zenodo.org/records/15103358).

The rapid development of single-cell sequencing technologies has provided a robust technical support for the efficient resolution of multiple levels of molecular information from a single-cell population. However, the data produced by these technologies often contain a lot of noise and differences in characteristics that make it difficult to integrate and analyze single-cell multi-omics data. In this study, there is a growing demand for methods to integrate single-cell multi-omics data, which is expected to enhance the ability to reveal cellular heterogeneity and provide new biological perspectives for a deeper understanding of cellular phenotypes by jointly analyzing multi-omics data. We propose LONMF, a non-negative matrix factorization algorithm combining graph Laplacian and optimal transmission to enhance clustering performance and interpretability. We apply LONMF to visualize and cluster multi-pair single-cell multi-omics data, including 10X-multi-group, CITE-seq, and TEA-multi-group seq, to facilitate marker characterization and gene ontology enrichment analysis and to provide rich biological insights for downstream analyses. Our comprehensive benchmarking demonstrates that LONMF exhibits comparable performance compared with the current state-of-the-art in cell clustering and outperforms other methods in terms of biological interpretability.