Multi-modal Entity Alignment (MMEA) aims to identify equivalent entities across diverse knowledge graphs by leveraging structural, attribute, and visual information. However, real-world datasets frequently suffer from missing modalities, necessitating feature imputation. A critical yet underexplored issue is that not all imputed modalities are inherently trustworthy. Ignoring the aleatoric uncertainty of such modalities introduces severe noise which propagates through the fusion process and degrades alignment performance. To address this challenge, we propose a novel MMEA framework, namely SURE, to Suppress Uncertainty for tRustworthy Entity alignment. Specifically, SURE introduces an uncertainty-aware variational imputation module to estimate the aleatoric uncertainty of generated features. Crucially, rather than using these imputation features blindly, SURE leverages the estimated uncertainty to suppress noise propagation via a confidence-gated multi-modal fusion and an adaptive contrastive learning objective. Extensive experiments on DBP15K datasets demonstrate that SURE significantly outperforms state-of-the-art baselines, exhibiting exceptional robustness particularly in scenarios with high modality missing rates.

In recent years, large language models (LLMs) have emerged as powerful tools for link prediction in knowledge graphs (KGs) due to their strong capabilities in understanding and generation. However, many LLM-based methods still heavily rely on textual descriptions of KGs, limiting their ability to capture structural information and to model complex relational patterns. Although some methods integrate structural embeddings into LLMs, their ability to harness the complementary strengths of both modalities and dynamically prioritize candidate entities based on query context remains limited. In this paper, we propose ST-KGLP, a novel framework that improves link prediction by aligning structural knowledge with textual knowledge and employing query-aware adaptive weighting for candidate selection. Specifically, our proposed ST-KGLP employs a knowledge aligner to bridge the information gap between structural and textual knowledge, and then utilizes a query-aware adaptive weighting strategy that dynamically computes attention weights between query representations and candidate entities, enabling contextually relevant candidate re-ranking for more accurate prediction. Extensive experiments on various datasets show that our ST-KGLP outperforms state-of-the-art approaches, achieving average improvements of 3.81%, 11.52%, 2.22%, and 1.55% across four evaluation metrics. Our code and datasets are available at https://github.com/shijielaw/ST-KGLP.

As generative AI becomes increasingly integrated into marketing, brands face both opportunities and challenges in using AI to co-create content that drives consumer engagement. However, empirical research on the impact of human-AI collaborative content remains limited. This study addresses this gap by combining the Cognitive-Affective Personality System (CAPS) and resonance theory to propose a model that examines how collaboration modes (AI-dominant vs. human-dominant) and content types (informational vs. emotional) influence online brand engagement through cognitive and emotional resonance. Two online experiments investigate their interaction and the mediating role of resonance. The results reveal a key matching effect: AI-dominant collaborations enhance engagement with informational content through cognitive resonance, while human-dominant collaborations strengthen engagement with emotional content through emotional resonance. These findings provide both theoretical and practical insights, demonstrating how aligning collaboration mode with content type can optimize human-AI content marketing and guide brands in leveraging generative AI to improve engagement.

To achieve multi-hop question answering over knowledge graphs (KGQA), many studies have explored converting retrieved subgraphs into textual form and feeding them into large language models (LLMs) to leverage their reasoning capabilities. However, due to the linear and discrete nature of text sequences, model performance may degrade when handling complex questions. To this end, we propose a novel structure-text knowledge synergistic method, BrikQA, which bridges the knowledge gap between knowledge graphs (KGs) and LLMs for multi-hop KGQA. LLMs and KGs complement each other by leveraging explicit topological patterns and implicit knowledge mining to enhance knowledge understanding and address sparsity issues. Experimental results on various datasets demonstrate that BrikQA outperforms state-of-the-art baselines. Our source code is available at https://github.com/shijielaw/BrikQA.

Blood glucose levels, when analyzed as time series data, contain vital information about a patient's glucose fluctuations and trends, which are crucial for assessing pancreatic function and insulin sensitivity. Leveraging these insights through feature extraction and classification can help determine the optimal insulin dosage for patients. Traditional methods of manual feature selection and extraction are laborintensive and inefficient. In this paper, we propose a novel local-global context encoding architecture to extract deep features from blood glucose time series data. We employ a weighted k-nearest neighbors (KNN) classifier to provide dynamic and personalized treatment recommendations. Our experiments, conducted using the UVA/Padova type 1 diabetes simulator and the ShanghaiT2DM real-world diabetes dataset, demonstrate that deep features significantly outperform manually extracted features in recommending insulin dosages. Our approach offers a more accurate and personalized method for managing multiple daily insulin injections, potentially reducing the workload of medical practitioners and improving the efficiency and effectiveness of diabetes management. Code is available at https://github.com/EricWvi/ykw.

With the burgeoning prevalence of diabetes and the urgent imperative for effective dietary management, harnessing the power of image analysis and computer vision technology presents a promising solution to the automatic nutrition estimation challenge. However, constructing a comprehensive frame-work for this endeavor presents its own set of challenges, encompassing multiple intricate procedures, including data collection, image segmentation, food recognition, and volume estimation. In this paper, we introduce a sophisticated and automated framework for Chinese food nutrition estimation, leveraging food images sourced from diabetic patients. In addition to developing models for the various procedures, the framework necessitates label images for model training. These label images encompass ground truth masks delineating precise object boundaries for image segmentation, food type labels for food recognition, and comprehensive nutrition tables for each food item. Through rigorous experimentation and validation, our framework has demonstrated its efficacy, offering a convenient and practical tool for managing dietary requirements in diabetes care.

Understanding documents with rich layouts and multi-modal components is a long-standing and practical task. Recent Large Vision-Language Models (LVLMs) have made remarkable strides in various tasks, particularly in single-page document understanding (DU). However, their abilities on long-context DU remain an open problem. This work presents MMLONGBENCH-DOC, a long-context, multi- modal benchmark comprising 1,082 expert-annotated questions. Distinct from previous datasets, it is constructed upon 135 lengthy PDF-formatted documents with an average of 47.5 pages and 21,214 textual tokens. Towards comprehensive evaluation, answers to these questions rely on pieces of evidence from (1) different sources (text, image, chart, table, and layout structure) and (2) various locations (i.e., page number). Moreover, 33.7\% of the questions are cross-page questions requiring evidence across multiple pages. 20.6\% of the questions are designed to be unanswerable for detecting potential hallucinations. Experiments on 14 LVLMs demonstrate that long-context DU greatly challenges current models. Notably, the best-performing model, GPT-4o, achieves an F1 score of only 44.9\%, while the second-best, GPT-4V, scores 30.5\%. Furthermore, 12 LVLMs (all except GPT-4o and GPT-4V) even present worse performance than their LLM counterparts which are fed with lossy-parsed OCR documents. These results validate the necessity of future research toward more capable long-context LVLMs.

Liangming Pan, Xiaobao Wu, Xinyuan Lu, Anh Tuan Luu, William Yang Wang, Min-Yen Kan, Preslav Nakov. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Current scientific fact-checking benchmarks exhibit several shortcomings, such as biases arising from crowd-sourced claims and an overreliance on text-based evidence.We present SCITAB, a challenging evaluation dataset consisting of 1.2K expert-verified scientific claims that 1) originate from authentic scientific publications and 2) require compositional reasoning for verification.The claims are paired with evidence-containing scientific tables annotated with labels.Through extensive evaluations, we demonstrate that SCITAB poses a significant challenge to state-of-the-art models, including table-based pretraining models and large language models.All models except GPT-4 achieved performance barely above random guessing.Popular prompting techniques, such as Chain-of-Thought, do not achieve much performance gains on SCITAB.Our analysis uncovers several unique challenges posed by SCITAB, including table grounding, claim ambiguity, and compositional reasoning.

Video harmonization aims to adjust the foreground of a composite video to make it compatible with the background. So far, video harmonization has only received limited attention and there is no public dataset for video harmonization. In this work, we construct a new video harmonization dataset HYouTube by adjusting the foreground of real videos to create synthetic composite videos. Moreover, we consider the temporal consistency in video harmonization task. Unlike previous works which establish the spatial correspondence, we design a novel framework based on the assumption of color mapping consistency, which leverages the color mapping of neighboring frames to refine the current frame. Extensive experiments on our HYouTube dataset prove the effectiveness of our proposed framework. Our dataset and code are available at https://github.com/bcmi/Video-Harmonization-Dataset-HYouTube.

Automatic question generation is an important problem in natural language processing. In this paper, we propose a novel adaptive copying recurrent neural network model to tackle the problem of question generation from sentences and paragraphs. The proposed model adds a copying mechanism component onto a bidirectional LSTM architecture to generate more suitable questions adaptively from the input data. Our experimental results show the proposed model can outperform the state-of-the-art question generation methods in terms of BLEU and ROUGE evaluation scores.