Multimodal Large Language Models (MLLMs) have displayed remarkable performance in multi-modal tasks, particularly in visual comprehension. However, we reveal that MLLMs often generate incorrect answers even when they understand the visual content. To this end, we manually construct a benchmark with 12 categories and design evaluation metrics that assess the degree of error in MLLM responses even when the visual content is seemingly understood. Based on this benchmark, we test 15 leading MLLMs and analyze the distribution of attention maps and logits of some MLLMs. Our investigation identifies two primary issues: 1) most instruction tuning datasets predominantly feature questions that "directly" relate to the visual content, leading to a bias in MLLMs’ responses to other indirect questions, and 2) MLLMs’ attention to visual tokens is notably lower than to system and question tokens. We further observe that attention scores between questions and visual tokens as well as the model’s confidence in the answers are lower in response to misleading questions than to straightforward ones. To address the first challenge, we introduce a paired positive and negative data construction pipeline to diversify the dataset. For the second challenge, we propose to enhance the model’s focus on visual content during decoding by refining the text and visual prompt. For the text prompt, we propose a content guided refinement strategy that performs preliminary visual content analysis to generate structured information before answering the question. Additionally, we employ a visual attention refinement strategy that highlights question-relevant visual tokens to increase the model’s attention to visual content that aligns with the question. Extensive experiments demonstrate that these challenges can be significantly mitigated with our proposed dataset and techniques.

In this paper, we address the significant challenge of data scarcity in the field of crack segmentation, a key aspect of structural health monitoring. To tackle this, we introduce the CrackGrowDiff framework, an innovative approach for expanding crack datasets. Utilizing a two-stage controllable generation process that combines a random walk algorithm and semantic diffusion models, our framework minimizes discrepancy of misalignment between synthetic data and original data while enhancing data informativeness. We further ensure the quality and informativeness of synthetic data through feature space diversity, employing a pre-trained Variational Autoencoder (VAE) for selection based on Kullback-Leibler (KL) divergence. Comparative experiments demonstrate CrackGrowDiff’s superiority over traditional data augmentation and GANs-based methods, making it a substantial advancement in addressing the data scarcity in crack segmentation tasks. A DEMO and related code will be made public: https://huggingface.co/spaces/QinLei086/Two-stage-SDM-for-crack-dataset-expending

This paper proposes Copy-On-Risk (COR) honeypot solution and implementation. Our prototype COR-Honeypot defends against attacks by creating on demand a personalized honeypot by cloning the victim and isolating the attack to the newly created honeypot. We measure the effectiveness of COR-Honeypot on resource usage and similarity of personalized honeypot to the victim. Results over different configurations of COR-Honeypot have demonstrated the benefits of the proposed solution.