Image captioning is crucial for multimodal understanding, bridging visual content and natural language. Despite recent advancements in Large Multimodal Models (LMMs), when faced with unseen entities or scenes in the open world, even when attempting to leverage learned knowledge, models still struggle with vague and inaccurate descriptions, and may even generate knowledge hallucinations. A key reason is that the model fails to effectively integrate knowledge with visual information, limiting its understanding of visual content. Thus, we propose Adaptive Knowledge Graph-guided Multimodal Alignment (AKGMA) for image captioning, which enhances semantic understanding in open-world scenes through visual knowledge reasoning, reducing knowledge hallucinations and improving caption quality. It consist three key components: Entity-guided Knowledge Aligner (EKA), Adaptive Knowledge Graph Construction (AKGC), and Scene-Context Knowledge Adapter (SCKA). EKA connects visual entities to knowledge graphs, providing structured knowledge to a small language model, which interacts with a visual encoder to acquire visual knowledge. AKGC uses reinforcement learning to build image-relevant subgraphs to optimize knowledge prompts and improve knowledge hallucinations. SCKA leverages scene graph annotations to extract visual contextual knowledge and inject it into Large Language Models (LLMs), ensuring the generated descriptions are consistent with the image's details. Additionally, we introduce UniKnowCap, a new image knowledge description dataset spanning various open-world knowledge domains, designed to evaluate the knowledge accuracy and detail consistency of model-generated descriptions. Extensive experiments show our model outperforms baselines across multiple metrics.

Social media is crucial for information spread, necessitating effective rumor detection to curb misinformation’s societal effects. Current methods struggle against complex propagation influenced by bots, coordinated accounts, and echo chambers, which fragment information and increase risks of misjudgments and model vulnerability. To counteract these issues, we introduce a new rumor detection framework, the Narrative-Integrated Metapath Graph Auto-Encoder (NIMGA). This model consists of two core components: (1) Metapath-based Heterogeneous Graph Reconstruction. (2) Narrative Reordering and Perspective Fusion. The first component dynamically reconstructs propagation structures to capture complex interactions and hidden pathways within social networks, enhancing accuracy and robustness. The second implements a dual-agent mechanism for viewpoint distillation and comment narrative reordering, using LLMs to refine diverse perspectives and semantic evolution, revealing patterns of information propagation and latent semantic correlations among comments. Extensive testing confirms our model outperforms existing methods, demonstrating its effectiveness and robustness in enhancing rumor representation through graph reconstruction and narrative reordering.

The social biases and unwelcome stereotypes revealed by pretrained language models are becoming obstacles to their application. Compared to numerous debiasing methods targeting word level, there has been relatively less attention on biases present at phrase level, limiting the performance of debiasing in discipline domains. In this paper, we propose an automatic multi-token debiasing pipeline called General Phrase Debiaser, which is capable of mitigating phrase-level biases in masked language models. Specifically, our method consists of a phrase filter stage that generates stereotypical phrases from Wikipedia pages as well as a model debias stage that can debias models at the multi-token level to tackle bias challenges on phrases. The latter searches for prompts that trigger model’s bias, and then uses them for debiasing. State-of-the-art results on standard datasets and metrics show that our approach can significantly reduce gender biases on both career and multiple disciplines, across models with varying parameter sizes.

As Large Language Models (LLMs) continue to enhance their capabilities in multi-agent collaborative applications, the unpredictability of the generative content risks has intensified. Particularly in ongoing interaction scenarios with users, it remains unclear whether there is generative content risk drift over time. In this context, "drift risk" refers to the trend of progressively intensified content risk that emerges during sustained adversarial interactions among LLM agents. Additionally, the high cost associated with constructing complex adversarial environments for agents impedes the transferability of current assessment methods for LLMs to multi-agent adversarial scenarios. In this paper, we introduce a low-cost and lightweight framework for assessing content risk drift of LLMs, named CRDA. This framework, bypassing the need for constructing complex adversarial environments, offers a method that integrates roles and responses memory to guide automatically multi-round adversarial interactions among LLM agents, that is, multiple agents as avatars of a single LLM. In this approach, LLM agents enable the analysis of content risk drift of this LLM. Moreover, we explore the impact of restricted roles and the unsafe content with negative viewpoints in responses memory on the content risk drift of LLMs. Considering the rapid advancement of Chinese LLM capabilities, this study selects real adversarial topics in Chinese and assesses content risk drift of five representative Chinese LLMs. The research finds that these LLMs exhibit significant content risk drift even after a certain safety alignment, showing an initial increase followed by a gradual decrease. As the adversarial process progresses, under restricted roles, agents more effectively breach the model's safety alignment, leading to content risk drift of the LLM. The content drift risk assessment can be quantified specifically by measuring the deterioration rate at which LLM agents deteriorate from positive to negative and analyzing the underlying trends during the automatically multi-round adversarial interactions. In restricted and general roles adversarial interactions, all agents of five Chinese LLMs exhibit an overall average increase of 31.5% and 16.38% in the cumulative deterioration rate respectively by the 10th round, compared to the baseline no-roles adversarial interactions. Finally, we hope that the framework and findings presented in this paper will offer valuable insights for research on safety alignment in LLM agents during adversarial processes.

Nowadays, private clouds are widely used for resource sharing. Hadoop-based clusters are the most popular implementations for private clouds. However, because workload traces are not publicly available, few previous work compares and evaluates different cloud solutions with publicly available benchmarks. In this paper, we use a recently-released Cloud benchmarks suite - CloudRank-D to quantitatively evaluate five different Hadoop task schedulers, including FIFO, capacity, naïve fair sharing, fair sharing with delay, and HOD (Hadoop On Demand) scheduling. Our experiments show that with an appropriate scheduler, the throughput of a private cloud can be improved by 20%.