No existing dataset adequately tests how well language models can incrementally update entity summaries – a crucial ability as these models rapidly advance. The Incremental Entity Summarization (IES) task is vital for maintaining accurate, up-to-date knowledge. To address this, we introduce , a fully synthetic dataset designed to expose real-world IES challenges. This dataset addresses issues like incorrect entity association and incomplete information, capturing real-world complexity by generating diverse attributes, summaries, and unstructured paragraphs with 99% alignment accuracy between generated summaries and paragraphs. Extensive experiments demonstrate the dataset’s difficulty – state-of-the-art LLMs struggle to update summaries with an F1 higher than 80.4%. We will open-source the benchmark and the evaluation metrics to help the community make progress on IES tasks.

Intent, typically clearly formulated and planned, functions as a cognitive framework for communication and problem-solving. This paper introduces the concept of Speaking with Intent (SWI) in large language models (LLMs), where the explicitly generated intent encapsulates the model’s underlying intention and provides high-level planning to guide subsequent analysis and action. By emulating deliberate and purposeful thoughts in the human mind, SWI is hypothesized to enhance the reasoning capabilities and generation quality of LLMs. Extensive experiments on text summarization, multi-task question answering, and mathematical reasoning benchmarks consistently demonstrate the effectiveness and generalizability of Speaking with Intent over direct generation without explicit intent. Further analysis corroborates the generalizability of SWI under different experimental settings. Moreover, human evaluations verify the coherence, effectiveness, and interpretability of the intent produced by SWI. The promising results in enhancing LLMs with explicit intents pave a new avenue for boosting LLMs’ generation and reasoning abilities with cognitive notions.

Despite recent advancements in visionlanguage models, their performance remains suboptimal on images from non-western cultures, due to underrepresentation in training datasets.Various benchmarks have been proposed to test models' cultural inclusivity, but they have limited coverage of cultures and do not adequately assess cultural diversity across universal as well as culture-specific local concepts.To address these limitations, we introduce the GLOBALRG benchmark, comprising two challenging tasks: retrieval across universals and cultural visual grounding.The former task entails retrieving culturally-diverse images for universal concepts from 50 countries, while the latter aims at grounding culture-specific concepts within images from 15 countries.Our evaluation across a wide range of models reveals that the performance varies significantly across cultures -underscoring the necessity for enhancing multicultural understanding in vision-language models.Our

Memes are a widely popular tool for web users to express their thoughts using visual metaphors.Understanding memes requires recognizing and interpreting visual metaphors with respect to the text inside or around the meme, often while employing background knowledge and reasoning abilities.We present the task of meme captioning and release a new dataset, MEMECAP.Our dataset contains 6.3K memes along with the title of the post containing the meme, the meme captions, the literal image caption, and the visual metaphors.Despite the recent success of vision and language (VL) models on tasks such as image captioning and visual question answering, our extensive experiments using state-of-the-art VL models show that they still struggle with visual metaphors, and perform substantially worse than humans.

Generating commonsense explanations requires reasoning about commonsense knowledge beyond what is explicitly mentioned in the context.Existing models use commonsense knowledge graphs such as ConceptNet to extract a subgraph of relevant knowledge pertaining to concepts in the input.However, due to the large coverage and, consequently, vast scale of ConceptNet, the extracted subgraphs may contain loosely related, redundant and irrelevant information, which can introduce noise into the model.We propose to address this by applying a differentiable graph compression algorithm that focuses on more salient and relevant knowledge for the task.The compressed subgraphs yield considerably more diverse outputs when incorporated into models for the tasks of generating commonsense and abductive explanations.Moreover, our model achieves better quality-diversity tradeoff than a large language model with 100 times the number of parameters.Our generic approach can be applied to additional NLP tasks that can benefit from incorporating external knowledge.1

Many individual investors who have relatively little financial knowledge rely heavily on the information from news stories when making investment decisions in the stock market. However, these stories may not reflect future stock prices because of the subjectivity in the news; stock prices may instead affect the news contents. This study aims to discover whether it is news or stock prices that have a greater impact on the other. To achieve this, we analyze the relationship between news sentiment and stock prices based on time series analysis using five different classification models. Our experimental results show that stock prices have a bigger impact on the news contents than news does on stock prices.