Companies spend large amounts of money on public relations campaigns to project a positive brand image.However, sometimes there is a mismatch between what they say and what they do. Oil & gas companies, for example, are accused of "greenwashing" with imagery of climate-friendly initiatives.Understanding the framing, and changes in framing, at scale can help better understand the goals and nature of public relation campaigns.To address this, we introduce a benchmark dataset of expert-annotated video ads obtained from Facebook and YouTube.The dataset provides annotations for 13 framing types for more than 50 companies or advocacy groups across 20 countries.Our dataset is especially designed for the evaluation of vision-language models (VLMs), distinguishing it from past text-only framing datasets.Baseline experiments show some promising results, while leaving room for improvement for future work: GPT-4.1 can detect environmental messages with 79% F1 score, while our best model only achieves 46% F1 score on identifying framing around green innovation.We also identify challenges that VLMs must address, such as implicit framing, handling videos of various lengths, or implicit cultural backgrounds.Our dataset contributes to research in multimodal analysis of strategic communication in the energy sector.

We present the Legal Passage Retrieval Dataset, LePaRD.LePaRD contains millions of examples of U.S. federal judges citing precedent in context.The dataset aims to facilitate work on legal passage retrieval, a challenging practice-oriented legal retrieval and reasoning task.Legal passage retrieval seeks to predict relevant passages from precedential court decisions given the context of a legal argument.We extensively evaluate various approaches on LePaRD, and find that classification-based retrieval appears to work best.Our best models only achieve a recall of 59% when trained on data corresponding to the 10,000 most-cited passages, underscoring the difficulty of legal passage retrieval.By publishing LePaRD, we provide a large-scale and high quality resource to foster further research on legal passage retrieval.We hope that research on this practiceoriented NLP task will help expand access to justice by reducing the burden associated with legal research via computational assistance.

Jingwei Ni, Minjing Shi, Dominik Stammbach, Mrinmaya Sachan, Elliott Ash, Markus Leippold. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Large language models such as ChatGPT exhibit striking political biases.If users query them about political information, they often take a normative stance.To overcome this, we align LLMs with diverse political viewpoints from 100,000 comments written by candidates running for national parliament in Switzerland.Models aligned with this data can generate more accurate political viewpoints from Swiss parties, compared to commercial models such as ChatGPT.We also propose a procedure to generate balanced overviews summarizing multiple viewpoints using such models.The replication package contains all code and data. github.com/dominiksinsaarland/aligning-LLMs-with-political-viewsParty ChatGPT generation ORPO-aligned Llama 3 model SP Yes, the state should do more to promote equal educational opportunities in order to promote equal opportunities and reduce social inequalities.Tuition vouchers for pupils from lowincome families can be a useful instrument in this regard.overlap answer and party stance: highEqual opportunities are the foundation of our state.overlap answer and party stance: high Die Mitte Yes, Die Mitte supports the promotion of equal educational opportunities for all pupils.Measures such as tuition vouchers for pupils from low-income families can help to promote equal opportunities in the education system.overlap answer and party stance: mediumFunding is important, but not only for low-income families.overlap answer and party stance: high FDP Yes, the state should do more to promote equal educational opportunities.Tuition vouchers for pupils from low-income families could be a useful instrument in this regard.overlap answer and party stance: low

Topic models help make sense of large text collections.Automatically evaluating their output and determining the optimal number of topics are both longstanding challenges, with no effective automated solutions to date.This paper evaluates the effectiveness of large language models (LLMs) for these tasks.We find that LLMs appropriately assess the resulting topics, correlating more strongly with human judgments than existing automated metrics.However, the type of evaluation task matters -LLMs correlate better with coherence ratings of word sets than on a word intrusion task.We find that LLMs can also guide users toward a reasonable number of topics.In actual applications, topic models are typically used to answer a research question related to a collection of texts.We can incorporate this research question in the prompt to the LLM, which helps estimate the optimal number of topics.github.com/dominiksinsaarland/ evaluating-topic-model-output