While Large language models (LLMs) have shown to exhibit remarkable performance in a wide range of NLP tasks, they often struggle to interpret and reason over multi-hop questions in open-domain question answering (ODQA) settings. While popular prompt approaches such as Chain-of-Thought and Plan-and-Solve facilitate more manageable questions for OQDA via task decomposition, these approaches are prone to generating erroneous and redundant intermediate steps in multi-hop queries due to limited capacity for modeling complex entity relationships. In this paper, we introduce a novel prompt approach for multi-hop QA viz., MoDeGraph (Multi-Hop Dependency Graphs), that is designed to steer LLMs to extract and model entity relationships in complex questions. MoDeGraph constructs a dependency graph from LLM-generated entity-relation triples to enable more coherent and human-like multi-step reasoning. Experimental results in knowledge-intensive tasks for multi-hop QA demonstrate our approach produces more coherent and faithful reasoning chains as well as consistent increase in QA performance across several benchmark datasets.

Financial sentiment analysis can help in understanding market trends and help organizations and individuals make important business decisions. Several machine learning approaches have been used for financial sentiment analysis over the years ranging from lexicon-based approaches to the use of deep neural networks and transformer-based models. Recent advances in Large Language Models (LLM) have prompted the use of these models for various Natural Language Processing (NLP) tasks, however, these models have not yet been substantially explored in the financial domain. In this paper, we evaluate the performance of various LLMs and we introduce a small benchmark dataset consisting of excerpts extracted from the Federal Reserve chair’s speeches. We use this dataset along with other existing datasets to evaluate LLMs using in-context learning approaches. We compare the F1 scores of these models with the state-of-the-art BERT-based models and analyze our results.

Abstract The OffensEval shared tasks organized as part of SemEval-2019–2020 were very popular, attracting over 1300 participating teams. The two editions of the shared task helped advance the state of the art in offensive language identification by providing the community with benchmark datasets in Arabic, Danish, English, Greek, and Turkish. The datasets were annotated using the OLID hierarchical taxonomy, which since then has become the de facto standard in general offensive language identification research and was widely used beyond OffensEval. We present a survey of OffensEval and related competitions, and we discuss the main lessons learned. We further evaluate the performance of Large Language Models (LLMs), which have recently revolutionalized the field of Natural Language Processing. We use zero-shot prompting with six popular LLMs and zero-shot learning with two task-specific fine-tuned BERT models, and we compare the results against those of the top-performing teams at the OffensEval competitions. Our results show that while some LMMs such as Flan-T5 achieve competitive performance, in general LLMs lag behind the best OffensEval systems.