Stance detection aims to determine the attitude or viewpoint expressed in a document regarding a specific target. Recent advancements in Large Language Models (LLMs), such as Chain-of-Thought (CoT) prompting, have improved the reasoning capabilities of these models by integrating intermediate rationales. However, the efficacy of CoT can be limited by the model’s internal knowledge, resulting in inaccurate rationales that compromise the subsequent stance prediction. This limitation could further lead to hallucinations, where LLMs produce unfaithful responses and erroneous reasoning, affecting the output’s reliability and precision. Moreover, CoT can be challenging to implement on smaller language models with constrained knowledge and reasoning depth, which raises concerns about efficiency. In response to these issues, we propose the Ladder-of-Thought (LoT), a novel framework using knowledge as steps to elevate stance detection. LoT implements a triple-phase Progressive Optimization Framework: 1) External Knowledge Injection, which aims to enrich the model’s intrinsic knowledge base; 2) Intermediate Knowledge Generation, allowing the model to generate more accurate and dependable intermediate knowledge to enhance the downstream prediction; and 3) Downstream Fine-tuning & Prediction, which aims to improve the model’s prediction accuracy. This sequential approach symbolizes ascending a ladder, with each phase representing a progressive step towards achieving optimal reasoning and prediction performance. Our empirical results have demonstrated that LoT achieves state-of-the-art results in zero-shot/few-shot and in-target stance detection, marking a 16% improvement over ChatGPT and a 10% enhancement compared to ChatGPT with CoT on stance detection task.

Janosch Haber, Bertie Vidgen, Matthew Chapman, Vibhor Agarwal, Roy Ka-Wei Lee, Yong Keong Yap, Paul Röttger. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Stance detection determines whether the author of a piece of text is in favor of, against, or neutral towards a specified target, and can be used to gain valuable insights into social media.The ubiquitous indirect referral of targets makes this task challenging, as it requires computational solutions to model semantic features and infer the corresponding implications from a literal statement.Moreover, the limited amount of available training data leads to subpar performance in out-of-domain and cross-target scenarios, as data-driven approaches are prone to rely on superficial and domain-specific features.In this work, we decompose the stance detection task from a linguistic perspective, and investigate key components and inference paths in this task.The stance triangle is a generic linguistic framework previously proposed to describe the fundamental ways people express their stance.We further expand it by characterizing the relationship between explicit and implicit objects.We then use the framework to extend one single training corpus with additional annotation.Experimental results show that strategically-enriched data can significantly improve the performance on out-of-domain and cross-target evaluation.