Large Language Models (LLMs) can perform sentiment analysis via natural language instructions, yet their predictions are highly sensitive to prompt phrasing.Prior work has shown that sentiment is encoded linearly in LLM representations, but the model's ability to utilize this information remains surprisingly fragile to prompt variations.To understand this behavior, we leverage Sparse Autoencoders (SAEs) to extract interpretable features from LLM activations and apply circuit-level analysis to uncover causal mechanisms underlying sentiment prediction.We identify a sentiment analysis circuit and find that prompt sensitivity may stem from task activation failure.Based on this insight, we propose a simple inference-time intervention method that amplifies circuit features to compensate for insufficient activation.Experiments across diverse datasets, templates, and languages show consistent improvements, offering an interpretable and training-free alternative to manual prompt engineering.

Model editing aims at selectively updating a small subset of a neural model's parameters with an interpretable strategy to achieve desired modifications. It can significantly reduce computational costs to adapt to large language models(LLMs). Given its ability to precisely target critical components within LLMs, model editing shows great potential for efficient fine-tuning applications. In this work, we investigate model editing to serve as an efficient method for adapting LLMs to solve aspect-based sentiment classification. Through causal interventions, we trace and determine which neuron hidden states are essential for the model’s predictions. By performing interventions and restorations on each component of an LLM, we identify the importance of these components for aspect-based sentiment classification. Our findings reveal that a distinct set of mid-layer representations is essential for detecting the sentiment polarity of given aspect words. Leveraging these insights, we develop a model editing approach that focuses exclusively on these critical parts of the LLM, leading to a more efficient method for adapting LLMs. Our in and out of domain experiments demonstrate that this approach achieves competitive results compared to the currently strongest methods with significantly fewer trainable parameters, highlighting a more efficient and interpretable fine-tuning strategy.

Multimodal sentiment analysis for fashion-related social media is essential for understanding how consumers appraise fashion products across platforms like Instagram and Twitter, where both textual and visual elements contribute to sentiment expression. However, a notable challenge in this task is the modality gap, where the different information density between text and images hinders effective sentiment analysis. In this paper, we propose a novel multimodal framework that addresses this challenge by introducing pseudo data generated by a two-stage framework. We further utilize a multimodal fusion approach that efficiently integrates the information from various modalities for sentiment classification of fashion posts. Experiments conducted on a comprehensive dataset demonstrate that our framework significantly outperforms existing unimodal and multimodal baselines, highlighting its effectiveness in bridging the modality gap for more accurate sentiment classification in fashion-related social media posts.

Online tool condition monitoring (TCM) is a pivotal capability in many manufacturing applications including ultrasonic metal welding (UMW). Effective and efficient TCM can facilitate predictive maintenance, improve product quality, and enhance productivity. Existing online TCM systems based on conventional machine learning models often require a large amount of labeled data, the collection of which is cost-prohibitive, time-consuming, and labor-intensive. Such models fail to satisfy the requirements of cost-effectiveness and agility posed by modern, reconfigurable UMW systems. As such, data-efficient TCM methods with excellent generalization ability are of vital importance. To this end, we develop a novel similarity-based meta-representation learning (SMRL) method for domain generalization. SMRL effectively learns high-level meta-knowledge that is shared among different welding scenarios or domains. Therefore, the model trained in source domains can be generalized to other domains without access to labeled data in the training phase. Case studies are performed using four welding scenarios with varied welding materials and welding parameters. It is demonstrated that the proposed method is superior to the baseline methods, including neural network, hierarchical neural network, model-agnostic meta-learning (MAML), and hierarchical MAML. Compared with the baseline methods, SMRL offers an average improvement of 15.44%–31.62% in TCM accuracy. These results show that SMRL is readily applicable to industrial applications to enable cost-effective and data-efficient TCM.

Teaching small-scale language models to perform math reasoning is a valuable yet challenging task.Besides obtaining labeled data from human experts, one of the most common ways to collect high-quality data is by sampling from a larger and more powerful language model.Although previous works have demonstrated the effectiveness of this method, such a knowledge distillation paradigm can be costly and unstable, especially considering that many large language models, such as GPT-4 (Ope-nAI, 2023), are closed-source, proprietary, and their behaviors are unpredictable.In this work, to avoid relying on outputs from large models, we demonstrate that the reasoning abilities of small-scale language models can be enhanced through self-training, which involves training models with their own outputs.We also show that the conventional self-training can be further augmented by an alignment algorithm called Direct Preference Optimization (DPO) (Rafailov et al., 2023).We empirically found that models trained with the DPO objective are capable of making better generations that largely benefit multi-turn self-training.The experimental results show our models outperform the existing models with comparable sizes on the GSM8K benchmark with minimal resource requirements.1

Metaphors are commonly found in advertising and internet memes.However, the free form of internet memes often leads to a lack of high-quality textual data.Metaphor detection demands a deep interpretation of both textual and visual elements, requiring extensive common-sense knowledge, which poses a challenge to language models.To address these challenges, we propose a compact framework called C4MMD, which utilizes a Chain-of-Thought(CoT) method for Multi-modal Metaphor Detection.Specifically, our approach designs a three-step process inspired by CoT that extracts and integrates knowledge from Multi-modal Large Language Models(MLLMs) into smaller ones.We also developed a modality fusion architecture to transform knowledge from large models into metaphor features, supplemented by auxiliary tasks to improve model performance.Experimental results on the MET-MEME dataset demonstrate that our method not only effectively enhances the metaphor detection capabilities of small models but also outperforms existing models.To our knowledge, this is the first systematic study leveraging MLLMs in metaphor detection tasks.The code for our method is publicly available at https: //github.com/xyz189411yt/C4MMD.

Employing pre-trained generation models for cross-domain aspect-based sentiment classification has recently led to large improvements. However, they ignore the importance of syntactic structures, which have shown appealing effectiveness in classification based models. Different from previous studies, efficiently encoding the syntactic structure in generation model is challenging because such models are pretrained on natural language, and modeling structured data may lead to catastrophic forgetting of distributional knowledge. In this study, we propose a novel structure-aware generation model to tackle this challenge. In particular, a prompt-driven strategy is designed to bridge the gap between different domains, by capturing implicit syntactic information from the input and output sides. Furthermore, the syntactic structure is explicitly encoded into the structure-aware generation model, which can effectively learn domain-irrelevant features based on syntactic pivot features. Empirical results demonstrate the effectiveness of the proposed structure-aware generation model over several strong baselines. The results also indicate the proposed model is capable of leveraging the input syntactic structure into the generation model.