Xinglin Wang, Peiwen Yuan, Shaoxiong Feng, Yiwei Li, Boyuan Pan, Heda Wang, Yao Hu, Kan Li. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

ABSTRACT This paper introduces an enhanced BERT‐DPCNN model for the task of Chinese news text classification. The model addresses the common challenge of balancing accuracy and computational efficiency in existing models, especially when dealing with large‐scale, high‐dimensional text data. To tackle this issue, the paper proposes an improved BERT‐DPCNN model that integrates BERT's pre‐trained language model with DPCNN's efficient convolutional structure to capture deep semantic information and key features from the text. Additionally, the paper incorporates the zebra optimization algorithm (ZOA) to dynamically optimize the model's hyperparameters, overcoming the limitations of manual tuning in traditional models. By automatically optimizing hyperparameters such as batch size, learning rate, and the number of filters through ZOA, the model's classification performance is significantly enhanced. Experimental results demonstrate that the improved ZOA‐BERT‐DPCNN model outperforms traditional methods on the THUCNEWS Chinese news dataset, not only verifying its effectiveness in news text classification tasks but also showcasing its potential to enhance classification performance.

Recently, audiences have increasingly watched diverse dramas and movies on streaming platforms, prompting platform administrators to enhance their understanding of video semantics. In particular, capturing and interpreting social relations among characters is critical for content-driven intelligent services and enhancing user experience. However, most existing research has solely approached social relations recognition as a classification problem, without justifiable interpretation of prediction results within the video context. To address this issue, we study cognitive science research and the Chain-of-Thought (CoT) strategy. Based on these foundations, we propose SaMo-CoT, an approach that leverages large language models (LLMs) in step-by-step reasoning. This approach simulates human cognitive processes by combining social scenarios in videos and incorporating empirical social interaction knowledge. In this way, we enable verbal rationales for determining social relations in video understanding. Furthermore, we present an innovative doubly-right social relation recognition framework that predicts both correct social relation labels and correct scenario rationales. Specifically, we translate verbal CoT into multimodal CoT by leveraging scenario-aware prompts and contrastive learning. Extensive experiments demonstrate significant improvements in classification accuracy and interpretability compared to traditional approaches.

Large Language Models (LLMs) have performed well on various reasoning tasks, but their inaccessibility and numerous parameters hinder wide application in practice. One promising way is distilling the reasoning ability from LLMs to small models by the generated chain-of-thought reasoning paths. In some cases, however, LLMs may produce incorrect reasoning chains, especially when facing complex mathematical problems. Previous studies only transfer knowledge from positive samples and drop the synthesized data with wrong answers. In this work, we illustrate the merit of negative data and propose a model specialization framework to distill LLMs with negative samples besides positive ones. The framework consists of three progressive steps, covering from training to inference stages, to absorb knowledge from negative data. We conduct extensive experiments across arithmetic reasoning tasks to demonstrate the role of negative data in distillation from LLM.

Xinglin Wang, Yiwei Li, Shaoxiong Feng, Peiwen Yuan, Boyuan Pan, Heda Wang, Yao Hu, Kan Li. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Peiwen Yuan, Shaoxiong Feng, Yiwei Li, Xinglin Wang, Boyuan Pan, Heda Wang, Yao Hu, Kan Li. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Peiwen Yuan, Xinglin Wang, Shaoxiong Feng, Boyuan Pan, Yiwei Li, Heda Wang, Xupeng Miao, Kan Li. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Peiwen Yuan, Shaoxiong Feng, Yiwei Li, Xinglin Wang, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Heda Wang, Yao Hu, Kan Li. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

Self-consistency (SC) has been a widely used decoding strategy for chain-of-thought reasoning. Despite bringing significant performance improvements across a variety of multi-step reasoning tasks, it is a high-cost method that requires multiple sampling with the preset size. In this paper, we propose a simple and scalable sampling process, Early-Stopping Self-Consistency (ESC), to greatly reduce the cost of SC without sacrificing performance. On this basis, one control scheme for ESC is further derivated to dynamically choose the performance-cost balance for different tasks and models. To demonstrate ESC's effectiveness, we conducted extensive experiments on three popular categories of reasoning tasks: arithmetic, commonsense and symbolic reasoning over language models with varying scales. The empirical results show that ESC reduces the average number of sampling of chain-of-thought reasoning by a significant margin on six benchmarks, including MATH (-33.8%), GSM8K (-80.1%), StrategyQA (-76.8%), CommonsenseQA (-78.5%), Coin Flip (-84.2%) and Last Letters (-67.4%), while attaining comparable performances.

Instruction data is crucial for improving the capability of Large Language Models (LLMs) to align with human-level performance. Recent research LIMA demonstrates that alignment is essentially a process where the model adapts instructions' interaction style or format to solve various tasks, leveraging pre-trained knowledge and skills. Therefore, for instructional data, the most important aspect is the task it represents, rather than the specific semantics and knowledge information. The latent representations of instructions play roles for some instruction-related tasks like data selection and demonstrations retrieval. However, they are always derived from text embeddings, encompass overall semantic information that influences the representation of task categories. In this work, we introduce a new concept, instruction embedding, and construct Instruction Embedding Benchmark (IEB) for its training and evaluation. Then, we propose a baseline Prompt-based Instruction Embedding (PIE) method to make the representations more attention on tasks. The evaluation of PIE, alongside other embedding methods on IEB with two designed tasks, demonstrates its superior performance in accurately identifying task categories. Moreover, the application of instruction embeddings in four downstream tasks showcases its effectiveness and suitability for instruction-related tasks.