Large Language Models (LLMs) have shown remarkable success on a wide range of math and reasoning benchmarks. However, we observe that they often struggle when faced with unreasonable math problems. Instead of recognizing these issues, models frequently proceed as if the problem is well-posed, producing incorrect answers or falling into overthinking and verbose self-correction. To systematically investigate this overlooked vulnerability, we propose the Unreasonable Math Problems (UMP) benchmark, designed to evaluate LLMs' ability to detect and respond to unreasonable math problem statements. Based on extensive experiments covering 19 LLMs, we find that even state-of-the-art general models like GPT-4o struggle on UMP. While reasoning models such as DeepSeek-R1 demonstrate a higher sensitivity to unreasonable inputs, this often comes at the cost of generating overly long and meaningless responses that fail to converge. We further find that prompting and fine-tuning enhance the detection of unreasonable inputs, with minor and acceptable trade-offs, making them practical solutions in this challenging setting.

Large reasoning models (LRMs) have demonstrated remarkable proficiency in tackling complex tasks through step-by-step thinking.However, this lengthy reasoning process incurs substantial computational and latency overheads, hindering the practical deployment of LRMs.This work presents a new approach to mitigating overthinking in LRMs via black-box persuasive prompting.By treating LRMs as black-box communicators, we investigate how to persuade them to generate concise responses without compromising accuracy.We introduce WHISPER, an iterative refinement framework that generates high-quality persuasive prompts from diverse perspectives.Experiments across multiple benchmarks demonstrate that WHIS-PER consistently reduces token usage while preserving performance.Notably, WHISPER achieves a 3× reduction in average response length on simple GSM8K questions for the Qwen3 series and delivers an average ∼40% token reduction overall.For closed-source APIs, WHISPER reduces token usage on MATH-500 by 46% for Claude-3.7 and 50% for Gemini-2.5.Further analysis reveals the broad applicability of WHISPER across data domains, model scales, and families, underscoring the potential of black-box persuasive prompting as a practical strategy for enhancing LRM efficiency. 1 * Work done during the author's internship at Sea AI Lab.† Corresponding authors. 1 We release our code and top-performing prompt candidates at https://github.com/hemingkx/Whisper.Question: John arm wrestles 20 people.He beats 80%.How many people did he lose to?

To facilitate robust and trustworthy deployment of large language models (LLMs), it is essential to quantify the reliability of their generations through uncertainty estimation.While recent efforts have made significant advancements by leveraging the internal logic and linguistic features of LLMs to estimate uncertainty scores, our empirical analysis highlights the pitfalls of these methods to strike a harmonized estimation between indication, balance, and calibration, which hinders their broader capability for accurate uncertainty estimation.To address this challenge, we propose CUE (Corrector for Uncertainty Estimation): A straightforward yet effective method that employs a lightweight model trained on data aligned with the target LLM's performance to adjust uncertainty scores.Comprehensive experiments across diverse models and tasks demonstrate its effectiveness, which achieves consistent improvements of up to 60% over existing methods.Resources are available at https://github. com/O-L1RU1/Corrector4UE.

With rapid advancement and increasing accessibility of LLMs, fine-tuning aligned models has become a critical step for adapting them to real-world applications, which makes the safety of this fine-tuning process more important than ever.However, recent studies have highlighted a critical challenge: even when fine-tuning with benign datasets, the safety alignment of aligned LLMs can be compromised, making them more susceptible to malicious instructions.In this paper, we show that fine-tuning datasets often contain safetydegrading samples that are not easily identifiable on the surface.These samples can easily degrade the safety alignment of LLMs during fine-tuning.To address this issue, we propose LARF, a Layer-Aware Representation Filtering method.This method identifies safety-sensitive layers within the LLM and leverages data representations to detect safetydegrading data samples in the fine-tuning dataset.Experimental results demonstrate that LARF can efficiently and effectively identify safety-degrading data.After removing such data, the safety alignment degradation caused by fine-tuning is mitigated.Please see our code at https://github.com/LLLeoLi/LARF.

Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, Zhifang Sui. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.