Logical reasoning is essential for large language models (LLMs) to ensure accurate and coherent inference.However, LLMs struggle with reasoning order variations and fail to generalize across logically equivalent transformations.LLMs often rely on fixed sequential patterns rather than true logical understanding.To address this issue, we introduce an order-centric data augmentation framework based on commutativity in logical reasoning.We first randomly shuffle independent premises to introduce condition order augmentation.For reasoning steps, we construct a directed acyclic graph (DAG) to model dependencies between steps, which allows us to identify valid reorderings of steps while preserving logical correctness.By leveraging order-centric augmentations, models can develop a more flexible and generalized reasoning process.Finally, we conduct extensive experiments across multiple logical reasoning benchmarks, demonstrating that our method significantly enhances LLMs' reasoning performance and adaptability to diverse logical structures.We release our codes and augmented data in https://github.com/qianxiHe147/ Order-

Recent advancements in large language models (LLMs) have shifted the post-training paradigm from traditional instruction tuning and human preference alignment toward reinforcement learning (RL) focused on reasoning capabilities.However, numerous technical reports indicate that purely rule-based reward RL frequently results in poor-quality reasoning chains or inconsistencies between reasoning processes and final answers, particularly when the base model is of smaller scale.During the RL exploration process, models might employ lowquality reasoning chains due to the lack of knowledge, occasionally producing correct answers randomly and receiving rewards based on established rule-based judges.This constrains the potential for resource-limited organizations to conduct direct reinforcement learning training on smaller-scale models.We propose a novel confidence-based reward model tailored for enhancing STEM reasoning capabilities.Unlike conventional approaches, our model penalizes not only incorrect answers but also low-confidence correct responses, thereby promoting more robust and logically consistent reasoning.We validate the effectiveness of our approach through static evaluations, Bestof-N inference tests, and PPO-based RL training.Our method outperforms several stateof-the-art open-source reward models across diverse STEM benchmarks.We release our codes and model in https://github.com/ qianxiHe147/C2RM.

Large language models (LLMs) can understand human instructions, showing their potential for pragmatic applications beyond traditional NLP tasks. However, they still struggle with complex instructions, which can be either complex task descriptions that require multiple tasks and constraints, or complex input that contains long context, noise, heterogeneous information and multi-turn format. Due to these features, LLMs often ignore semantic constraints from task descriptions, generate incorrect formats, violate length or sample count constraints, and be unfaithful to the input text. Existing benchmarks are insufficient to assess LLMs’ ability to understand complex instructions, as they are close-ended and simple. To bridge this gap, we propose CELLO, a benchmark for evaluating LLMs' ability to follow complex instructions systematically. We design eight features for complex instructions and construct a comprehensive evaluation dataset from real-world scenarios. We also establish four criteria and develop corresponding metrics, as current ones are inadequate, biased or too strict and coarse-grained. We compare the performance of representative Chinese-oriented and English-oriented models in following complex instructions through extensive experiments. Resources of CELLO are publicly available at https://github.com/Abbey4799/CELLO.