Existing studies on preference optimization (PO) have been focused on constructing pairwise preference data following simple heuristics, such as maximizing the margin between chosen and rejected responses based on human (or AI) ratings. In this work, we develop a novel PO framework that provides theoretical guidance to effectively sample rejected responses. To achieve this, we formulate PO as minimizing the negative log-likelihood (NLL) of a probability model and propose a sampling-based solution to estimate its normalization constant via contrastive divergence. We show that these estimative samples can act as rejected responses in PO. Leveraging the connection established between PO and NLL estimation, we propose a novel PO algorithm, called Monte-Carlo-based PO (MC-PO), that applies a MC kernel to sample *hard negatives* w.r.t.~the log-likelihood of the target policy. Intuitively, these hard negatives represent the rejected samples that are most difficult for the current policy to differentiate. We show that MC-PO outperforms existing SOTA baselines on popular alignment benchmarks.

Numerical reasoning over documents, which demands both contextual understanding and logical inference, is challenging for low-capacity local models deployed on computation-constrained devices. Although such complex reasoning queries could be routed to powerful remote models like GPT-4, exposing local data raises significant data leakage concerns. Existing mitigation methods generate problem descriptions or examples for remote assistance. However, the inherent complexity of numerical reasoning hinders the local model from generating logically equivalent queries and accurately inferring answers with remote guidance. In this paper, we present a model collaboration framework with two key innovations: (1) a context-aware synthesis strategy that shifts the query topics while preserving reasoning patterns; and (2) a tool-based answer reconstruction approach that reuses the remote-generated plug-and-play solution with code snippets. Experimental results demonstrate that our method achieves better reasoning accuracy than solely using local models while providing stronger data protection than fully relying on remote models. Furthermore, our method improves accuracy by 16.2% - 43.6% while reducing data leakage by 2.3% - 44.6% compared to existing data protection approaches.

Kiran Ramnath, Kang Zhou, Sheng Guan, Soumya Smruti Mishra, Xuan Qi, Zhengyuan Shen, Shuai Wang, Sangmin Woo, Sullam Jeoung, Yawei Wang, Haozhu Wang, Han Ding, Yuzhe Lu, Zhichao Xu, Yun Zhou, Balasubramaniam Srinivasan, Qiaojing Yan, Yueyan Chen, Haibo Ding, Panpan Xu, Lin Lee Cheong. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

ABSTRACT In recent years, intelligent control methods based on surface electromyographic signals (sEMG) have received extensive attention in the fields of bionic prosthetics and human‐computer interaction. Compared with high‐density sampling, sparse sampling sEMG has significant advantages due to its convenient collection, strong non‐intrusiveness and high equipment flexibility. However, it has deficiencies in spatial resolution and muscle activity information acquisition, which can easily lead to a decline in the recognition accuracy of complex gestures. In view of the limitations of sparse sEMG in feature extraction and channel information utilization, this paper proposes a hybrid neural network SERes‐L based on the attention mechanism. This model introduces the squeeze‐and‐excitation (SE) attention module in the residual unit to construct a residual attention structure to enhance the channel modeling of key information. At the same time, it combines the long short‐term memory network (LSTM) to capture the long‐term temporal dependencies in gesture actions, achieving a deep integration of spatial and temporal features. Experiments on the self‐built dataset show that SERes‐L achieves an average recognition accuracy rate of 93.90%. To verify its generalization ability, further tests were conducted on three public datasets, NinaPro DB1, DB4, and DB5. The average classification accuracy rates of the model reached 90.28%, 84.72%, and 91.87% respectively, significantly outperforming many existing mainstream methods. The above results indicate that the proposed SERes‐L architecture can effectively alleviate the challenges of insufficient utilization of channel information and inadequate feature extraction in sparse sEMG, demonstrating excellent generalization performance and practical application potential.

ABSTRACT The ladle is a critical piece of equipment for transporting high‐temperature molten steel in the steelmaking process, and its operational performance directly impacts the quality of the final product, energy efficiency, and overall production costs. With the advancement of continuous casting and external refining technologies, the stability of ladles under high‐temperature and high‐intensity service conditions faces increasingly stringent demands, particularly as the issue of thermal stress damage to the refractory lining becomes more pronounced. Based on typical steelmaking conditions, this study establishes a multi‐stage service cycle model for a 350‐ton ladle. Utilizing a parameterized finite element approach, we conduct a coupled transient thermo‐mechanical simulation to analyze the temperature and stress fields, specifically investigating the synergistic effects of thermal expansion, temperature gradient, and ferrostatic pressure. The results demonstrate that thermal stress is the predominant factor responsible for lining damage. Notably, the thermal expansion behavior of the refractory materials exerts a significant influence on the distribution and magnitude of thermal stress within the ladle structure. In contrast, mechanical loads such as the static pressure from the molten steel contribute minimally to the overall stress state. Furthermore, the study reveals that the stress on the ladle shell significantly reduces after the refractory lining fails and expansion pressure diminishes, quantitatively highlighting the critical role of interfacial expansion constraints. This research provides a comprehensive theoretical foundation and valuable engineering insights for the optimized design and longevity enhancement of ladle lining structures.

Large language models (LLMs) have shown remarkable generalization capability with exceptional performance in various language modeling tasks. However, they still exhibit inherent limitations in precisely capturing and returning grounded knowledge. While existing work has explored utilizing knowledge graphs (KGs) to enhance language modeling via joint training and customized model architectures, applying this to LLMs is problematic owing to their large number of parameters and high computational cost. Therefore, how to enhance pre-trained LLMs using grounded knowledge, e.g., retrieval-augmented generation, remains an open question. In this work, we propose Graph Neural Prompting (GNP), a novel plug-and-play method to assist pre-trained LLMs in learning beneficial knowledge from KGs. GNP encompasses various designs, including a standard graph neural network encoder, a cross-modality pooling module, a domain projector, and a self-supervised link prediction objective. Extensive experiments on multiple datasets demonstrate the superiority of GNP on both commonsense and biomedical reasoning tasks across different LLM sizes and settings. Code is available at https://github.com/meettyj/GNP.

IMPORTANCE: Reinforcement learning (RL) represents a pivotal avenue within natural language processing (NLP), offering a potent mechanism for acquiring optimal strategies in task completion. This literature review studies various NLP applications where RL has demonstrated efficacy, with notable applications in healthcare settings. OBJECTIVES: To systematically explore the applications of RL in NLP, focusing on its effectiveness in acquiring optimal strategies, particularly in healthcare settings, and provide a comprehensive understanding of RL's potential in NLP tasks. MATERIALS AND METHODS: Adhering to the PRISMA guidelines, an exhaustive literature review was conducted to identify instances where RL has exhibited success in NLP applications, encompassing dialogue systems, machine translation, question-answering, text summarization, and information extraction. Our methodological approach involves closely examining the technical aspects of RL methodologies employed in these applications, analyzing algorithms, states, rewards, actions, datasets, and encoder-decoder architectures. RESULTS: The review of 93 papers yields insights into RL algorithms, prevalent techniques, emergent trends, and the fusion of RL methods in NLP healthcare applications. It clarifies the strategic approaches employed, datasets utilized, and the dynamic terrain of RL-NLP systems, thereby offering a roadmap for research and development in RL and machine learning techniques in healthcare. The review also addresses ethical concerns to ensure equity, transparency, and accountability in the evolution and application of RL-based NLP technologies, particularly within sensitive domains such as healthcare. DISCUSSION: The findings underscore the promising role of RL in advancing NLP applications, particularly in healthcare, where its potential to optimize decision-making and enhance patient outcomes is significant. However, the ethical challenges and technical complexities associated with RL demand careful consideration and ongoing research to ensure responsible and effective implementation. CONCLUSIONS: By systematically exploring RL's applications in NLP and providing insights into technical analysis, ethical implications, and potential advancements, this review contributes to a deeper understanding of RL's role for language processing.

OBJECTIVE: Occupational injuries (OIs) cause an immense burden on the US population. Prediction models help focus resources on those at greatest risk of a delayed return to work (RTW). RTW depends on factors that develop over time; however, existing methods only utilize information collected at the time of injury. We investigate the performance benefits of dynamically estimating RTW, using longitudinal observations of diagnoses and treatments collected beyond the time of initial injury. MATERIALS AND METHODS: We characterize the difference in predictive performance between an approach that uses information collected at the time of initial injury (baseline model) and a proposed approach that uses longitudinal information collected over the course of the patient's recovery period (proposed model). To control the comparison, both models use the same deep learning architecture and differ only in the information used. We utilize a large longitudinal observation dataset of OI claims and compare the performance of the two approaches in terms of daily prediction of future work state (working vs not working). The performance of these two approaches was assessed in terms of the area under the receiver operator characteristic curve (AUROC) and expected calibration error (ECE). RESULTS: After subsampling and applying inclusion criteria, our final dataset covered 294 103 OIs, which were split evenly between train, development, and test datasets (1/3, 1/3, 1/3). In terms of discriminative performance on the test dataset, the proposed model had an AUROC of 0.728 (90% confidence interval: 0.723, 0.734) versus the baseline's 0.591 (0.585, 0.598). The proposed model had an ECE of 0.004 (0.003, 0.005) versus the baseline's 0.016 (0.009, 0.018). CONCLUSION: The longitudinal approach outperforms current practice and shows potential for leveraging observational data to dynamically update predictions of RTW in the setting of OI. This approach may enable physicians and workers' compensation programs to manage large populations of injured workers more effectively.

In many settings, it is important that a model be capable of providing reasons for its predictions (ıe, the model must be interpretable). However, the model's reasoning may not conform with well-established knowledge. In such cases, while interpretable, the model lacks credibility. In this work, we formally define credibility in the linear setting and focus on techniques for learning models that are both accurate and credible. In particular, we propose a regularization penalty, expert yielded estimates (EYE), that incorporates expert knowledge about well-known relationships among covariates and the outcome of interest. We give both theoretical and empirical results comparing our proposed method to several other regularization techniques. Across a range of settings, experiments on both synthetic and real data show that models learned using the EYE penalty are significantly more credible than those learned using other penalties. Applied to two large-scale patient risk stratification task, our proposed technique results in a model whose top features overlap significantly with known clinical risk factors, while still achieving good predictive performance.