Neural solvers have demonstrated remarkable success in combinatorial optimization, often surpassing traditional heuristics in speed, solution quality, and generalization. However, their efficacy deteriorates significantly when confronted with complex constraints that cannot be effectively managed through simple masking mechanisms. To address this limitation, we introduce Universal Constrained Preference Optimization (UCPO), a novel plug-and-play framework that seamlessly integrates preference learning into existing neural solvers via a specially designed loss function, without requiring architectural modifications. UCPO embeds constraint satisfaction directly into a preference-based objective, eliminating the need for meticulous hyperparameter tuning. Leveraging a lightweight warm-start fine-tuning protocol, UCPO enables pre-trained models to consistently produce near-optimal, feasible solutions on challenging constraint-laden tasks, achieving exceptional performance with as little as 1% of the original training budget.

Neural Combinatorial Optimization (NCO) has emerged as a promising approach for NP-hard problems. However, prevailing RL-based methods suffer from low sample efficiency due to sparse rewards and underused solutions. We propose Best-anchored and Objective-guided Preference Optimization (BOPO), a training paradigm that leverages solution preferences via objective values. It introduces: (1) a best-anchored preference pair construction for better explore and exploit solutions, and (2) an objective-guided pairwise loss function that adaptively scales gradients via objective differences, removing reliance on reward models or reference policies. Experiments on Job-shop Scheduling Problem (JSP), Traveling Salesman Problem (TSP), and Flexible Job-shop Scheduling Problem (FJSP) show BOPO outperforms state-of-the-art neural methods, reducing optimality gaps impressively with efficient inference. BOPO is architecture-agnostic, enabling seamless integration with existing NCO models, and establishes preference optimization as a principled framework for combinatorial optimization.

Large language models (LLMs) have recently attracted significant attention and permeated diverse fields and disciplines. This paper aims to investigate the efficacy of LLMs in efficiently tackling combinatorial optimization problems and integrating them with traditional heuristic algorithms. Firstly, we describe the fundamental concepts and developmental history of LLMs, outlining the basic LLM framework involving the instance prompt, solution prompt, and algorithm prompt. Subsequently, we introduce a novel LLM implemented simulated annealing (SA) approach that enhances the basic LLM method. In the experiments, we present the average iterations required, convergence speed, and overall solution quality of LLM-based approaches in addressing the Traveling Salesman Problem (TSP). The results demonstrate that the integration of LLM with SA can enhance TSP-solving capabilities. Our research endeavors to empower non-specialists to effectively address combinatorial optimization problems.