Column Generation (CG) is an effective and iterative algorithm to solve large-scale linear programs (LP). During each CG iteration, new columns are added to improve the solution of the LP. Typically, CG greedily selects one column with the most negative reduced cost, which can be improved by adding more columns at once. However, selecting all columns with negative reduced costs would lead to the addition of redundant columns that do not improve the objective value. Therefore, selecting the appropriate columns to add is still an open problem and previous machine-learning-based approaches for CG only add a constant quantity of columns per iteration due to the state-space explosion problem. To address this, we propose Fast Family Column Generation (FFCG) — a novel reinforcement-learning-based CG that selects a variable number of columns as needed in an iteration. Specifically, we formulate the column selection problem in CG as an MDP and design a reward metric that balances both the convergence speed and the number of redundant columns. In our experiments, FFCG converges faster on the common benchmarks and reduces the number of CG iterations by 77.1% for Cutting Stock Problem (CSP) and 84.8% for Vehicle Routing Problem with Time Windows (VRPTW), and a 71.4% reduction in computing time for CSP and 84.0% for VRPTW on average compared to several state-of-the-art baselines.

Branch-and-bound methods are pivotal in solving Mixed Integer Linear Programming (MILP), where the challenge of node selection arises, necessitating the prioritization of different regions of the space for subsequent exploration. While machine learning techniques have been proposed to address this, two crucial problems concerning \textbf{(P1)} how to sufficiently extract features from the branch-and-bound tree, and \textbf{(P2)} how to assess the node quality comprehensively based on the features remain open. To tackle these challenges, we propose to tackle the node selection problem employing a novel Tripartite graph representation and Reinforcement learning with a Graph Neural Network model (TRGNN). The tripartite graph is theoretically proved to encompass sufficient information for tree representation in information theory. We learn node selection via reinforcement learning for learning delay rewards and give more comprehensive node metrics. Experiments show that TRGNN significantly improves the efficiency of solving MILPs compared to human-designed and learning-based node selection methods on both synthetic and large-scale real-world MILPs. Moreover, experiments demonstrate that TRGNN well generalizes to MILPs that are significantly larger than those seen during training.

Many real-world applications, such as logistics, routing, scheduling, and production planning, involve dynamic systems that require continuous updates to solutions for new Mixed Integer Linear Programming (MILP) problems. These systems often require rapid updates to their solutions to accommodate slight modifications in constraints or objectives introduced by evolving conditions. While reoptimization techniques have been explored for Linear Programming (LP) and certain specific MILP problems, their effectiveness in addressing general MILP is limited. In this work, we propose a two-stage reoptimization framework for efficiently identifying high-quality feasible solutions. Specifically, we first utilize the historical solving process information to predict a high confidence solution space for modified MILPs, which is likely to contain high-quality solutions. Building on the prediction results, we fix a part of variables within the predicted intervals and apply the Thompson Sampling algorithm to determine which variables to fix. This is done by updating the Beta distributions based on the solutions obtained from the solver. Extensive experiments across nine reoptimization datasets show that our VP-OR outperforms the state-of-the-art methods, achieving higher-quality solutions under strict time limits.

Optimization problems are ubiquitous across various domains, such as resource scheduling, production planning, and sales management. Traditionally, they are modeled manually, leading to inefficiencies due to difficulties in communication and collaboration between modeling and domain experts. The emergence of Large Language Models (LLMs) has made automated modeling possible. However, real-world applications are often large-scale and have numerous variables and constraints, limiting the applicability of existing methods. To address this, we propose PaMOP, a novel modeling framework based on LLMs, to model optimization problems automatically, given only natural language descriptions. Specifically, we extract and partition the problems using a tree structure, guiding the LLMs to model each set of constraints with self-augmented prompts, thus reducing the demands on the LLM's capabilities of large contents. The mathematical model is then iteratively corrected and validated through our correction procedures. The experiments demonstrate that our method improves performance on the common benchmark dataset NLP4LP, achieving an accuracy of 62.3% and a code executability rate of 86.8% when tested on GPT-4. Additionally, we demonstrate the effectiveness of our PaMOP in handling large real-world problems.

Non-stationary multi-armed bandits (nsMAB) enable agents to adapt to changing environments by incorporating mechanisms to detect and respond to shifts in reward distributions, making them well-suited for dynamic settings. However, existing approaches typically assume that reward feedback is available at every round—an assumption that overlooks many real-world scenarios where feedback is limited. In this paper, we take a significant step forward by introducing a new model of *constrained feedback in non-stationary multi-armed bandits* (ConFee-nsMAB), where the availability of reward feedback is restricted. We propose the first prior-free algorithm—that is, one that does not require prior knowledge of the degree of non-stationarity—that achieves near-optimal dynamic regret in this setting. Specifically, our algorithm attains a dynamic regret of $\tilde {\mathcal{O}}({K^{1/3} V_T^{1/3} T }/{ B^{1/3}})$, where $T$ is the number of rounds, $K$ is the number of arms, $B$ is the query budget, and $V_T$ is the variation budget capturing the degree of non-stationarity.

In response to the escalating prevalence of inference attacks on network traffic, traffic shaping emerges as a highly effective strategy to curb the leakage of user privacy information. Although many traffic shaping mechanisms have been proposed, their privacy-preserving performance has mostly been evaluated empirically or by focusing solely on inter-event traffic shaping methods, thereby neglecting intra-event information. In this work, we develop a general framework considering intra-event characteristics for quantifying privacy leakage. The proposed evaluation framework provides quantitative assessments of privacy risks and can guide the design and improvement of traffic-shaping mechanisms. Our theoretical results identify the limitation of existing approaches, namely, the absence of a flexible trade-off between privacy-preserving effectiveness and cost. This trade-off is critical for IoT applications, as devices are highly heterogeneous in terms of resources (i.e., bandwidth) and need to meet service-level objectives (e.g., latency). To achieve this design goal, we model the trade-off problem as multi-armed bandits and propose an online traffic-shaping algorithm named DPShaping. DPShaping has a proven privacy-preserving guarantee and supports flexible trade-offs between effectiveness and communication cost. We compare our approach with three representative algorithms. Experimental results show that compared with state-of-the-art schemes, DPShaping achieves lower attack accuracy (only 16.8%) and reduces bandwidth and delay.

Best Arm Identification (BAI) is a general online pure exploration framework to identify optimal decisions among candidates via sequential interactions. We pioneer the Optimal Arms identification with Knapsacks (OAK) problem, which extends the BAI setting to model the resource consumption. We present a novel OAK algorithm and prove the upper bound of our algorithm by exploring the relationship between selecting optimal actions and the structure of the feasible region. Our analysis introduces a new complexity measure, which builds a bridge between the OAK setting and bandits with knapsacks problem. We establish the instance-dependent lower bound for the OAK problem based on the new complexity measure. Our results show that the proposed algorithm achieves a near-optimal probability bound for the OAK problem. In addition, we demonstrate that our algorithm recovers or improves the state-of-the-art upper bounds for several special cases, including the simple OAK setting and some classical pure exploration problems.

Many efforts have been devoted to online pricing mechanism design for different settings. In this work, we consider a common but challenging setting where the buyers have private time-sensitive valuations and the seller has limited supply. The seller offers a take-it-or-leave-it posted price for each arriving buyer and aims to maximize the expected total revenue. The unknown distribution of time-sensitive valuations and limited supply significantly increase the difficulty of searching the optimal dynamic posted prices. Given B identical items to sell, when the time-dependent valuations can be estimated with a factor of α, we prove Ω(log(1/α)) lower bound with respect to the optimal fixed distribution over prices and design an algorithm achieving tight O(log(1/α)) competitive ratio. When the seller has no information about the future trends of buyers’ valuations, we prove Ω(log B) lower bound and show that there is an algorithm with tight O(log B) competitive ratio by modeling the problem as adversarial bandits with knapsacks optimization. Extensive simulation studies show that our algorithm outperforms previous mechanisms in various settings.

Wireless virtual reality (VR) systems are able to provide users with immersive experiences, and require low latency and high data rate. To meet these conflicting requirements with limited radio resources, edge intelligence is a promising architecture. It exploits the edge server co-located at the base station to predict the field of view (FoV) of the next VR video segment, pre-render the three-dimensional video within the predicted FoV, and transmit it to the user in advance. Since the prediction is not error-free, the predicted FoV may not cover the actual FoV requested by the user, and hence may result in video quality loss. To address this issue, we first formulate a constrained partially observable Markov decision process problem to optimize the redundant range of the FoV according to the head motion prediction and the redundant range for the previous video segment. Then, we develop a constrained deep reinforcement learning algorithm to minimize the video quality loss ratio subject to the latency constraint. Simulation results show that the proposed algorithm outperforms the existing methods in terms of video quality loss ratio (from 6.9% to 4.9%) and latency (from 0.72 s to 0.63 s).