Combinatorial optimization (CO) is fundamental to several realworld applications, from logistics and scheduling to hardware design and resource allocation.Deep reinforcement learning (RL) has recently shown significant benefits in solving CO problems, reducing reliance on domain expertise and improving computational efficiency.However, the absence of a unified benchmarking framework leads to inconsistent evaluations, limits reproducibility, and increases engineering overhead, raising barriers to adoption for new researchers.To address these challenges, we introduce RL4CO, a unified and extensive benchmark with in-depth library coverage of 27 CO problem environments and 23 state-of-the-art baselines.Built on efficient software libraries and best practices in implementation, RL4CO features modularized implementation and flexible configurations of diverse environments, policy architectures, RL algorithms, and utilities with extensive documentation.RL4CO helps researchers build on existing successes while exploring and developing their own designs, facilitating the entire research process by decoupling science from heavy engineering.We finally provide extensive benchmark studies to inspire new insights and future work.RL4CO has already attracted numerous researchers in the community and is open-sourced at https://github.com/ai4co/rl4co 1 .

Wind farms generate electricity from wind, offering a sustainable and eco-friendly power source.In wind farms, determining the positions of wind turbines is crucial for high energy production due to complex wake interactions.However, optimizing wind farm layouts for given wind conditions remains challenging.While existing methods try to solve the problem by training a surrogate model and employing optimization algorithms based on the model, these methods require a large number of simulations to obtain highly productive layouts, which is time-consuming in large-scale tasks.Furthermore, they mostly yield less diverse layouts, making the deployment of such layouts into real-world scenarios difficult.To address these challenges, we introduce a novel conditional generative modeling approach to find wind farm layouts that maximize energy production.Our method consists of four stages.First, we collect datasets that consist of layouts and their corresponding annual energy production (AEP).Then, we train a diffusion model conditioned on AEP and wind scenarios using the collected dataset.Specifically, we use a Graph Neural Network as the backbone for the denoising network to ensure permutation invariance.Next, we sample promising layouts from the trained diffusion model by conditioning with a high AEP and given wind scenario.We also introduce a local search algorithm, which enables us to adjust layouts that violate design constraints.Finally, we evaluate the generated layouts and augment the dataset.We repeat these processes iteratively to further optimize the layouts.Throughout these processes, we effectively generate highly productive and diverse wind farm layouts in a sample-efficient manner.Experimental results demonstrate that our approach outperforms state-of-the-art methods in maximizing energy production across various scenarios.

Complex urban road networks with high vehicle occupancy frequently face severe traffic congestion. Designing an effective strategy for managing multiple traffic lights plays a crucial role in managing congestion. However, most current traffic light management systems rely on human-crafted decisions, which may not adapt well to diverse traffic patterns. In this paper, we delve into two pivotal design components of the traffic light management system that can be dynamically adjusted to various traffic conditions: phase combination and phase time allocation. While numerous studies have sought an efficient strategy for managing traffic lights, most of these approaches consider a fixed traffic pattern and are limited to relatively small road networks. To overcome these limitations, we introduce a novel and practical framework to formulate the optimization of such design components using an offline meta black-box optimization. We then present a simple yet effective method to efficiently find a solution for the aforementioned problem. In our framework, we first collect an offline meta dataset consisting of pairs of design choices and corresponding congestion measures from various traffic patterns. After collecting the dataset, we employ the Attentive Neural Process (ANP) to predict the impact of the proposed design on congestion across various traffic patterns with well-calibrated uncertainty. Finally, Bayesian optimization, with ANP as a surrogate model, is utilized to find an optimal design for unseen traffic patterns through limited online simulations. Our experiment results show that our method outperforms state-of-the-art baselines on complex road networks in terms of the number of waiting vehicles. Surprisingly, the deployment of our method into a real-world traffic system was able to improve traffic throughput by 4.80% compared to the original strategy.

This paper presents a novel approach for modeling and predicting patterns of events in time-series learning, named graph recurrent temporal point process (GRTPP). Prior research has focused on using deep learning techniques, such as recurrent neural networks (RNNs) or attention-based sequential data embedding, on modeling the time-varying intensity of events. However, these models were typically limited to modeling a single intensity function capturing the event occurrence of all event types simultaneously. GRTPP addresses this issue by encoding multivariate event sequences into a sequence of graphs, where each node contains information about the event occurrence and time. The sequence of graphs is then embedded into node embeddings for each event type, taking into account the relationships between the event types. By integrating the estimated intensity functions, GRTPP predicts the event type and the timing of the next event. The proposed GRTPP model offers improved effectiveness and explainability compared to previous models, as demonstrated through empirical evaluations on five real-world datasets and the actual credit card transaction dataset. The code is available at https://github.com/im0j/GRTPP https://github.com/im0j/GRTPP.

In this study, we propose an adversarially-trained phase-consistent network (APCNet), which is a semi-supervised signal classification approach. The proposed classification model is trained with datasets that contain a small fraction of labeled output so as to design (1) an effective representation of the input time series (vibration signal) to extract important factors for the model to discriminate between different bearing conditions, and (2) a latent representation for the data to reflect the true data distribution precisely. To achieve these goals, APCNet suggests three novelties: the vibration-specific encoder, the phase-consistency regularization, and the adversarially-trained latent distribution alignment of the labeled and unlabeled distributions. We conduct experiments on two public bearing datasets and one public motor operating dataset to evaluate the performance of APCNet. We interpret the model's capabilities with different data label ratios and latent distribution analysis. The results show that APCNet performs well on datasets with small labeled to unlabeled data ratio. Also, we show that APCNet achieves our objectives of capturing important vibration signals features and modeling the true data distribution effectively.

In this study, we present a hypergraph convolutional recurrent neural network (HGC-RNN), which is a prediction model for structured time-series sensor network data. Representing sensor networks in a graph structure is useful for expressing structural relationships among sensors. Conventional graph structure, however, has a limitation on representing complex structure in real world application, such as shared connections among multiple nodes. We use a hypergraph, which is capable of modeling complicated structures, for structural representation. HGC-RNN performs a hypergraph convolution operation on the input data represented in the hypergraph to extract hidden representations of the input, while considering the structural dependency of the data. HGC-RNN employs a recurrent neural network structure to learn temporal dependency from the data sequence. We conduct experiments to forecast taxi demand in NYC, traffic flow in the overhead hoist transfer system, and gas pressure in a gas regulator. We compare the performance of our method with those of other existing methods, and the result shows that HGC-RNN has strengths over baseline models.

The use of data-driven predictive models is becoming increasingly popular in engineering and manufacturing sectors. This paper discusses the deployment of Gaussian Process Regression (GPR) predictive models for smart manufacturing. A scoring engine is developed based on the Predictive Model Markup Language (PMML) standard to illustrate the portability of predictive models among different statistical tools and different platforms. Specifically, we evaluate the tradeoffs between embedding GPR-based predictive models on a physical device and executing the predictive models on a managed cloud platform like the Google Compute Engine. We compare the performance of the two deployment strategies with two predictive models, namely an energy consumption model and a milling tool condition model, that are built with data from a Mori Seiki CNC milling machine. We describe how the response time of the two deployment strategies is related to the network latency and computational speed of the scoring machine hardware. It is shown that the time required to calculate model predictions is a significant factor in the overall response time of the embedded scoring engine. We demonstrate that the scoring engine on the cloud platform can achieve a lower response time and higher prediction rate than the microcomputer, due to the superior computational performance of the cloud-based hardware.

This paper describes a real-time data collection framework and an adaptive machining learning method for constructing a real-time energy prediction model for a machine tool. To effectively establish the energy consumption pattern of a machine tool over time, the energy prediction model is continuously updated with new measurement data to account for time-varying effects of the machine tool, such as tool wear and machine tool deterioration. In this work, a real-time data collection and processing framework is developed to retrieve raw data from a milling machine tool and its sensors and convert them into relevant input features. The extracted input features are then used to construct the energy prediction model using Gaussian Process (GP) regression. To update the GP regression model with real-time streaming data, we investigate the use of sparse representation of the covariance matrix to reduce the computational and storage demands of the GP regression. We compare computational efficiency of sparse GP to that of full GP regression model and show the effectiveness of the sparse GP regression model for tracking the variation in the energy consumption pattern of the target machine.

Recent advances in machine automation and sensing technology offer new opportunities for continuous condition monitoring of an operating machine. This paper describes an intelligent machine monitoring framework that integrates and utilizes data collection, management, and analytics to derive an adaptive predictive model for the energy usage of a milling machine. This model is designed using a Gaussian Process (GP) regression algorithm, which is a flexible regression method that also provides an uncertainty estimate. To improve computational efficiency, we propose a Collective Gaussian Process (CGP) in which the overall energy prediction is made by constructing local GP models weighted by probability distribution functions obtained using the Gaussian Mixture Model (GMM) technique. Finally, we demonstrate the ability of the proposed monitoring framework to construct an energy prediction model to predict the energy used to machine a part.