This paper reports on the first international competition on AI for the traveling salesman problem (TSP) at the International Joint Conference on Artificial Intelligence 2021 (IJCAI-21). The TSP is one of the classical combinatorial optimization problems, with many variants inspired by real-world applications. This first competition asked the participants to develop algorithms to solve an orienteering problem with stochastic weights and time windows (OPSWTW). It focused on two learning approaches: surrogate-based optimization and deep reinforcement learning. In this paper, we describe the problem, the competition setup, and the winning methods, and give an overview of the results. The winning methods described in this work have advanced the state-of-the-art in using AI for stochastic routing problems. Overall, by organizing this competition we have introduced routing problems as an interesting problem setting for AI researchers. The simulator of the problem has been made open-source and can be used by other researchers as a benchmark for new learning-based methods. The instances and code for the competition are available at https://github.com/paulorocosta/ai-for-tsp-competition.

Deep Reinforcement Learning (RL) has achieved high success in solving routing problems. However, state-of-the-art deep RL approaches require a considerable amount of data before they reach reasonable performance. This may be acceptable for small problems, but as instances grow bigger, this fact severely limits the applicability of these methods to many real-world instances. In this work, we study a setting where the agent can access data from previously handcrafted heuristics for the Traveling Salesman Problem. In our setting, the agent has access to demonstrations from 2-opt improvement policies. Our goal is to learn policies that can surpass the quality of the demonstrations while requiring fewer samples than pure RL. In this study, we propose to first learn policies with Imitation Learning (IL), leveraging a small set of demonstration data to accelerate policy learning. Afterward, we combine on policy and value approximation updates to improve performance over the expert's performance. We show that our method learns good policies in a shorter time and using less data than classical policy gradient, which does not incorporate demonstration data into RL. Moreover, in terms of solution quality, it performs similarly to other state-of-the-art deep RL approaches.

Recent works using deep learning to solve the Traveling Salesman Problem (TSP) have focused on learning construction heuristics. Such approaches find TSP solutions of good quality but require additional procedures such as beam search and sampling to improve solutions and achieve state-of-the-art performance. However, few studies have focused on improvement heuristics, where a given solution is improved until reaching a near-optimal one. In this work, we propose to learn a local search heuristic based on 2-opt operators via deep reinforcement learning. We propose a policy gradient algorithm to learn a stochastic policy that selects 2-opt operations given a current solution. Moreover, we introduce a policy neural network that leverages a pointing attention mechanism, which unlike previous works, can be easily extended to more general $k$-opt moves. Our results show that the learned policies can improve even over random initial solutions and approach near-optimal solutions at a faster rate than previous state-of-the-art deep learning methods.

Water distribution networks have shown an increased rate of failure due to material deterioration. In this paper, we apply a Recurrent Neural Hawkes Process model to learn the failure intensity function of water pipes. The failure intensity function is learned based on two components: the base failure rate that is determined by the unique pipe profile attributes, and the effect of past failures. Compared to the existing solutions, our model is able to predict the time to next failure on an individual water pipe level. The learned failure intensity function is used to identify value points in the deterioration process of water pipes that represent their economical end-of-life. We use data from a Dutch water distribution network that consists of 49,600 km of pipelines to test the performance of the proposed model. We have made this dataset available online.

In this work we study a dynamic pricing problem with demand censoring and limited price changes. In our problem there is a seller of a single product that aims to maximize revenue over a finite sales horizon. The seller does not know the form of the mean demand function but does have some limited knowledge. We assume that the seller has a hypothesis set of mean demand functions and that the true mean demand function is an element of this set. Furthermore, the seller faces a business constraint on the number of price changes that is allowed during the sales horizon. More specifically, the number of price changes that the seller is allowed to make is bounded above by a finite integer. We furthermore assume that the seller can only observe the sales (minimum between realized demand and available inventory) and thus that demand is censored. In each period the seller can replenish his inventory to a particular level. The objective of the seller is to set the best price and inventory level in each period of the sales horizon in order to maximize his profit. The profit is determined by the revenue of the sales minus holding costs and costs for lost sales (unsatisfied demand). In determining the best price and inventory level the seller faces and exploration-exploitation trade-off. The seller has to experiment with different prices and inventory levels in order to learn from historical sales data which contains information about market responses to offered prices. On the other hand, the seller also needs to exploit what it has learned and set prices and inventory levels that are optimal given the information collected so far. We propose a heuristic policy for this problem and study its performance using numerical experiments. The results are promising and indicate that the growth rate of regret of the policy is sub-linear with respect to the sales horizon.

In many situations, simulation models are developed to handle complex real-world business optimisation problems. In our case, a discrete-event simulation model is used to simulate trailer management and fleet configuration in a big Fast-Moving Consumer Goods company. To address the problem of finding suitable simulation inputs for optimisation, we propose a simulation optimisation approach. The simulation optimisation model combines metaheuristic search (genetic algorithm), with an approximation model filter (feed-forward neural network) to optimise the input configuration of the simulation model. In this work, we introduce an ensure probability that overrules the rejection of potential solutions by the approximation model and demonstrate its effectiveness. In addition, we evaluate the impact of the genetic algorithm parameters and show how population size, filter threshold, and mutation probability impact overall fleet optimisation performance. Lastly, we compare the proposed method with a single global approximation model and a random-based approach, our results demonstrate the advantage of our method in terms of computational time and solution quality.