This paper presents Post-Decision Proximal Policy Optimization (PDPPO), a novel variation of the leading deep reinforcement learning method, Proximal Policy Optimization (PPO). The PDPPO state transition process is divided into two steps: a deterministic step resulting in the post-decision state and a stochastic step leading to the next state. Our approach incorporates post-decision states and dual critics to reduce the problem's dimensionality and enhance the accuracy of value function estimation. Lot-sizing is a mixed integer programming problem for which we exemplify such dynamics. The objective of lot-sizing is to optimize production, delivery fulfillment, and inventory levels in uncertain demand and cost parameters. This paper evaluates the performance of PDPPO across various environments and configurations. Notably, PDPPO with a dual critic architecture achieves nearly double the maximum reward of vanilla PPO in specific scenarios, requiring fewer episode iterations and demonstrating faster and more consistent learning across different initializations. On average, PDPPO outperforms PPO in environments with a stochastic component in the state transition. These results support the benefits of using a post-decision state. Integrating this post-decision state in the value function approximation leads to more informed and efficient learning in high-dimensional and stochastic environments.

Abstract In social networks, the influence maximization problem requires selecting an initial set of nodes to influence so that the spread of influence can reach its maximum under certain diffusion models. Usually, the problem is formulated in a two‐stage un‐budgeted fashion: The decision maker selects a given number of nodes to influence and observes the results. In the adaptive version of the problem, it is possible to select the nodes at each time step of a given time interval. This allows the decision‐maker to exploit the observation of the propagation and to make better decisions. This paper considers the adaptive budgeted influence maximization problem, that is, the adaptive problem in which the decision maker has a finite budget to influence the nodes, and each node requires a cost to be influenced. We present two solution techniques: The first is an approximated value iteration leveraging mixed integer linear problems while the second exploits new concepts from graph neural networks. Extensive numerical experiments demonstrate the effectiveness of the proposed approaches.

Social Engagement is a novel business model transforming final users of a service from passive into active components.In this framework, people are contacted by a company and they are asked to perform tasks in exchange for a reward.This arises the complicated optimization problem of allocating the different types of workforce so as to minimize costs.We address this problem by explicitly modeling the behavior of contacted candidates through consolidated concepts from utility theory and proposing a chance-constrained optimization model aiming at optimally deciding which user to contact, the amount of the reward proposed, and how many employees to use in order to minimize the total expected costs of the operations.A solution approach is proposed and its computational efficiency is investigated through experiments.

Abstract Computer network systems are often subject to several types of attacks. For example, an excessive traffic load sent to a web server for making it unusable is the main technique introduced by the Distributed Denial of Service (DDoS) attack. A well-known method for detecting attacks consists in analyzing the sequence of source IP addresses for detecting possible anomalies. With the aim of predicting the next IP address, the Probability Density Function of the IP address sequence is estimated. Anomalous requests are detected via predicting source’s IP addresses in future accesses to the server. Thus, when an access to the server occurs, the server accepts only the requests from the predicted IP addresses and it blocks all the others. The approaches used to estimate the Probability Density Function of IP addresses range from the sequence of IP addresses seen previously and stored in a database to address clustering, for instance via the K-Means algorithm. Instead, the sequence of IP addresses is considered as a numerical sequence in this paper, and non-linear analysis of this numerical sequence is applied. In particular, we exploited non-linear analysis based on Volterra Kernels and Hammerstein models. The experiments carried out with datasets of source IP address sequences show that the prediction errors obtained with Hammerstein models are smaller than those obtained both with the Volterra Kernels and with the sequence clustering based on the K-Means algorithm.

Tactical Capacity Planning (TCP) is becoming a crucial part of logistics in the current environment of demand-driven economics. This paper proposes an innovative approach in the TCP setting, consisting of using the collected historical data of the geographical position and the volume of the orders to plan the capacity requirements for the next day. To this end, the clustering of the city to microzones is introduced using K-means clustering. Then, four different methods (Gaussian Process regression, ARIMA model, Neural Network regression, and Long Short Term Memory network) are used to forecast the next day order volume for each of the clusters. Finally, the Variable Cost and Size Bin Packing problem solved with the predicted demand to outline the usage of a heterogeneous fleet required to serve the next time period. Through experiments on the real data, we conclude, that the proposed algorithm is satisfying the decision safety framework with completely unknown demand and could also be used for other demand forecast applications.

In recent years, researchers have oriented their studies towards new technologies based on quantum physics that should resolve complex problems currently considered to be intractable.This new research area is called Quantum Computing.What makes Quantum Computing so attractive is the particular way with which quantum technology operates and the great potential it can offer to solve real-world problems.This work focuses on solving assignment-like combinatorial optimization problems by exploiting this novel computational approach.A case-study, denoted as the Seating Arrangement Optimization problem, is considered.It is modeled through the Quadratic Unconstrained Binary Optimization paradigm and solved through two tools made available by the D-Wave Systems company, QBSolv, and a quantum-classical hybrid system.The obtained experimental results are compared in terms of solution quality and computational efficiency.

Modern cloud applications are required to be distributed, scalable, and reliable. The microservice architectural style enables developers to achieve this goal with reduced effort. Nonetheless, microservices deployment is not trivial due to the heterogeneity of the microservices in terms of both functional and non-functional requirements. This is also true when considering the monitoring requirements that are specific to each microservice and must be satisfied in order to enable the verification of the application objectives satisfaction. However, not all providers offer the same set of metrics with the same quality. The goal of this paper is to provide an approach for supporting the deployment of microservices in multi-cloud environments focusing on the Quality of Monitoring. Adopting a multi-objective mixed integer linear optimisation problem, our approach supports the application owner in finding the optimal deployment for satisfying all the constraints and maximising the quality of monitored data, while minimising the costs. To this end, a knowledge base is introduced to mediate between the perspectives of the cloud provider and the application owner, while a Bayesian Network is adopted to enhance the provider’s monitoring capabilities by estimating metrics requested by the application owners that the cloud provider is not able to monitor.

OLAP is an authoritative analytical tool in the emerging big data analytics context, with particular regards to the target distributed environments (e.g., Clouds). Here, privacy-preserving OLAP-based big data analytics is a critical topic, with several amenities in the context of innovative big data application scenarios like smart cities, social networks, bio-informatics, and so forth. The goal is that of providing privacy preservation during OLAP analysis tasks, with particular emphasis on the privacy of OLAP aggregates. Following this line of research, in this paper we provide a deep contribution on experimenting and assessing a state-of-the-art distributed privacy-preserving OLAP framework, named as SPPOLAP, whose main benefit is that of introducing a completely-novel privacy notion for OLAP data cubes.

Online scheduling has been an attractive field of research for over three decades.Some recent developments suggest that Reinforcement Learning (RL) techniques have the potential to deal with online scheduling issues effectively.Driven by an industrial application, in this paper we apply four of the most important RL techniques, namely Q-learning, Sarsa, Watkins's Q(λ), and Sarsa(λ), to the online single-machine scheduling problem.Our main goal is to provide insights on how such techniques perform.The numerical results show that Watkins's Q(λ) performs best in minimizing the total tardiness of the scheduling process.

In this paper, we complement the research results provided with the OO-XAHM (Object-Oriented XML Adaptive Hypermedia Model), a state-of-the-art proposal for supporting adaptation features of the Web. In particular, in this contribution we provide: (i) the complete database-like implementation of OO-XAHM: (ii) a complete case study that focuses the attention on the well-known Italian archeological site Pompeii.

Electric vehicles are accelerating the world's transition to sustainable energy.Nevertheless, the lack of a proper charging station infrastructure in many real implementations still represents an obstacle for the spread of such a technology.In this paper, we present a real case application of optimization techniques in order to solve the location problem of electric charging stations in the district of Biella, Italy.The plan is composed by several progressive installations and decision makers pursue several objectives that might be in contrast.For this reason, we present an innovative framework based on the comparison of several ad-hoc Key Performance Indicators for evaluating many different aspects of a location solution.

Due to the diffusion of Internet of Things (IoT), many devices such as water meters, smart dumpsters, and many other objects have the capacity to record data. Gathering these data from the devices is a problem that could be solved in three ways: by building a huge network infrastructure, by using regular workforce or by using opportunistic IoT networks, i.e. by using as mobile hotspots the devices of selected users. The latter is cheaper than the others, requiring only the payment of a reward to the users. In this paper, we introduce a Multi Period Assignment problem, i.e. a problem for planning the operations of Opportunistic IoT networks. The problem minimizes the sum of user rewards, while gathering data from all devices. An effective heuristic method able to deal with realistic-sized instances is presented. The heuristic is able to find, by using a reasonable amount of time, the optimum for 124 out of 128 instances and reach gaps smaller than 0.1% for the remaining 4 instances.