Abstract To address the problem of high network energy consumption and imbalanced load distribution in data transmission, this paper proposes a novel deep reinforcement learning algorithm to jointly optimize energy consumption and link load distribution. We design a three-layer back-propagation neural network. It iteratively adjusts weights based on past states and actions, and minimizes the loss to improve action prediction accuracy. It guides the agent to make reasonable routing decisions within complex network environments. Based on this, we leverage Q-learning to seek paths for transmission demands. By dynamically aggregating and balancing traffic, energy efficiency and load balance can be achieved. We design reward functions from the viewpoint of link and node, respectively, for different optimization objectives. In order to obtain high efficiency and good robustness, we improve roulette-based Chebyshev scalarization function to solve the weight selection problem among multi-objectives. We update the Pareto set via multiple state transitions to approximate optimal solution. We use the Euclidean distance to measure optimizing effect of both objectives. Simulation results illustrate that our algorithm can effectively reduce network energy consumption and balance load distribution.

Abstract Simultaneously reducing network energy consumption and delay is a hot topic today. This paper addresses this issue by designing a novel multi-objective data transmission optimization algorithm based on deep reinforcement learning. A three-layer back propagation (BP) neural network is designed to improve the accuracy of environmental state prediction, by learning from historical state and action sequence data, which can help the agent make better decision for routing selection in complex network environment. Based on this, we use Q-Learning to find routing for transmission demands, aggregating more traffic through less links and routers, to reduce energy consumption and delay. To enhance the efficiency and robustness of the algorithm, a new reward mechanism is designed based on the traffic demand and the link state. The algorithm divides candidate links into three levels for path selection so that a better solution can be obtained on the basis of ensuring feasible solutions are obtained. Continuous updating of the Pareto set through multiple state steps approximates the optimal solution. We leverage the Euclidean distance to the reference point to measure the optimization effect of the two objectives. The simulation results show that this algorithm outperforms existing algorithms in reducing energy consumption and network delay.

Software Defined Network (SDN) has been considered as one of the most promising next-generation network solutions due to its network programmability. However, the upgrade from legacy IP network to pure SDN network is in general a gradual process. From a global perspective, a dynamic upgrade strategy should not only aim to pursue the local goal at each step, but also strive to optimize the final global solution when the upgrading process terminates. This raises three essential questions: which switches to upgrade, when to upgrade, and how to deploy controllers. Due to the interaction between the local goals at intermediate steps and the global goal at the final step, answering these questions altogether from a global perspective is challenging. In this paper, we study the dynamic SDN upgrade problem from a global perspective and answer these three questions altogether. We formulate the problem as a dynamic optimization problem that optimizes the global and local goals at the same time. We then propose two new formulations to combine the global and local goals, and two heuristic algorithms to solve them, respectively. We evaluate the proposed model and algorithms on realistic network topologies. The results show their feasibility and superiority.

Summary The transmission process of disaster backup with long distance and massive data causes huge energy consumption. Reducing the number of occupied intermediate forwarding devices and shortening the transmission completion time are two key factors for energy saving. Not jointly considering them, previous work can hardly realize optimal energy‐aware transmission only by single objective optimization. For the first time, we leverage multiobjective reinforcement learning to simultaneously minimize the number of occupied intermediate forwarding devices and the transmission completion time in software defined network. We propose two‐level reinforcement learning, consisting of search and selection operation. In the internal reinforcement learning, we aim to reduce hop number, improve node sharing degree, and give priority to the links with larger residual capacity. Then in the external level, we aim to reduce the total number of occupied devices and increase the total backup flow. We leverage Chebyshev scalarization function based on pseudo‐random proportional rule to simplify weight selection, and enforce exploration to avoid falling into local optimum. We design the vector of rewards for different objectives, and update Pareto approximate set by multiple state steps to approach the optimal solution. Our strategy solves the weight selection problem successfully and achieves lower energy consumption.

Summary In cloud datacenter network, deadline‐aware disaster evacuation transfers the endangered data out of disaster zone using limited residual network resources. Previous work has not jointly considered the selection of safe datacenter and reasonable allocation of bandwidth proportion in time‐varying postdisaster network environment. Therefore, they cannot make full use of network transmission capability. Based on our earlier work, we propose a new time‐varying disaster evacuation strategy with flexible traffic scheduling. We aim to maximize disaster evacuation capability in the disaster spread scenario. We construct a new disaster‐aware time‐expanded network model to divide time slots according to progressive disaster spread, and optimize the utilization of evacuation capability in the current disaster stage. In each time slot, we carry out two‐step optimization including safe datacenter selection and proportional bandwidth allocation. Especially, we select store‐and‐forward node to ensure the safety of evacuated data in the next time slot, and use marked evacuation routing search based on transmission requirement to improve the utilization of evacuation capability. Through extensive simulations we demonstrate that our strategy achieves better performance with higher evacuation transmission efficiency in the disaster spread scenario.

Summary To leverage periodic disaster backup in a cloud data center (DC) network, previous studies employ disjoint unicast paths for bulk data transfers among multiple geographically distributed DCs, causing massive unnecessary traffic duplication. This not only adds the overhead but also may result in severe network congestion. With flexible network resource management in software‐defined networks and powerful traffic aggregation capability of multicast, we propose capacity‐constrained multicast to realize cost‐efficient disaster backup. First, considering limited backup storage capacity and essential redundancy guarantee, we construct a capacity‐constrained multicasting backup model. Then, we formulate the disaster backup problem as capacity‐constrained multiple Steiner tree problem, which is NP‐hard. To solve this problem, we design a new multicasting backup ant colony optimization algorithm based on requirement‐aware growth. It directly optimizes every disaster‐backup multicast tree (DBMT) from its root node to cover enough destination nodes guaranteeing sufficient redundancy and then expands them into the forest under the guidance of a multicast tree shared degree, the ratio of available storage capacity, and backup load distribution offset. We introduce unique edge fitness evaluation and pheromones for every DBMT to reduce mutual influences among multiple trees. Extensive simulations demonstrate that our strategy performs with less bandwidth consumption cost and relatively good backup load distribution fairness simultaneously.

Mobile crowdsourcing takes advantage of mobile devices such as smart phones and tablets to process data for a lot of applications (e.g., geotagging for mobile touring guiding monitoring and spectrum sensing). In this paper, we propose a mobile crowdsourcing paradigm to make a task requester exploit encountered mobile workers for high-quality results. Since a task may be too complex for a single worker, it is necessary for a task requester to divide a complex task into several parts so that a mobile worker can finish a part of the task easily. We describe the task crowdsourcing process and propose the worker arrival model and task model. Furthermore, the probability that all parts of the complicated task are executed by mobile workers is introduced to evaluate the result of task crowdsourcing. Based on these models, considering computing capacity and rewards for mobile workers, we formulate a task partition problem to maximize the introduced probability which is used to evaluate the result of task crowdsourcing. Then, using a Markov chain, a task partition policy is designed for the task requester to realize high-quality mobile crowdsourcing. With this task partition policy, the task requester is able to divide the complicated task into precise number of parts based on mobile workers’ arrival, and the probability that the total parts are executed by mobile workers is maximized. Also, the invalid number of task assignment attempts is analyzed accurately, which is helpful to evaluate the resource consumption of requesters due to probing potential workers. Simulations show that our task partition policy improves the results of task crowdsourcing.

To prevent data losses and service interruptions caused by natural disasters or human misconduct, we need to leverage periodic disaster backup among geographically distributed multiple datacenters. Previous works aimed at bandwidth allocation to achieve maximum network flow for every backup pair one by one or fair load distribution for backup datacenters respectively, without jointly optimizing the two problems to realize rapid and fair disaster backup. In this paper, we propose a new Receiving-Capacity-Constrained Rapid and Fair Disaster Backup strategy in the Software Defined Network scenarios. We formulate the disaster backup problem as a Receiving-Capacity-Constrained Capacitated Multi-Commodity Flow problem which is NP-complete. To solve the problem, we first construct a new effective Receiving-Capacity-Aware network model guaranteeing upper bound of bandwidth allocation to achieve fair load distribution for backup datacenters. And in this network model, we further propose a Bound-Aware Ant Colony Optimization algorithm satisfying backup flow constraint and lower bound constraint to achieve fast data transmission for backup pairs. Through extensive simulations, we demonstrate that our strategy has better performance with less total backup time and more fair load distribution than state-of-the-art algorithms.