The surge of artificial intelligence (AI) has intensified compute-intensive tasks, sharply increasing the need for energy-efficient management in geo-distributed data centers. Existing approaches struggle to coordinate task scheduling and cooling control due to mismatched time constants, stochastic Information Technology (IT) workloads, variable renewable energy, and fluctuating electricity prices. To address these challenges, we propose D3T, a dual-timescale deep reinforcement learning (DRL) framework that jointly optimizes task scheduling and thermal management for energy-efficient geo-distributed data centers. At the fast timescale, D3T employs Deep Q-Network (DQN) to schedule tasks, reducing operational expenditure (OPEX) and task sojourn time. At the slow timescale, a QMIX-based multi-agent DRL method regulates cooling across distributed data centers by dynamically adjusting airflow rates, thereby preventing hotspots and reducing energy waste. Extensive experiments were conducted using TRNSYS with real-world traces, and the results demonstrate that, compared to baseline algorithms, D3T reduces OPEX by 13% in IT subsystems and 29% in cooling subsystems, improves power usage effectiveness (PUE) by 7%, and maintains more stable thermal safety across geo-distributed data centers.

Multi-beam low earth orbit (LEO) satellites, with their wide coverage, high communication rates, low latency, and flexibility, are essential components in the 5G and 6G eras. However, challenges such as uneven ground user distribution, multi-dimensional resource coupling, inter-beam interference, and dynamic network topology complicate efficient resource management. This paper presents a joint beam hopping and resource allocation (BHRA) scheme within a Digital Twin (DT)empowered multi-beam LEO satellites network. The complex resource management problem is divided into two sub-problems: traffic-satellite allocation and joint multi-satellite BHRA. First, cell traffic is allocated among satellites to balance load using deep reinforcement learning (DRL). Then, Hierarchical Proximal Policy Optimization (HPPO) optimizes multi-satellite beam hopping and resource allocation to meet demand. Extensive simulation results demonstrate that the proposed method reduces satellite load imbalance by approximately 83%, achieves the highest throughput compared to other algorithms, and generalizes well as traffic demand increases.

Due to the uneven distribution of ground communication demands and the multi-coverage characteristics of Low Earth Orbit (LEO) satellites, a key challenge arises in aligning the limited onboard satellite resources with the heterogeneous ground demands through multi-satellite collaboration. Traditional approaches, which rely on centralized beam coverage planning by ground control centers, often result in scheduling delays and increased communication overhead, thereby constraining system responsiveness and flexibility. This paper proposes a neighboring satellite beam coordination framework based on satellite autonomous control, enabling local satellite decisionmaking for beam coverage and real-time responses to resource allocation demands. Specifically, the framework is divided into two phases: the prediction-approximation phase and the strategy decision phase. In the prediction-approximation phase, satellites predict current communication demands based on historical traffic data from ground cells and approximate the channel state information using easily observable elevation angle data. In the strategy decision phase, a neighboring satellite collaborative beam-hopping algorithm, based on global information sharing and independent policy reinforcement learning, is proposed. This approach enables beam coordination between neighboring satellites without the need for inter-satellite communication. In a twosatellite neighboring coordination system, the proposed method achieves a throughput improvement of 5.78%–28.54% and a delay reduction of 2.63%–26.56% compared to baseline schemes. Relative to ground control center-based approaches, the proposed method significantly reduces control latency and enables faster response. Further scalability and limitations analysis confirms that the proposed framework provides efficient, stable, and robust resource scheduling, with excellent scalability characteristics.

Low-Earth orbit (LEO) satellites utilizing beam hopping (BH) technology offer extensive coverage, low latency, high bandwidth, and significant flexibility. However, the uneven geographical distribution and temporal variability of ground traffic demands, combined with the high mobility of LEO satellites, present significant challenges for efficient beam resource utilization. Traditional BH methods based on GEO satellites fail to address issues such as satellite interference, overlapping coverage, and mobility. This paper explores a Digital Twin (DT)-based collaborative resource allocation network for multiple LEO satellites with overlapping coverage areas. A two-tier optimization problem, focusing on load balancing and cell service fairness, is proposed to maximize throughput and minimize inter-cell service delay. The DT layer optimizes the allocation of overlapping coverage cells by designing BH patterns for each satellite, while the LEO layer optimizes power allocation for each selected service cell. At the DT layer, an Actor-Critic network is deployed on each agent, with a global critic network in the cloud center. The A3C algorithm is employed to optimize the DT layer. Concurrently, the LEO layer optimization is performed using a Multi-Agent Reinforcement Learning algorithm, where each beam functions as an independent agent. The simulation results show that this method reduces satellite load disparity by about 72.5% and decreases the average delay to 12ms. Additionally, our approach outperforms other benchmarks in terms of throughput, ensuring a better alignment between offered and requested data.

Vehicular named data networking (V-NDN) is promising to match dynamic topology of vehicular ad hoc networks. However, it has some potential issues such as broadcast storm of Interest packets due to flooding and blind forwarding strategy at intermediate nodes. To address these issues, we propose a network coding enhanced forwarding strategy with predictive content discovery (NC-PCD) mechanism which functions at key nodes (e.g., road intersections). First, content providers provide innovative encoded data packets. Second, the NDN packet formats are extended to carry the information about the innovative degree, distance, congestion level and path quality, etc. Finally, the NDN router at the key node collects information to select the optimal forwarding direction for subsequent requests. This problem is formulated as a joint optimization problem and solved by deep reinforcement learning. Simulation results show that the proposed NC-PCD is significantly superior to the existing forwarding strategies in terms of Interest satisfied ratio, response delay and number of transmitted packets.

The Low Earth Orbit (LEO) satellite constellation has been recognized as an important component of the future 6G network. However, due to the high-speed movement of LEO satellites and the potential for link failures, achieving optimal satellite communication performance with a static inter-satellite links (ISLs) scheme is challenging. To solve this problem, this paper proposes an ISL re-planning algorithm with considering link failures based on multi-agent deep reinforcement learning (named ReISL). In ReISL, a multi-objective optimization problem is formulated to maximize the system capacity while minimizing the link switching costs. Then, multi-agent deep reinforcement learning is employed to derive the optimal ISL re-planning schemes, where each satellite utilizes Double Deep Q-Network (DDQN). Finally, extensive experiments are carried out and the results demonstrate that our proposed algorithm ReISL can outperform the baseline algorithms.

Earth observation satellites in low earth orbit (LEO) collect a large amount of image data daily, while space-to-ground links have become the major bottleneck for data transmission due to the limited bandwidth. Existing approaches focus on exploring more efficient routing strategies to achieve better data transmission but still struggle to keep pace with the surging volume of observed data. However, the advancement of onboard computing power has opened the possibility of processing data on satellites to reduce the transmitted data volume. This paper proposes a distributed deep reinforcement learning (DRL) algorithm to improve transmission efficiency by jointly optimizing computing and routing. Aiming to minimize task latency while considering the limitations of satellite storage resources, the problem is modeled as a partially observable Markov process (POMDP). An algorithm based on dueling double deep Q-Network (Dueling-DDQN) is proposed to achieve dynamic decision-making utilizing local and neighboring resource states. Furthermore, a method for dynamic optimization of backhaul destinations is proposed, using the pre-trained Q-network to estimate action values across multiple candidate destination satellites, thus enabling further optimization of data transmission without additional training. Simulation results indicate that the proposed algorithm achieves the lowest latency across various task loads compared to baseline methods.

High-quality video can provide viewers with a better visual and immersive experience, but it typically requires a significantly higher network bandwidth to accommodate the larger amount of video data. The existing and commonly-used adaptive bitrate (ABR) approaches cannot provide high-quality video services for viewers when the network is poor. To address this issue, we propose an edge super-resolution assisted adaptive video streaming, named SuperABR, to improve viewers' Quality of Experience (QoE) and mitigate the influence of poor networks. First, to prevent the mismatch between the available computing resources and the VSR workload, the dynamics and trends of the available computing capability are extracted from a series of historical VSR reconstructing delays. Second, to optimize the video quality, rebuffering time, and video quality jitter for viewers, we formulate the optimization model by fully considering the states of all caching queues in SuperAbr and imposing probability constraints on the playback queue. Third, a queue-learning-based adaptive video streaming algorithm is proposed in SuperABR to jointly determine the source transmission resolution and the edge VSR-reconstructed resolution, which is essentially a Deep Reinforcement Learning (DRL) method with queue constraints. Finally, extensive experiments illustrate that SuperABR can improve QoE by 20%-76% compared to four baseline algorithms.

The Low Earth Orbit (LEO) satellite constellation has been recognized as an important component of the future 6G network. Due to the high speed movement and limited on-board energy of LEO satellites, as well as the uneven distribution of service requests on the ground, it is difficult to achieve optimal satellite communication performance using a static inter-satellite links (ISLs) scheme and a fixed transmission power per link. To solve this problem, this paper proposes a joint optimization algorithm based on parameterized deep reinforcement learning (named DeepISL) for dynamic planning of ISLs between different planes and transmit power allocation per link. First, a partially observable Markov decision process (POMDP) is established by modeling the communication, energy, and overall energy efficiency as well as the antenna steering costs. Second, to solve the hybrid action space problem with discrete action variables for ISL planning and continuous action variables for power allocation, deep multi-agent reinforcement learning with parameterized action space is used to obtain the optimal joint strategy. Finally, extensive experiments illustrate that our proposed algorithm can improve the energy efficiency of the constellation by 4.2% ~ 10.5% compared to the comparison algorithms, and can also achieve better performance in terms of throughput and ISLs switching ratio.

High quality video can provide viewers with better visual and immersive experience in video streaming systems. However, traditional such systems with adaptive bitrate algorithms can only provide low-resolution videos when the network conditions deteriorate. To break the strong dependency of video transmission on network, we design and demonstrate iSAW, a super-resolution-assisted intelligent adaptive WebRTC video streaming system, by using two mobile devices and a policy server. iSAW performs a lightweight and scalable super-resolution (SR) model on the mobile devices to enhance the quality of received videos, and leverages an learning-based adaptive algorithm on the policy server to jointly adjust the transmitted video resolution and SR reconstructed video resolution according to dynamic network conditions and client computing power. The enhanced real-time video is smooth and the performance is improved obviously.

Unmanned aerial vehicle (UAV)-assisted communications are playing an increasingly critical role in emergency aids and disaster recovery, thanks to their excellent rapid deployment capabilities. However, limited on-board caches and power supplies pose a serious challenge to the cooperative scheduling of multiple UAVs, with additional complexity introduced by different natures of caching decision and power adjustments. To address the issues, we formulate joint optimization of collaborative caching and power allocation as a hybrid discrete-continuous optimization problem and introduce a parameterized deep Q-networks (PA-DQN) based collaborative caching and power allocation (PA-DQN-CCPA) joint optimization algorithm, taking into consideration of emergency service models. Extensive simulations demonstrate it significantly improves cache hit ratio by 18.75%, 10.14% and reduces average delay by 21.36%, 11.96% compared to the algorithm based on deep deterministic policy gradient (DDPG) and deep Q-networks (DQN), respectively.

The rapid advancements in cloud computing, Big Data and their related applications have led to a skyrocketing increase in data center energy consumption year by year. The prior approaches for improving data center energy efficiency mostly suffer from high system dynamics or the complexity of data centers. In this paper, we propose an optimization framework based on deep reinforcement learning, named DeepEE, to jointly optimize energy consumption from the perspectives of task scheduling and cooling control. In DeepEE, a PArameterized action space based Deep Q-Network (PADQN) algorithm is proposed to tackle the hybrid action space problem. Then, a dynamic time factor mechanism for adjusting cooling control interval is introduced into PADQN (PADQN-D) to achieve more accurate and efficient coordination of IT and cooling subsystems. Finally, in order to train and evaluate the proposed algorithms safely and quickly, a simulation platform is built to model the dynamics of IT and cooling subsystems. Extensive real-trace based experiments illustrate that: 1) the proposed PADQN algorithm can save up to 15% and 10% energy consumption compared with the baseline siloed and joint optimization approaches respectively; 2) the proposed PADQN-D algorithm with dynamic cooling control interval can better adapt to the change of IT workload; 3) our proposed algorithms achieve more stable performance gain in terms of power consumption by adopting the parameterized action space.

To reduce the skyrocketing energy consumption of data centers, the prevailing approaches adopt the time-driven manner to control IT and cooling subsystems. These methods suffer from highly dynamic system states, complex action spaces and the risk of instability caused by frequent and unnecessary control operations. To tackle these problems, we propose a novel event-driven control paradigm and an optimization algorithm, under the deep reinforcement learning (DRL) framework. The principle is to make decisions based on certain critical events (e.g., overheating), rather than fixed periodic control. Specifically, we design an event-driven optimization framework to trigger control operations. Then, we present several models to describe IT and cooling subsystems, and mathematically define events to capture four types of prior factors that impact system performance. Furthermore, we develop an event-driven DRL (E-DRL) optimization algorithm to dispatch jobs and regulate cooling facilities for energy efficiency. Using two different types of real workload traces, we conduct extensive experiments to demonstrate that: 1) E-DRL reduces the number of regulating decisions by 70%$\sim$95% while achieving a comparable or even better energy efficiency in comparison with the state-of-the-art algorithm; and 2) E-DRL can adapt the control frequency to the changing operational conditions and diverse workloads.

As the Low Earth Orbit (LEO) satellite has extremely high moving speed and limited networking resources, designing dynamic routing has become a promising approach to improve satellite communication performance. Due to the hundreds of satellites within a constellation and the complex attributes of each satellite, traditional routing strategies based on centralized paradigm derivation face increasingly complex challenges. To address these issues, this paper jointly optimizes queuing delay and propagation delay by proposing a fully distributed routing algorithm based on deep reinforcement learning. Each satellite builds a partially observable Markov decision process (POMDP) model based on the spatial location and queue length of surrounding nodes and adaptively selects the next hop by calculating the estimated residual propagation delay between the neighboring satellites and the destination satellite. Simulation analysis shows that our proposed method has tremendous advantages and effectiveness.

Video services such as short video sharing have exploded due to the rapid development of Internet social media platforms. Caching video segments on satellites effectively shortens service delay and speeds up video sharing, especially for users without terrestrial Internet access. However, where to place what video and how to replace it in time is by no means an easy task, requiring careful consideration of many factors, e.g., satellite coverage, video popularity, and limited caching resource. In this paper, we propose a coverage-aware cooperative video caching algorithm (CACVC) that considers the prevalence of video in the coverage area and the collaboration between adjacent satellites. In CACVC, we model the cache placement problem of video as a Partially Observable Markov Decision Process (POMDP) to optimize the service delay of video provided by access satellites, neighboring satellites, or ground stations. We derive the optimal cache strategy by utilizing Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm with a centralized training and distributed execution paradigm. Simulation results show that the cache hit ratio can be improved by 4%~18%, and the average service delay can be reduced by 1%~14%.

Low Earth Orbit (LEO) satellite constellations are promising to provide global coverage and low latency communication by deploying a large number of small satellites and widely establishing Inter-Satellite Links (ISLs). However, due to the high motion of the LEO satellites, fixed inter-plane ISLs cannot provide long-time continuous connectivities and guarantee high-throughput communication performance. The existing dynamic planning approaches almost only consider part of the constellation information and cannot derive the optimal inter-plane ISLs. This paper proposes a dynamic Inter-plane Inter-satellite Links Planning method based on Multi-Agent deep reinforcement learning (MA-IILP) to optimize the total throughput and inter-plane ISL switching rate. We formulate a Partially Observable Markov Decision Process (POMDP) model with taking into account the Euclidean distance, communication rate and link switching cost. We derive the optimal strategy by utilizing Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm with a centralized training and distributed execution paradigm. Finally, extensive experiments are carried out and the results illustrate that our proposed approach can increase the total throughput of the target constellation by 2.8%∼7.2%, and decrease the inter-plane ISL switching rate by 30.7%∼68.4% compared to the state-of-the-art baseline algorithms.

[J1C2 Presentation Abstract at IEEE SERVICES 2022 for IEEE Transactions on Services Computing DOI 10.1109/TSC.2022.3157145]

Content caching in vehicular networks is a promising technology to dramatically reduce the request-response time and transmission delay. The existing caching policies often suffer from high computation and communication overhead and fail to well capture the dynamics of the vehicular networks and content popularity. In this paper, we propose a novel Cooperative Caching algorithm for vehicular networks with multi-level federated Reinforcement Learning (named CoCaRL) to dynamically determine which contents should be replaced and where the content requests should be served. In CoCaRL, Deep Reinforcement Learning (DRL) is employed to optimize the cooperative caching policy between RoadSide Units (RSUs) of vehicular networks, while a federated learning framework applies to reduce the computation and communication overhead in a decentralized way. To speed-up the convergence rate, we also develop a two-level aggregation mechanism for federated learning, where the low-level aggregation is performed at the RSUs and the high-level aggregation is executed at a Global Aggregator (GA). Through extensive simulation experiments, we demonstrate that our algorithm can: 1) achieve a higher hit rate than four baseline algorithms, 2) converge faster than original federated reinforcement learning without multi-level aggregation, and 3) perform good adaptability to different cache capacities and content quantities.

Named data networking (NDN) is promising for Vehicular Ad hoc Networks (VANETs) owing to its data-centric communication paradigm. However, the existing forwarding strategies for vehicular named data networks (VNDN) cannot handle well the issues of high mobility and broadcast storm. In this paper, we propose a Mobility-Predict-based Forwarding Strategy (MPFS) to tackle these issues in VNDN. First, in order to solve the problem of outdated mobility information in the neighbor table, a lightweight but highly effective approach is proposed to predict the current position in MPFS. Then, the predicted mobility information of the vehicles in the neighbor table is applied to select the next-hop forwarder(s) in both directions (road direction and reverse road direction) of the consumer. Furthermore, the vehicle that is farthest from the current forwarder with a stable link is considered as the next-hop forwarder. Finally, extensive simulations are carried out to demonstrate that MPFS has a less number of Interest packets forwarded, while maintaining a higher ratio of satisfied Interest packets compared with the baseline forwarding strategies.

The past decade witnessed the tremendous growth of power consumption in data centers due to the rapid development of cloud computing, big data analytics, and machine learning, etc. The prior approaches that optimize the power consumption of the information technology (IT) system and/or the cooling system always fail to capture the system dynamics or suffer from the complexity of system states and action spaces. In this paper, we propose a Deep Reinforcement Learning (DRL) based optimization framework, named DeepEE, to improve the energy efficiency for data centers by considering the IT and cooling systems concurrently. In DeepEE, we first propose a PArameterized action space based Deep Q-Network (PADQN) algorithm to solve the hybrid action space problem and jointly optimize the job scheduling for the IT system and the airflow rate adjustment for the cooling system. Then, a two-time-scale control mechanism is applied in PADQN to coordinate the IT and cooling systems more accurately and efficiently. In addition, to train and evaluate the proposed PADQN in a safe and quick way, we build a simulation platform to model the dynamics of IT workload and cooling systems simultaneously. Through extensive real-trace based simulations, we demonstrate that: 1) our algorithm can save up to 15% and 10% energy consumption in comparison with the baseline siloed and joint optimization approaches respectively; 2) our algorithm achieves more stable performance gain in terms of power consumption by adopting the parameterized action space; and 3) our algorithm leads to a better tradeoff between energy saving and service quality.

Network measurement is critical in network management such as performance monitoring, diagnosis, and traffic engineering. However, conventional network measurement solutions are limited by simple and fixed functionalities as well as coarse-grained statistics which often fail to precisely illustrate network conditions. In this paper, we propose an event-driven declarative query language, EQuery, for programmable network management in order to design sophisticated measurement tasks and enable event mechanism to avoid human intervene. Furthermore, we design a compiler to support the query language on the EQuery Controller, which drives the chaining query workflow with nondeterministic finite automaton (NFA), and translates measurement jobs into low-level rules/states on the physical devices. Finally, we evaluate the effectiveness of our EQuery framework on a nation-wide operational network with real-time network statistics.

Scalable video coding is a promising technique to enable flexible video transmission for heterogeneous terminals and varying channel throughput. However, it is challenging to perform in-network adaptation in conventional networks because the network nodes are uncontrollable and transparent for media streaming applications. Software-defined networking (SDN) is an attractive network technology that supports the applications to collaborate with network nodes to achieve intelligent and dynamic service provisioning. Against this changing network landscape, we redesign the scalable multimedia multicast streaming by exploiting the complete network knowledge of the SDN controller to enable intelligent scalable video transmission. The proposed scalable multimedia multicast streaming framework is capable of in-network identifying, processing, and manipulating the media streams. In order to achieve the in-network adaptation, we apply equivalent bandwidth theory to estimate the affordable video layers that a link may accommodate, and apply finite-state machine to implement adaptive enhancement layer switching for multicast paths. In contrast to IP multicast, the proposed method is a controllable multicast scheme, which provides admission control in a multicast context, in-network adaptation, and supporting heterogeneous devices having different display capability. We further implement a prototype for illustrating the success of the proposed solution. The experimental results are also presented to show the effectiveness of the proposed equivalent bandwidth based adaptive enhancement layer switching algorithm.

Network function virtualization (NFV) introduces great flexibility in designing software-based network appliances to reduce cost and accelerate service deployment for network operators. However, with the fast development of high speed network of 100 GbE and beyond, how to efficiently design virtual network functions (VNF) on commodity servers has become a challenging problem. Although the advances in network hardware and software have facilitated the design of high speed network applications with hardware acceleration and kernel/driver optimization, how to leverage the existing techniques to design optimized high-performance VNFs still remains vague. In this study, we focus on the design and evaluation of four widely used VNFs covering the domains of network switching/routing, access control, measurement and security, by using a multicore platform supported by Intel DPDK fast packet I/O library. We describe the versatile network packet receiving and processing design options available for implementing such a programmable NFV platform for 100 Gbps network speed. With extensive experiments, we evaluate the performance of the each VNF and its design options in terms of packet drop rate, processing time per packet and delay per packet. Based on the evaluation over the collected data, we propose the optimal design with the given hardware resources to sustain the line rate while achieving the highest level of programmability.

Network function virtualization (NFV) allows software-oriented network functions executed on general-purpose servers or virtual machines (VMs) instead of dedicated hardware, greatly improving the flexibility and scalability of network services. Consequently, NFV would facilitate versatile and dynamic network measurement services to meet the increasingly diversified measurement demands. However, it is challenging to provision the measurement services in a virtualized environment due to the stochastic nature in measurement demand and the special requirements of measurement functions such as location constraint and execution time. In this paper, we compose a measurement service chain as a Virtual Measurement Function (VMF) graph, and then propose a dynamic VMF scheduling algorithm for a software-oriented measurement system using Lyapunov optimization technique to maximize the revenue of the system while guaranteeing the Quality of Service (QoS). The scheduling algorithm decides whether to accept a measurement service request and which measurement nodes (MNs) instantiate the VMF graph. Finally, the performance of our proposed algorithm is verified through theoretical analysis and numerical evaluation. The simulation results show that the proposed algorithm can increase the total avenue by up to 10%, reduce the service average queue delay by 24%, and decrease the service reject rate by up to 10%, comparing with a heuristic algorithm.

This paper solves the joint problem of admission control and routing for the video transmission in software-defined networking (SDN). We utilize next generation network called SDN technology to construct an architecture for the proposed combined optimization problem. Our heuristic algorithm of the proposed combined optimization problem can be deployed on this architecture. Based on this background, we formulate the combined optimization problem into Markov Decision Process (MDP) with the aim of maximizing the average reward. In allusion to the challenge of the curses of dimensionality, an online learning framework is designed by employing neuro-dynamic programming (NDP) method. We construct an emulation platform based on POX controller and Mininet to confirm high efficiency of our solution. Experiment results show that our NDP based algorithm has an observably performance improvement compared with OSPF based benchmark algorithm.

This paper considers the optimization problem of joint admission control and routing for the video streaming service in wired software-defined networking (SDN). With the aid of the network operating system, SDN is able to support the dynamic nature of future network functions and intelligent applications. Against this changing network landscape, we rely on FlowVisor-based virtualization in the context of OpenFlow-based wired SDN to design an open optimization architecture for the joint admission control and routing, which supports flexible and agile deployment of advanced joint admission control and routing strategies. Following this architecture, we interpret the joint admission control and routing problem into the Markov decision process for maximizing the overall “revenue.” In order to solve the issue of the curses of dimensionality, we invoke the function approximation technique in the context of approximate dynamic programming to conceive an online learning framework. By applying kernel-based autonomous feature extraction into the function approximation, we develop an approximate dynamic programming-based joint admission control and routing for video streaming service, which is apt to be implemented in the proposed open architecture. An emulation platform based on FlowVisor, POX, and Mininet is constructed for demonstrating the success of the proposed solution. The experimental results are presented to show the performance improvement of the proposed scheme by comparing it with the Q-learning algorithm and open shortest path first-based benchmark scheme.

In an IaaS cloud, virtual machines (VMs), also called instances, may be classified as reserved instances and on-demand instances. The reserved instances having long-term commitments and one-time payment are appropriate for the steady or predictable workloads, while for short-term, spiky or unpredictable workloads, the on-demand instances having flexible hourly payment and no long-term commitments may be more suitable for reducing the cost. In this paper, we consider the economical provisioning of reserved and/or on-demand instances for meeting time-varying computing workload of compute-intensive applications. In order to achieve this, we conceive a strategy for determining the amount of the purchased instances dynamically in order to minimize the total computing cost while keeping quality-of-service (QoS). By mapping QoS as the overload probability, we propose a dynamic instance provisioning strategy based on the large deviation principle, which is capable of calculating the minimum number of instances for the upcoming demands subject to the overload probability below a desired threshold. In addition, a reserved instance provisioning strategy for further reducing the total cost is also proposed by applying the autoregressive (AR) model to calculate the number of reserved instances for the average computation requirements. Finally, the simulations are performed based on real workload traces to show the attainable performance of the proposed instance provisioning strategy for the computing service in an IaaS cloud.

Scalable Video Coding (SVC) has been raised as a promising technique to enable flexible video transmission for mobile users with heterogeneous terminals and varying channel capacities. In this paper, we design an adaptive layer switching algorithm for on-demand scalable video service based on receiver's buffer underflow probability (BUP). Since the low quality of channel may lead to a low buffer fullness, the buffer fullness is an indicator for reflecting the channel condition and we define BUP for characterizing the mismatch between the video bitrate and the channel throughput. Accordingly, the adaptive SVC transmission problem is formulated as the adaptive adjustment of video layers based on BUP. This allows us to optimize the attainable video quality, while keeping BUP below a desired level. To estimate BUP, we derive an analytical model based on the large deviation principles. Then, an online layer switching algorithm is proposed using this estimation model, which is capable of accommodating different channel qualities without any prior knowledge of the channel variations and of the video characteristics. We further introduce a perturbation-based layer switching approach for reducing the quality fluctuating issue caused by frequent layer switches, thus improving the viewer's quality of experience. A system prototype is implemented to evaluate the success of the proposed method. We also conduct simulations in multiuser scenarios with real video traces and the results demonstrate that the proposed algorithm is capable of improving the playback experience, while keeping a low playback interruption rate and quality variation.

In this paper, we consider the adaptive transmission problem of scalable video in an energy harvesting communication system. The stochastic nature of the harvested energy puts a new challenge on the video transmission. Against this challenge, we formulate the adaptive scalable video transmission problem as maximizing the time average quality of the transmitted video subject to the energy constraint for reducing the playback interruption and the video quality smoothness constraint. In order to solve this problem, the Lyapunov optimization method is applied to derive an online dynamic layer transmission algorithm (DLTA). The simulation results show that the proposed DLTA can achieve better performance in terms of the received video quality and the convergence rate than a conventional reinforcement learning algorithm like the Q-learning method. It is also illustrated that the energy and smoothness constraints are beneficial for controlling the behavior of DLTA.

In video streaming service, multicast mode is a promising way to complement unicast delivery of content, since it deliveries the video content to a broad range of receivers. It is considered as an efficient scheme to reduce redundant traffic in the networks. In this paper, we propose an OpenFlow based architecture for implementing Scalable Video Coding (SVC) multicast streaming. It enables in-network identifying, processing and manipulating the media streams, which makes prompt bitrate adaptation possible in response to network fluctuations. The heterogeneous video quality demand from the heterogeneous device also can be satisfied by customizing the multicast traffic through a centralized OpenFlow controller. We implement a testbed following the proposed architecture in our campus. With OpenFlow, we deploy IP multicast in a new way without Internet Group Management Protocol (IGMP) or any multicast addresses. Experiments implemented on the testbed show that our approach can provide a flexible and controllable video multicast streaming service and improve the usage of bandwidth resource in the condition of guaranteeing multicast receivers' Quality of Experience (QoE).