Task migration and resource allocation are essential to integrate available resources for improving the efficiency of mobile-edge computing to support various computation-intensive and delay-sensitive Internet of Things applications, which cooperative optimization has yet been well addressed to achieve desirable performance. In this article, a deep reinforcement learning (DRL)-based adaptive cooperative optimization strategy is presented to fill this gap. Task migration and resource allocation are jointly employed to adaptively integrate available resources within cooperative edge nodes for processing various randomly offloaded tasks. The policy optimization to minimize the system cost and task dropout rates is formulated as Markov decision processes. A DRL algorithm that enhances deep deterministic policy gradient with a dual experience pool is proposed to jointly optimize the task migration and resource allocation in unknown stochastic application environments. Simulation experiments have been conducted to evaluate the performance of the presented strategy, and the results illustrate that it increases system rewards by 17.9%–61.5% and reduces task dropout rate by 5.2%–31.3% comparing with benchmarks.

Demand response (DR) plays an essential role in smart grids to reconfigure the load profile to match the energy supply for reliable and economic operation. This article presents an online adaptive DR strategy based on stochastic learning and optimization for smart microgrids with renewable energy to maximize the operation profit. To enable prompt interaction with the stochastic dynamics of usage demands, renewable generations and real-time pricing, a novel event-driven continuous-time Markov control processes model is introduced to formulate the DR optimization. Based on performance sensitivity analysis, a stochastic learning algorithm that combines potentials estimate and policy iteration is developed to find the optimal DR policy online. The presented strategy is environment-adaptive and computation-efficient, which is competent for real-time control and optimization in unknown environments. Simulations have been conducted to evaluate the performance of the presented strategy, and the results demonstrate the effectiveness.

Multicast is a promising solution to address spectrum scarcity in wireless video streaming, while facing tough challenge in provisioning quality of service (QoS) to heterogeneous users. This article presents an adaptive modulation and coding (AMC) scheme to accommodate both spatial diversity and temporal variation on users’ channel conditions for QoS provisioning in wireless scalable video multicast. The QoS-guaranteed AMC policy optimization is formulated as constrained stochastic optimization based on Markov decision processes. An adaptive policy iteration algorithm is developed to find the optimal AMC policy online without any prior knowledge of users’ channel statistics or intensive calculations. This algorithm is attractive due to adaptability to unknown environments and less run-time computation expenditure. Simulations results demonstrate the effectiveness of the proposed scheme.

Energy-efficient streaming of real-time scalable video over wireless fading channels with imperfect channel state information is considered in this paper. A traffic rate adaptation (TRA) scheme is presented to minimize the transmission energy consumption while satisfying the quality of experience (QoE) requirement by dynamically selecting the delivered encoding layers of streaming media depending on the channel condition. A partially observable Markov decision process model is introduced to formulate the energy-efficient QoE-guaranteed TRA problem. Based on the Lagrange method of multipliers, an online policy iteration algorithm is proposed to solve the constrained optimization problem. This algorithm performs policy evaluation and improvement without intensive computations and prior knowledge of channel statistics. The adaptability to unknown environments and high computation efficiency make the proposed scheme attractive for implementation in real-life mobile systems. Simulation results demonstrate the effectiveness of the proposed method.

In future mobile computing systems, cognitive radio (CR) emerges as a promising solution for alleviating spectrum shortage and satisfying the high bandwidth demand of multimedia streaming, while it presents tough challenges in provisioning user experience-perceived quality of service (QoS) and conserving transmission energy. In this paper, an adaptive rate control (ARC) scheme with the aid of the receive buffer is presented for energy-efficient transmission of streaming media over CR with QoS guarantee. The QoS metric, either display smoothness or transmission delay, is quantified by the state of the receive buffer. Cross-layer information is utilized to form a closed-loop feedback optimal control. A novel analytical model called event-driven discrete-time Markov control process is introduced to formulate the QoS-guaranteed energy-efficient ARC problem. Based on potential theory, a policy iteration algorithm that combines potentials estimation and stochastic approximation is proposed for finding the optimal policy online. By exploiting the system dynamics, this algorithm does not depend on any prior knowledge of channel availability or fading statistics, and it can converge to the global optimum with a low computational cost and reasonable speed. Simulation results demonstrate the effectiveness of the proposed method.