Due to the triple mobility of mobile users (MUs), unmanned aerial vehicle (UAV) relays, and low Earth orbit (LEO) satellites, handover becomes a critical and challenging issue for maintaining the continuity and quality of communication services in UAV-assisted LEO satellite networks. This paper proposes a distributed handover decision-making process aimed at improving scalability and reducing communication overhead. The handover problem is modeled as a decentralized Markov decision process (DEC-MDP) with the objective of maximizing the total end-to-end (E2E) throughput. We design an independent proximal policy optimization-based distributed intelligent handover (IPPO-DIH) algorithm within a centralized training with decentralized execution framework to solve the DEC-MDP. To analyze the theoretical optimal E2E throughput, we eliminate the correlation between handover decisions at different time steps. A three-sided matching algorithm with theoretical convergence guarantees is designed to obtain a stable matching among MUs, UAV relays, and LEO satellites at each time step. These stable matchings are combined to provide a theoretical performance benchmark for the handover algorithms. Simulation results validate the convergence of the proposed IPPO-DIH and three-sided matching algorithms. Additionally, the total E2E throughput achieved by the IPPO-DIH algorithm approaches the theoretical performance benchmark and outperforms typical handover algorithms.

To support the intelligent evolution of Internet of Things (IoT) systems toward enhanced comprehensiveness and sophistication, this paper proposes a distributed training and inference oriented toward continual learning. By integrating federated continual learning with inference offloading, the system addresses key challenges in IoT scenarios, including large-scale data processing, catastrophic forgetting during incremental updates, and resource constraints. A joint optimization framework of training-inference is established to analyze model accuracy, latency, and energy consumption. The optimization objective and strategy are formulated as a Markov Decision Process (MDP) with a hybrid action space and customized reward mechanism. The Soft Actor-Critic (SAC) algorithm enables action grouping and the transformation between discrete and continuous actions, achieving unified optimization of training and inference. Simulation results show that compared to existing approaches, the proposed method improves node selection, resource allocation, and offloading strategy by jointly considering communication and computation costs.

o address the critical need for stable communication and energy efficiency in dynamic unmanned aerial vehicle (UAV) scenarios,o address the critical need for stable communication and energy efficiency in dynamic unmanned aerial vehicle (UAV) scenarios,T a pinching-antenna system (PASS)-enabled UAV delivery framework is proposed, which exploits the capability of PASS to establish a strong line-of-sight link and reduce the free-space pathloss. Aiming at achieving a balance between communication performance and energy efficiency, we define an effective utility function, construct a utility maximization problem, and develop an iterative joint optimization algorithm for pinching antenna (PA) activation vector and UAV delivery sequencing (IJO-PADS). More specifically, to solve the highly coupled mixed-integer nonlinear programming problem of PA activation vector optimization, we propose a pair of algorithms: 1) Branch-and-Bound (BnB) algorithm for finding global optimum; 2) incremental search and local refinement (ISLR) algorithm for reducing computational complexity. With the optimized PA activation vector, we define the path weight between a pair of nodes, which accounts for communication rate reward and energy consumption penalty. To maximize sum pate weight, we propose a genetic algorithm and dynamic programming (GA-DP) hybrid optimization method to tackle the NP-hard problem of delivery sequence planning, where a GA performs global exploration to generate candidate solutions, while a DP performs local refinement to obtain elite solutions. Simulation results indicate that: i) the proposed IJO-PADS framework converges within a moderate number of iterations; ii) the proposed algorithms (BnB, ISLR, GA-DP) outperform several benchmarks, demonstrating the effectiveness of our designs for PA activation and delivery sequence planning; iii) PASS is superior to conventional MIMO systems, due to PASS’s flexible PA activation and low-attenuation waveguide transmission.

To facilitate low-latency and secure transmission in Internet of Everything, a secure short-packet transmission framework is proposed in an uplink non-orthogonal multiple access (NOMA)-based integrated sensing and communication (ISAC) system. A triple-functional base station is utilized, which simultaneously carries out the tasks of receiving short-packet messages, detecting potential eavesdroppers, and transmitting active jamming signals. The achievable secure short-packet transmission rate is developed to measure the security performance, where the practical cases of imperfect inter-functional and inter-device interference elimination are considered. To optimize the security performance by effectively leveraging sensing capabilities, a problem is formulated aiming at maximizing the secure short-packet transmission rate, while guaranteeing the sensing quality. A key challenge to solve this problem lies in the performance loss terms associated with decoding error and information leakage in short-packet transmission, which makes the optimization problem highly coupled and strictly non-convex. To tackle this challenge, an alternating optimization (AO)-based approach is devised to solve the formulated problem iteratively, where an approximation method is proposed to convert the secure short-packet transmission rate into a manageable form. The convergence and effectiveness of the proposed design are validated by simulation results, which reveal that i) the devised AO-based algorithm converges within a modest number of iteration times; ii) the proposed NOMA-ISAC-based security design outperforms other benchmark schemes, demonstrating the benefit of utilizing sensing capabilities to enhance security.

The management of computing resources through the computing power network (CPN) has gradually become a focal point of research. With the development of the 6th generation (6G) mobile networks, some promising technologies such as satellite-terrestrial integrated network (STIN) and smart endogenous network driven by artificial intelligence (AI) are increasingly being applied in Industrial Internet of Things (IIoT). However, several issues in current studies are worthy of attention: 1) the large number of devices powered by battery in IIoT, 2) the complex environments of communication, 3) the finite computing resources for task data processing. To cope with these challenges, a satellite-terrestrial integrated computing power network (STICPN) framework is introduced in this article. Within this framework, a task offloading link selection scheme is proposed, which minimizes the delay and the consumption of energy. The task offloading optimization problem is modeled as a Markov decision process (MDP). Meanwhile, deep reinforcement learning (DRL) algorithm is employed to adapt to the dynamic states of environment. Specifically, a dueling double deep Q network (D3QN) is used to make optimal decisions and delay as well as energy consumption can be reduced significantly. Moreover, the D3QN-based scheme extends the usage time of IIoT devices. The simulation results indicate that the proposed scheme outperforms comparison schemes significantly.

The management of computing resources through the computing power network (CPN) has gradually become a focal point of research. With the development of the 6th generation (6G) mobile networks, some promising technologies, such as satellite-terrestrial integrated network (STIN) and smart endogenous network driven by artificial intelligence (AI) are increasingly being applied in Industrial Internet of Things (IIoT). However, several issues in current studies are worthy of attention: 1) the large number of devices powered by battery in IIoT; 2) the complex communication environments; and 3) the finite computing resources for task data processing. To cope with these challenges, a satellite-terrestrial integrated CPN (STICPN) framework is introduced in this article. Within this framework, a task offloading link selection scheme is proposed, which minimizes the delay and the consumption of energy. The task offloading optimization problem is modeled as a markov decision process (MDP). Meanwhile, deep reinforcement learning (DRL) algorithm is employed to adapt to the dynamic states of environment. Specifically, a Dueling Double Deep Q Network (D3QN) is used to make optimal decisions and delay as well as energy consumption can be reduced significantly. Moreover, the D3QN-based scheme extends the usage time of IIoT devices. The simulation results indicate that the proposed scheme outperforms the comparison schemes significantly.

With the advancement of urbanization, communication-based train control (CBTC) systems for urban rail transit and train-to-train (T2T) communication have garnered significant attention. T2T communication establishes mobile ad hoc networks (MANETs), similar to those in vehicular ad hoc networks (VANETs). Building upon this foundation, we propose an adaptive cooperative (ADCO) MAC protocol for T2T communication. The scheme introduces clustering and cooperative transmission mechanisms, which enhance the efficiency and reliability of safety packet transmission. Additionally, the protocol assigns different priorities to packets engaging in contention on the control channel (CCH) and enables trains to access service channels (SCHs) without contention through pre-reserved time slots. To analyze the transmission probabilities and success rates of packets with varying priorities, a Markov-based model is utilized, ultimately determining the optimal ratio of the CCH interval (CCHI) to the SCH interval (SCHI) for maximizing channel utilization. Theoretical analysis and simulation results demonstrate that the proposed MAC protocol ensures reliable transmission of safety packets while simultaneously optimizing the throughput on SCHs.

Unmanned aerial vehicle (UAV) communications have become a prominent technology for emergency communications to enhance network services. This article investigates computing offloading and resource allocation in nonorthogonal multiple access (NOMA)-based UAV emergency communication scenarios. To minimize the computational overhead of the terminal device, a joint task offloading and resource allocation problem is investigated, where the computation overhead of the marine Internet of Things (IoT) device is measured as a weighting of the task completion time and the energy consumption of the device. The optimization of the transmission of IoT devices, the allocation of computing resources to UAVs, task offloading, and carrier allocation are formulated in the considered problem, which is an NP-hard mixed integer nonlinear programming problem. To reduce the complexity, we decompose it into two parts from the property of the problem: 1) the resource optimization problem and 2) the task offloading problem. To solve the resource allocation problem, we first decouple the problem and then use the proposed quasi-convex and convex optimization methods. Meanwhile, a low-complexity task offloading algorithm is designed to achieve a Nash-stable solution by introducing a coalition game approach based on this. Numerical results verify the algorithm’s effectiveness and are compared with other schemes in the literature.

Federated learning, leveraging distributed data from multiple nodes to train a common model, allows for the use of more data to improve the model while also protecting the privacy of original data. However, challenges still exist in ensuring privacy and security within the interactions. To address these issues, this paper proposes a federated learning approach that incorporates blockchain, homomorphic encryption, and reputation. Using homomorphic encryption, edge nodes possessing local data can complete the training of ciphertext models, with their contributions to the aggregation being evaluated by a reputation mechanism. Both models and reputations are documented and verified on the blockchain through consensus process, which then determines the rewards based on the incentive mechanism. This approach not only incentivizes participation in training, but also ensures the privacy of data and models through encryption. Additionally, it addresses security risks associated with both data and network attacks, ultimately leading to a highly accurate trained model. To enhance the efficiency of learning and the performance of the model, a joint adaptive aggregation and resource optimization algorithm is introduced. Finally, simulations and analyses demonstrate that the proposed scheme enhances learning accuracy while maintaining privacy and security.

The emergence of communication-based train control (CBTC) system within urban rail transport has improved the efficiency of safe train operations. At the same time, the CBTC system enhances the reliability of the train system and lowers latency. Nevertheless, there are still certain critical issues that need to be considered in the CBTC: 1) limited coverage and high maintenance costs of wayside equipment; 2) multiple ground devices configuration and complex system architecture; and 3) insufficient computing capacity of the train leads to heavy latency and energy consumption. The multi-hop ad hoc network coexisting with train-to-train communication and train-to-wayside communication is applied to simplify the networking architecture, together with the employment of mobile edge computing (MEC) servers to provide massive computing and communication resources for trains. Therefore, in this paper, a new multi-hop ad hoc network and MEC-assisted CBTC framework are developed for computing offloading and resource allocation. Offloading decisions, offloading ratio, computing and communication resource allocation are integrated to minimize latency and energy consumption. Furthermore, the proposed problem is a mixed-integer non-convex problem that is transformed into a solvable convex problem, and the consensus alternating direction method of multipliers-based (ADMM) algorithm is employed to solve the problem. The simulation results show that our proposed method has remarkable advantages over other schemes.

Driven by numerous emerging mobile devices and various quality of service requirements, mobile edge computing (MEC) has been recognized as a prospective paradigm to promote the computation capability of mobile devices, as well as reduce energy overhead and service latency of applications for the Internet of Things (IoT). However, there are still some open issues in the existing research works: 1) limited network and computing resource, 2) simple or non-intelligent resource management, 3) ignored security and reliability. In order to cope with these issues, in this article, 6G and blockchain technology are considered to improve network performance and ensure the authenticity of data sharing for the MEC-enabled IoT. Meanwhile, a novel intelligent optimization method named as collective reinforcement learning (CRL) is proposed and introduced, to realize intelligent resource allocation, meet distributed training results sharing and avoid excessive consumption of system resources. Based on the designed network model, a cloud-edge collaborative resource allocation framework is formulated. By joint optimizing the offloading decision, block interval and transmission power, it aims to minimize the consumption overheads of system energy and service latency. Then the formulated problem is designed as a Markov decision process, and the optimal strategy can be obtained by the CRL. Some evaluation results reveal that the system performance based on the proposed scheme outperforms other existing schemes obviously.

Immutability, decentralization, and linear promoted scalability make sharded blockchain a promising solution, which can effectively address the trust issue in the large-scale Internet of Things (IoT). However, currently, the throughput of sharded blockchains is still limited when it comes to high proportions of cross-shard transactions (CST). On the other hand, assemblage characteristics of collaborative computing in IoT have not been received attention. Therefore, in this paper, we present a clustering-based sharded blockchain strategy for collaborative computing in the IoT, where the sharding of the blockchain system is implemented in two steps: k-means clustering-based user grouping and the assignment of consensus nodes. In this framework, how to reasonably group the IoT users while simultaneously guaranteeing the system performance is the key point. Specifically, we describe the data transactions among IoT devices by data transaction flow graph (DTFG) based on a dynamic stochastic block model. Then, formed as a Markov decision process (MDP), the optimization of the cluster number (shard number) and the adjustment of consensus parameters are jointly trained by deep reinforcement learning (DRL). Simulation results show that the proposed scheme improves the scalability of the sharded blockchain in the IoT application.

The explosion of intelligent mobile applications has generated an unprecedented increase in the demand for diverse social content access. Device-to-device (D2D) communication, which enables direct transmission of content, can partly support these needs by reusing cellular spectrum resources. Most existing works focus on mitigating interference in D2D communication only in physical space. However, the effect of user content preference on interference in social space is ignored. In this article, a satisfied matching-embedded Social Internet of Things (IoT) is proposed for content preference-aware resource allocation (SIoT-RA). The Social IoT architecture is explored for user social characteristic analysis scenarios to ensure credible data management. Specifically, the user’s content preference characteristics are modeled as the probability of selecting similar content in the social IoT by employing the Dirichlet process. A satisfied matching-based model is proposed and embedded into the architecture as the core algorithm to achieve content preference-aware resource allocation. This design not only helps to reduce interference from the perspective of social space but also realizes maximized and satisfactory resource allocation performance by a matching algorithm that can quantify satisfaction as perceived utility. Extensive simulation results demonstrate the significant performance gains and high cost effectiveness of the proposed core algorithm of SIoT-RA in terms of the weighted sum data rate and users matching satisfaction.

Driven by numerous emerging mobile devices and various Quality-of-Service (QoS) requirements, mobile-edge computing (MEC) has been recognized as a prospective paradigm to promote the computation capability of mobile devices, as well as reduce energy overhead and service latency of applications for the Internet of Things (IoT). However, there are still some open issues in the existing research works: 1) limited network and computing resource; 2) simple or nonintelligent resource management; and 3) ignored security and reliability. In order to cope with these issues, in this article, 6G and blockchain technology are considered to improve network performance and ensure the authenticity of data sharing for the MEC-enabled IoT. Meanwhile, a novel intelligent optimization method named as collective reinforcement learning (CRL) is proposed and introduced, to realize intelligent resource allocation, meet distributed training results sharing, and avoid excessive consumption of system resources. Based on the designed network model, a cloud–edge collaborative resource allocation framework is formulated. By joint optimizing the offloading decision, block interval, and transmission power, it aims to minimize the consumption overheads of system energy and latency. Then, the formulated problem is designed as a Markov decision process, and the optimal strategy can be obtained by the CRL. Some evaluation results reveal that the system performance based on the proposed scheme outperforms other existing schemes obviously.

Immutability, decentralization, and linear promoted scalability make the sharded blockchain a promising solution, which can effectively address the trust issue in the large-scale Internet of Things (IoT). However, currently, the throughput of sharded blockchains is still limited when it comes to high proportion of cross-shard transactions (CSTs). On the other hand, the assemblage characteristic of the collaborative computing in IoT has not been received attention. Therefore, in this article, we present a clustering-based sharded blockchain strategy for collaborative computing in the IoT, where the sharding of the blockchain system is implemented in two steps:K-means-clustering-based user grouping and the assignment of consensus nodes. In this framework, how to reasonably group the IoT users while simultaneously guaranteeing the system performance is the key point. Specifically, we describe the data transactions among IoT devices by data transaction flow graph (DTFG) based on a dynamic stochastic block model. Then, formed as a Markov decision process (MDP), the optimization of the cluster number (shard number) and the adjustment of consensus parameters are jointly trained by deep reinforcement learning (DRL). Simulation results show that the proposed scheme improves the scalability of the sharded blockchain in the IoT application.

Nowadays, the rise of the Internet of Vehicles (IoV) has led to the rapid development of smart transportation. To increase the computing capacity of mobile vehicles and decrease the content delivery latency of suppliers, mobile edge computing (MEC) is considered as an indispensable solution. However, there are some essential issues to be considered: 1) security and privacy of data transmission, and 2) reasonable resource allocation for collaborative computing and caching. In this paper, to solve above issues, blockchain technology is adopted to ensure reliable transmission and interaction of data. Meanwhile, we develop an intelligent resource framework about computing and caching for blockchain-enabled MEC systems in IoV. Through jointly considering and optimizing offloading decision of computation task carried by vehicle, caching decision, the number of offloaded consensus nodes, block interval and block size, the energy consumption and computation overheads can be decreased, and the data throughput of the blockchain can be increased significantly. Moreover, the proposed optimization problem is modeled and formulated as a Markov decision process. Facing the complexity and dynamic of resource allocation, the asynchronous advantage actor-critic approach is considered and applied to solve the optimization problem. Experiment results demonstrate that the advantages of the proposed optimization scheme are obvious compared with other existing schemes.

Recently, electric vehicles (EVs) have been widely used under the call of green travel and environmental protection, and diverse requirements for charging are also increasing gradually. In order to ensure the authenticity and privacy of charging information interaction, blockchain technology is proposed and applied in charging station billing systems. However, there are some issues in blockchain itself, including lower computing efficiency of the nodes and higher energy consumption in the consensus process. To handle the above issues, in this paper, combining blockchain and mobile edge computing, we develop a reliable billing data transmission scheme to improve the computing capacity of nodes and reduce the energy consumption of the consensus process. By jointly optimizing the primary and replica nodes offloading decisions, block size and block interval, the transaction throughput of the blockchain system is maximized, as well as the consumption costs of latency and energy consumption is minimized. Moreover, we formulate the joint optimization problem as Markov decision process (MDP). To tackle this dynamic and continuity of the system state, the actor–critic reinforcement learning is introduced to solve the MDP problem. Finally, simulation results demonstrate that the performance improvement of the proposed scheme through comparison with other existing schemes.

Industrial Internet of Things (IIoT) has emerged with the developments of various communication technologies. In order to guarantee the security and privacy of massive IIoT data, blockchain is widely considered as a promising technology and applied into IIoT. However, there are still several issues in the existing blockchain-enabled IIoT: 1) unbearable energy consumption for computation tasks; 2) poor efficiency of consensus mechanism in blockchain; and 3) serious computation overhead of network systems. To handle the above issues and challenges, in this article, we integrate mobile-edge computing (MEC) into blockchain-enabled IIoT systems to promote the computation capability of IIoT devices and improve the efficiency of the consensus process. Meanwhile, the weighted system cost, including the energy consumption and the computation overhead, are jointly considered. Moreover, we propose an optimization framework for blockchain-enabled IIoT systems to decrease consumption, and formulate the proposed problem as a Markov decision process (MDP). The master controller, offloading decision, block size, and computing server can be dynamically selected and adjusted to optimize the devices energy allocation and reduce the weighted system cost. Accordingly, due to the high-dynamic and large-dimensional characteristics, deep reinforcement learning (DRL) is introduced to solve the formulated problem. Simulation results demonstrate that our proposed scheme can improve system performance significantly compared to other existing schemes.

Recently, the development of the Internet of Things (IoT) provides plenty of opportunities and challenges in various fields. As an essential part of IoT, machine-to-machine (M2M) communications open a novel way that the machine-type communication devices (MTCDs) are connected and communicated without any human intervention. Meanwhile, delay-tolerant data play an important role in M2M communications-based IoT, and it puts more emphasis on powerful data caching, computing, and processing, as well as the security and stability of data transmission. To meet these requirements in M2M communications networks, in this article, we introduce some promising technologies, such as edge computing and blockchain, and propose a joint optimization framework about caching, computation, and security for delay-tolerant data in M2M communications networks based on dueling deep Q-network (DQN). According to the dynamic decision process by DQN, the optimal selection and decision of caching servers, computing servers, and blockchain systems can be made to achieve maximum system rewards, which includes higher efficiency of data processing, lower network costs, and better security of data interaction. Extensive simulation results with different system parameters show that our proposed framework can effectively improve the system performance for blockchain-enabled M2M communications compared to the existing schemes.

With the rapid developments of wireless communication and increasing number of connected vehicles, Vehicular Ad Hoc Networks (VANETs) enable cyberinteractions in the physical transportation system. Future networks require real-time control capability to support delay-sensitive application such as connected autonomous vehicles. In recent years, fog computing becomes an emerging technology to deal with the insufficiency in traditional cloud computing. In this paper, a fog-based distributed network control design is proposed toward connected and automated vehicle application. The proposed architecture combines VANETs with the new fog paradigm to enhance the connectivity and collaboration among distributed vehicles. A case study of connected cruise control (CCC) is introduced to demonstrate the efficiency of the proposed architecture and control design. Finally, we discuss some future research directions and open issues to be addressed.

As an emerging type of Internet of Things (IoT), Internet of Vehicles (IoV) denotes the vehicle network capable of supporting diverse types of intelligent services and has attracted great attention in the 5G era. In this study, we consider the multimedia content caching with multicast beamforming in IoV-based vehicular edge networks. First, we formulate a joint vehicle-to-vehicle- (V2V-) assisted clustering, caching, and multicasting optimization problem, to minimize the weighted sum of flow cost and power cost, subject to the quality-of-service (QoS) constraints for each multicast group. Then, with the two-timescale setup, the intractable and stochastic original problem is decoupled at separate timescales. More precisely, at the large timescale, we leverage the sample average approximation (SAA) technique to solve the joint V2V-assisted clustering and caching problem and then demonstrate the equivalence of optimal solutions between the original problem and its relaxed linear programming (LP) counterpart; and at the small timescale, we leverage the successive convex approximation (SCA) method to solve the nonconvex multicast beamforming problem, whereby a series of convex subproblems can be acquired, with the convergence also assured. Finally, simulations are conducted with different system parameters to show the effectiveness of the proposed algorithm, revealing that the network performance can benefit from not only the power saving from wireless multicast beamforming in vehicular networks but also the content caching among vehicles.

Recent advances in Internet of Things (IoT) provide plenty of opportunities for various areas. Nevertheless, the machine-to-machine (M2M) communications-based IoT develops rapidly but suffers from extra energy consumption, large data transmission latency as well as overmuch network cost, because various of machine-type communication devices (MTCDs) are deployed in the network. To meet the requirements of energy efficient M2M communications, in this paper, we introduce a promising technology named as mobile edge computing (MEC), and propose a performance optimization framework with MEC for M2M communications network based on deep reinforcement learning (DRL). According to dynamic decision process by DRL, the appropriate access networks and the computing servers can be determined and selected with the minimum system cost, which includes lower network cost, time cost and energy consumption for data transmission and computing tasks execution. Extensive simulation results with different system parameters show that our proposed framework can effectively improve the system performance for M2M communications compared to the existing schemes.

In this paper, an optimal control algorithm for connected vehicle systems is proposed in order to ensure the vehicular platoon stable as well as reduce the transmission power consumptions in the presence of the network-induced delays. First, the vehicle dynamic modeling and power consumption analysis are addressed based on a typical 3-vehicle platoon. With the objective of minimizing the deviations of vehicle's headway and velocity as well as reducing power consumption, an optimization problem is formulated using a quadratic cost function. Then, the design of the optimal control strategy with minimum power consumption for connected vehicle systems can be divided into two steps: first the minimal hop routings for human- driven vehicles are obtained based on the network topology, and then the optimal control strategy is derived based on the determined transmission routing.

Wireless sensor and actuator networks (WSANs), as the promising technologies to realize efficient and energy-saving control, recently have been one of the main research focuses in academic fields as well as in control applications. In this paper, considering the network-induced delays, the joint design of optimal control strategy and power consumption for WSANs is addressed. First, the WSAN system with multiple transmission paths is modeled as a linear system and the joint optimization problem is formulated by using a quadratic cost function. Then, a two-step control scheme is presented to realize the joint design of control law and power consumption. In particular, the optimal control strategy for each given path is iteratively derived, and then the optimal transmission path is selected with the minimum system cost. Finally, numerical simulations in both generic control systems and power grid systems demonstrate the effectiveness of the proposed control scheme.

In most cases, the batteries of sensor nodes in the Internet of Things (IoT) are usually constrained by size and weight, and are difficult to recharge or replace. In traditional wireless sensor networks, data is transmitted in a multi-hop manner, which may cause the high data transmission delay and unbalanced traffic load. In this paper, an Unmanned Aerial Vehicle (UAV)-assisted IoT architecture is introduced, in which UAV is utilized to achieve low-latency and seamless-coverage acquisition of the sensing data. Furthermore, based on the recent advances on deep reinforcement learning algorithms, considering both data delay requirements and network energy consumption, a real-time flight path planning scheme of the UAV in the dynamic IoT sensor networks has been proposed based on dueling deep Q-network (DQN). Besides, the grid-based method is used to handle the network state modeling, which effectively reduces the complexity of the proposed scheme. Simulation results show that the proposed scheme significantly improves the network performance.

As a promising technique, mobile edge computing (MEC) has attracted significant attention from both academia and industry. However, the offloading decision for computing tasks in MEC is usually complicated and intractable. In this paper, we propose a novel framework for offloading decision in MEC based on Deep Reinforcement Learning (DRL). We consider a typical network architecture with one MEC server and one mobile user, in which the tasks of the device arrive as a flow in time. We model the offloading decision process of the task flow as a Markov Decision Process (MDP). The optimization object is minimizing the weighted sum of offloading latency and power consumption, which is decomposed into the reward of each time slot. The elements of DRL such as policy, reward and value are defined according to the proposed optimization problem. Simulation results reveal that the proposed method could significantly reduce the energy consumption and latency compared to the existing schemes.

With an increasing number of machine-type communication devices (MTCDs), machine-to-machine (M2M) communications have attracted great attentions from both academia and industry. Different from traditional communication networks, the data connections with M2M communications are typically small-sized but with high frequency, necessitating the efficiency optimization of both energy consumption and computation. In this paper, we introduce mobile edge computing (MEC) into virtualized cellular networks with M2M communications, to decrease the energy consumption and optimize the computing resource allocation as well as improve computing capability. Moreover, based on different functions and quality of service (QoS) requirements, the physical network can be virtualized into several virtual networks, and then each MTCD selects the corresponding virtual network to access through the embedded-SIM (eSIM) technology. Meanwhile, the random access process of MTCDs is formulated as a partially observable Markov decision process (POMDP) to minimize the system cost, which consists of both the energy consumption and execution time of computing tasks. Furthermore, to facilitate the network architecture integration, software-defined networking (SDN) is introduced to deal with the diverse protocols and standards in the networks. Extensive simulation results with different system parameters reveal that the proposed scheme could significantly improve the system performance compared to the existing schemes.

With the explosion in the number of connected devices and Internet of Things (IoT) services in smart city, the challenges to meet the demands from both data traffic delivery and information processing are increasingly prominent. Meanwhile, the connected vehicle networks have become an essential part in smart city, bringing massive data traffic as well as significant networking, caching and computing resources. In this paper, we propose a novel vehicle network architecture, mitigating the network congestion with the joint optimization of networking, caching and computing. Cloud computing at the data centers as well as mobile edge computing (MEC) at the evolved node Bs (eNodeBs) and on-board units (OBUs) are taken as the paradigms to provide caching and computing resources. The programmable control principle originated from software-defined networking (SDN) paradigm has been introduced to facilitate the system architecture and resource integration. With the careful modeling of the services, the vehicle mobility and the system state, a joint resource management scheme is proposed and formulated as a partially observable Markov decision process (POMDP) to minimize system cost, which consists of both network overhead and execution time of computing tasks. Extensive simulation results with different system parameters reveal that the proposed scheme could significantly improve the system performance compared to the existing schemes.

As an important part of the Internet-of-Things (IoT), machine-to-machine (M2M) communications have attracted great attention. In this paper, we introduce mobile edge computing (MEC) into virtualized cellular networks with M2M communications, to decrease the energy consumption and optimize the computing resource allocation as well as improve computing capability. Moreover, based on different functions and quality of service (QoS) requirements, the physical network can be virtualized into several virtual networks, and then each MTCD selects the corresponding virtual network to access. Meanwhile, the random access process of MTCDs is formulated as a partially observable Markov decision process (POMDP) to minimize the system cost, which consists of both the energy consumption and execution time of computing tasks. Furthermore, to facilitate the network architecture integration, software-defined networking (SDN) is introduced to deal with the diverse protocols and standards in the networks. Extensive simulation results with different system parameters reveal that the proposed scheme could significantly improve the system performance compared to the existing schemes.

Machine-to-machine (M2M) communications have attracted great attention from both academia and industry. In this paper, with recent advances in wireless network virtualization and software- defined networking (SDN), we propose a novel framework for M2M communications in software- defined cellular networks with wireless network virtualization. In the proposed framework, according to different functions and quality of service (QoS) requirements of machine-type communication devices (MTCDs), a hypervisor enables the virtualization of the physical M2M network, which is abstracted and sliced into multiple virtual M2M networks. Moreover, we formulate a decision-theoretic approach to optimize the random access process of M2M communications. In addition, we develop a feedback and control loop to dynamically adjust the number of resource blocks (RBs) that are used in the random access phase in a virtual M2M network by the SDN controller. Extensive simulation results with different system parameters are presented to show the performance of the proposed scheme.