Due to the triple mobility of mobile users (MUs), unmanned aerial vehicle (UAV) relays, and low Earth orbit (LEO) satellites, handover becomes a critical and challenging issue for maintaining the continuity and quality of communication services in UAV-assisted LEO satellite networks. This paper proposes a distributed handover decision-making process aimed at improving scalability and reducing communication overhead. The handover problem is modeled as a decentralized Markov decision process (DEC-MDP) with the objective of maximizing the total end-to-end (E2E) throughput. We design an independent proximal policy optimization-based distributed intelligent handover (IPPO-DIH) algorithm within a centralized training with decentralized execution framework to solve the DEC-MDP. To analyze the theoretical optimal E2E throughput, we eliminate the correlation between handover decisions at different time steps. A three-sided matching algorithm with theoretical convergence guarantees is designed to obtain a stable matching among MUs, UAV relays, and LEO satellites at each time step. These stable matchings are combined to provide a theoretical performance benchmark for the handover algorithms. Simulation results validate the convergence of the proposed IPPO-DIH and three-sided matching algorithms. Additionally, the total E2E throughput achieved by the IPPO-DIH algorithm approaches the theoretical performance benchmark and outperforms typical handover algorithms.

A novel non-orthogonal multiple access (NOMA) based low-delay service framework is proposed for fog radio access networks (F-RANs). Fog access points (FAPs) leverage NOMA for local delivery of cached content, while the cloud access point employs NOMA to simultaneously push content to FAPs and directly serve users. Based on this model, a delay minimization problem is formulated by jointly optimizing user association, cache placement, and power allocation. To address this non-convex mixed-integer nonlinear programming problem, an alternating optimization (AO) algorithm is developed, which decomposes the original problem into two subproblems, namely joint user association and cache placement, and power allocation. In particular, a low-complexity algorithm is designed to optimizing the user association and cache placement strategy using the McCormick envelope theory and Lagrangian partial relaxation. The power allocation is optimized by invoking the successive convex approximation. Simulation results reveal that: 1) the proposed AO-based algorithm effectively balances between the achieved performance and computational efficiency, and 2) the proposed NOMA-based F-RANs framework significantly outperforms orthogonal multiple access-based F-RANs systems in terms of average transmission delay in different scenarios.

The management of computing resources through the computing power network (CPN) has gradually become a focal point of research. With the development of the 6th generation (6G) mobile networks, some promising technologies, such as satellite-terrestrial integrated network (STIN) and smart endogenous network driven by artificial intelligence (AI) are increasingly being applied in Industrial Internet of Things (IIoT). However, several issues in current studies are worthy of attention: 1) the large number of devices powered by battery in IIoT; 2) the complex communication environments; and 3) the finite computing resources for task data processing. To cope with these challenges, a satellite-terrestrial integrated CPN (STICPN) framework is introduced in this article. Within this framework, a task offloading link selection scheme is proposed, which minimizes the delay and the consumption of energy. The task offloading optimization problem is modeled as a markov decision process (MDP). Meanwhile, deep reinforcement learning (DRL) algorithm is employed to adapt to the dynamic states of environment. Specifically, a Dueling Double Deep Q Network (D3QN) is used to make optimal decisions and delay as well as energy consumption can be reduced significantly. Moreover, the D3QN-based scheme extends the usage time of IIoT devices. The simulation results indicate that the proposed scheme outperforms the comparison schemes significantly.

With the advancement of urbanization, communication-based train control (CBTC) systems for urban rail transit and train-to-train (T2T) communication have garnered significant attention. T2T communication establishes mobile ad hoc networks (MANETs), similar to those in vehicular ad hoc networks (VANETs). Building upon this foundation, we propose an adaptive cooperative (ADCO) MAC protocol for T2T communication. The scheme introduces clustering and cooperative transmission mechanisms, which enhance the efficiency and reliability of safety packet transmission. Additionally, the protocol assigns different priorities to packets engaging in contention on the control channel (CCH) and enables trains to access service channels (SCHs) without contention through pre-reserved time slots. To analyze the transmission probabilities and success rates of packets with varying priorities, a Markov-based model is utilized, ultimately determining the optimal ratio of the CCH interval (CCHI) to the SCH interval (SCHI) for maximizing channel utilization. Theoretical analysis and simulation results demonstrate that the proposed MAC protocol ensures reliable transmission of safety packets while simultaneously optimizing the throughput on SCHs.

With the accelerated development of autonomous driving and large language model, blockchain-supported data interaction and artificial intelligence (AI)-assisted performance optimization is the current mainstream research in the Internet of Vehicles (IoV). However, the trial-and-error behavior of the AI algorithm during the training process is a threat to road safety. Therefore, this paper proposes a general constrained policy optimization (CPO)-based modeling and solution for highdimensional constrained optimization problems. We focus on intelligent driving information sharing in blockchain-enhanced IoV and optimize the service rewards in the sharing requests scheduling problem while ensuring the frequency resource limitation, service quality constraint, and road safety constraint. The constrained state space (CSS) is innovatively proposed to abstract the environment mathematically with the definition of constraint hyperplanes and distance. Accordingly, the constrained Markov Decision process (CMDP) and the optimization problem are formulated. With the practical implementation of the CPO theory, the constrained sharing requests scheduling (CSRS) algorithm is proposed. Ablation experiments are deep reinforcement learning-based methods without using the CSS-based constraint modeling or without using the CPO-based constrained problem solving process. Results show the effectiveness of CSS and CSRS algorithm in improving the policy training efficiency, and the testing results shows excellent generalization ability.

Efficient data processing is crucial for industrial Internet of Things (IIoT) applications, but the limited energy and computing resources in IIoT devices (IIoT-Ds) pose constraints. This article utilizes a unmanned aerial vehicle (UAV) as a computing server for enhanced IIoT mission execution. Specifically, the energy consumption of IIoT-Ds and the UAV, as well as the weighted cost of the communication and computing scheduling strategy in the UAV-aided IIoT, are jointly taken into account. An optimization problem based on the system energy consumption is built under the constraints of UAV motion, computing offloading, and transmitting power allocation. A problem decoupling-based alternating optimization method is proposed to solve the minimization problem by decomposing it into three subproblems: 1) UAV motion optimization; 2) computing offloading configuration; and 3) transmitting power allocation. Through comparing the proposed communication and computing scheduling strategy with existing methods, simulation results illustrate its attainment of quasi-optimal performance, thereby validating the effectiveness of the alternating optimization method.

The emergence of communication-based train control (CBTC) system within urban rail transport has improved the efficiency of safe train operations. At the same time, the CBTC system enhances the reliability of the train system and lowers latency. Nevertheless, there are still certain critical issues that need to be considered in the CBTC: 1) limited coverage and high maintenance costs of wayside equipment; 2) multiple ground devices configuration and complex system architecture; and 3) insufficient computing capacity of the train leads to heavy latency and energy consumption. The multi-hop ad hoc network coexisting with train-to-train communication and train-to-wayside communication is applied to simplify the networking architecture, together with the employment of mobile edge computing (MEC) servers to provide massive computing and communication resources for trains. Therefore, in this paper, a new multi-hop ad hoc network and MEC-assisted CBTC framework are developed for computing offloading and resource allocation. Offloading decisions, offloading ratio, computing and communication resource allocation are integrated to minimize latency and energy consumption. Furthermore, the proposed problem is a mixed-integer non-convex problem that is transformed into a solvable convex problem, and the consensus alternating direction method of multipliers-based (ADMM) algorithm is employed to solve the problem. The simulation results show that our proposed method has remarkable advantages over other schemes.

Driven by numerous emerging mobile devices and various quality of service requirements, mobile edge computing (MEC) has been recognized as a prospective paradigm to promote the computation capability of mobile devices, as well as reduce energy overhead and service latency of applications for the Internet of Things (IoT). However, there are still some open issues in the existing research works: 1) limited network and computing resource, 2) simple or non-intelligent resource management, 3) ignored security and reliability. In order to cope with these issues, in this article, 6G and blockchain technology are considered to improve network performance and ensure the authenticity of data sharing for the MEC-enabled IoT. Meanwhile, a novel intelligent optimization method named as collective reinforcement learning (CRL) is proposed and introduced, to realize intelligent resource allocation, meet distributed training results sharing and avoid excessive consumption of system resources. Based on the designed network model, a cloud-edge collaborative resource allocation framework is formulated. By joint optimizing the offloading decision, block interval and transmission power, it aims to minimize the consumption overheads of system energy and service latency. Then the formulated problem is designed as a Markov decision process, and the optimal strategy can be obtained by the CRL. Some evaluation results reveal that the system performance based on the proposed scheme outperforms other existing schemes obviously.

With the rapid development of driverless technology, supporting the safe and rapid path planning of Unmanned Ground Vehicles in complex environments turns to be a significant challenge. In this paper, we propose a 3D environment-based multiobjective path planning strategy with consideration of the environmental characteristics and time cost By using the deep reinforcement learning algorithm, we further transform the initial problem into Markov decision process which can be solved. We consider grid quantization approach where the 3D terrain of the environment surface can be converted into a height-based hierarchical matrix and feed the matrix into the reinforcement learning architecture as the information. Meanwhile, we propose an action judgement mechanism to judge the legitimacy of actions in advance before execution to solve the collision problem in the real training process of the agent. Simulation results show the effectiveness and robustness of the proposed strategy with applications to two different types of 3D terrains and varying degrees of terrain disturbances.

Abstract Infrared (IR) and visible (VI) image fusion play an important role in improving the sea scene perception and ship target detection. Although there have been many studies on image fusion considering the sea scene characteristics, there has still been no adequate extraction method for detailed information of a ship target in a sea scene. To overcome this shortcoming, this paper proposes a fusion method based on a scale‐aware feature decomposition, which can accurately extract features of a ship target at different scales. First, a hybrid feature decomposition method based on the scale‐aware structure‐preserving filter and Gaussian filter is designed. The proposed method separated source images into region, structure, and texture layers, and thus achieved a finer‐scale division than traditional multiscale decomposition methods in the sea‐clutter background. Then, further decomposition of the texture layer was performed by the shearlet transform to obtain the directional texture feature. According to the characteristics of each layer, different strategies were adopted for the fusion, and the weighting factor was used to obtain the final fusion result. Experimental results indicated that the proposed method could achieve better subjective and objective results than current state‐of‐the‐art methods. This method introduces a scale‐aware edge preserving filter to improve the ability of detail extraction and is suitable for image fusion of ship targets.

Driven by numerous emerging mobile devices and various Quality-of-Service (QoS) requirements, mobile-edge computing (MEC) has been recognized as a prospective paradigm to promote the computation capability of mobile devices, as well as reduce energy overhead and service latency of applications for the Internet of Things (IoT). However, there are still some open issues in the existing research works: 1) limited network and computing resource; 2) simple or nonintelligent resource management; and 3) ignored security and reliability. In order to cope with these issues, in this article, 6G and blockchain technology are considered to improve network performance and ensure the authenticity of data sharing for the MEC-enabled IoT. Meanwhile, a novel intelligent optimization method named as collective reinforcement learning (CRL) is proposed and introduced, to realize intelligent resource allocation, meet distributed training results sharing, and avoid excessive consumption of system resources. Based on the designed network model, a cloud–edge collaborative resource allocation framework is formulated. By joint optimizing the offloading decision, block interval, and transmission power, it aims to minimize the consumption overheads of system energy and latency. Then, the formulated problem is designed as a Markov decision process, and the optimal strategy can be obtained by the CRL. Some evaluation results reveal that the system performance based on the proposed scheme outperforms other existing schemes obviously.

Heterogeneous networks (HetNets) with end-to-end (E2E) network slicing are regarded as effective approaches to meet diverse service requirements from vertical industries. Due to the dense deployment of base stations (BSs) and the complicated associations between BSs and E2E network slices (NSs) in the scenario, the handoff problem faces challenges of the huge system state space and handoff action space and the considerable communication overhead. In this paper, we take these issues into account and consider a distributed E2E NS handoff decision framework in the HetNet. A decentralized Markov decision process (DEC-MDP)-based model is formulated for the distributed E2E NS handoff problem, and the jointly observable and random characteristics of the DEC-MDP are analyzed. To obtain a theoretical performance reference, the original distributed E2E NS handoff problem is simplified, and a Nash equilibrium-based performance bound is given. More practically, the multi-agent double deep Q-network-based distributed handoff (MA-DDQN-DH) algorithm with the centralized training and decentralized executing framework is proposed. Simulation results show that the Nash equilibrium-based performance bound is reasonable, and the proposed MA-DDQN-DH algorithm performs well in the comparison.

Nowadays, the rise of the Internet of Vehicles (IoV) has led to the rapid development of smart transportation. To increase the computing capacity of mobile vehicles and decrease the content delivery latency of suppliers, mobile edge computing (MEC) is considered as an indispensable solution. However, there are some essential issues to be considered: 1) security and privacy of data transmission, and 2) reasonable resource allocation for collaborative computing and caching. In this paper, to solve above issues, blockchain technology is adopted to ensure reliable transmission and interaction of data. Meanwhile, we develop an intelligent resource framework about computing and caching for blockchain-enabled MEC systems in IoV. Through jointly considering and optimizing offloading decision of computation task carried by vehicle, caching decision, the number of offloaded consensus nodes, block interval and block size, the energy consumption and computation overheads can be decreased, and the data throughput of the blockchain can be increased significantly. Moreover, the proposed optimization problem is modeled and formulated as a Markov decision process. Facing the complexity and dynamic of resource allocation, the asynchronous advantage actor-critic approach is considered and applied to solve the optimization problem. Experiment results demonstrate that the advantages of the proposed optimization scheme are obvious compared with other existing schemes.

Recently, electric vehicles (EVs) have been widely used under the call of green travel and environmental protection, and diverse requirements for charging are also increasing gradually. In order to ensure the authenticity and privacy of charging information interaction, blockchain technology is proposed and applied in charging station billing systems. However, there are some issues in blockchain itself, including lower computing efficiency of the nodes and higher energy consumption in the consensus process. To handle the above issues, in this paper, combining blockchain and mobile edge computing, we develop a reliable billing data transmission scheme to improve the computing capacity of nodes and reduce the energy consumption of the consensus process. By jointly optimizing the primary and replica nodes offloading decisions, block size and block interval, the transaction throughput of the blockchain system is maximized, as well as the consumption costs of latency and energy consumption is minimized. Moreover, we formulate the joint optimization problem as Markov decision process (MDP). To tackle this dynamic and continuity of the system state, the actor–critic reinforcement learning is introduced to solve the MDP problem. Finally, simulation results demonstrate that the performance improvement of the proposed scheme through comparison with other existing schemes.

End-to-end network slicing, as a key technology in 5G and B5G mobile communication systems, is to enable traditional wireless networks to support different services in vertical industries. In a heterogeneous cellular network (HetNets) with network slicing functions, due to the dense deployment of base stations (BSs) and the mobility of user equipments (UEs), dynamically switching network slices (NSs) is necessary for better system performance. This paper models the handoff problem of end-to-end NS as a Markov decision process (MDP) maximizing the utility related to the UE's profit of being served, the handoff cost and the outage penalty. Both the states of the radio access network resources and the core network resources of each end-to-end NS are considered. The deep reinforcement learning (DRL) is adopted as the solution, and a double deep Q network (DQN) based NS handoff algorithm is designed. Numerical results confirm the convergence of the DQN used to make handoff decisions and show that compared with typical handoff algorithms, the algorithm we proposed performs the best from the aspect of the cumulative reward designed in this paper.

Industrial Internet of Things (IIoT) has emerged with the developments of various communication technologies. In order to guarantee the security and privacy of massive IIoT data, blockchain is widely considered as a promising technology and applied into IIoT. However, there are still several issues in the existing blockchain-enabled IIoT: 1) unbearable energy consumption for computation tasks; 2) poor efficiency of consensus mechanism in blockchain; and 3) serious computation overhead of network systems. To handle the above issues and challenges, in this article, we integrate mobile-edge computing (MEC) into blockchain-enabled IIoT systems to promote the computation capability of IIoT devices and improve the efficiency of the consensus process. Meanwhile, the weighted system cost, including the energy consumption and the computation overhead, are jointly considered. Moreover, we propose an optimization framework for blockchain-enabled IIoT systems to decrease consumption, and formulate the proposed problem as a Markov decision process (MDP). The master controller, offloading decision, block size, and computing server can be dynamically selected and adjusted to optimize the devices energy allocation and reduce the weighted system cost. Accordingly, due to the high-dynamic and large-dimensional characteristics, deep reinforcement learning (DRL) is introduced to solve the formulated problem. Simulation results demonstrate that our proposed scheme can improve system performance significantly compared to other existing schemes.

Nowadays, the Internet of Things (IoT) is playing an important role in our life. This inevitably generates mass data and requires a more secure transmission. As blockchain technology can build trust in a distributed environment and ensure the data traceability and tamper resistance, it is a promising way to support IoT data transmission and sharing. In this paper, edge computing is considered to provide adequate resources for end users to offload computing tasks in the blockchain enabled IoT system, and the node pairing problem between end users and edge computing servers is researched with the consideration of wireless channel quality and the service quality. From the perspective of the end users, the objective optimization is designed to maximize the profits and minimize the payments for completing the tasks and ensuring the resource limits of the edge servers at the same time. The deep reinforcement learning (DRL) method is utilized to train an intelligent strategy, and the policy gradient based node pairing (PG-NP) algorithm is proposed. Through a deep neural network, the well-trained policy matched the system states to the optimal actions. The REINFORCE algorithm with baseline is applied to train the policy network. According to the training results, as the comparison strategies are max-credit, max-SINR, random and max-resource, the PG-NP algorithm performs about 57% better than the second-best method. And testing results show that PGNP also has a good generalization ability which is negatively correlated with the training performance to a certain extend.

Nowadays, the Internet of Things (IoT) has developed rapidly. To deal with the security problems in some of the IoT applications, blockchain has aroused lots of attention in both academia and industry. In this paper, we consider the mobile blockchain supporting IoT applications, and the mobile edge computing (MEC) is deployed at the Small-cell Base Station (SBS) as a supplement to enhance the computation ability of IoT devices. To encourage the participation of the SBS in the mobile blockchain networks, the long-term revenue of the SBS is considered. The task scheduling problem maximizing the long-term mining reward and minimizing the resource cost of the SBS is formulated as a Markov Decision Process (MDP). To achieve an efficient intelligent strategy, the deep reinforcement learning (DRL) based solution named policy gradient based computing tasks scheduling (PG-CTS) algorithm is proposed. The policy mapping from the system state to the task scheduling decision is represented by a deep neural network. The episodic simulations are built and the REINFORCE algorithm with baseline is used to train the policy network. According to the training results, the PG-CTS method is about 10% better than the second-best method greedy. The generalization ability of PG-CTS is proved theoretically, and the testing results also show that the PG-CTS method has better performance over the other three strategies, greedy, first-in-first-out (FIFO) and random in different environments.

Recently, the development of the Internet of Things (IoT) provides plenty of opportunities and challenges in various fields. As an essential part of IoT, machine-to-machine (M2M) communications open a novel way that the machine-type communication devices (MTCDs) are connected and communicated without any human intervention. Meanwhile, delay-tolerant data play an important role in M2M communications-based IoT, and it puts more emphasis on powerful data caching, computing, and processing, as well as the security and stability of data transmission. To meet these requirements in M2M communications networks, in this article, we introduce some promising technologies, such as edge computing and blockchain, and propose a joint optimization framework about caching, computation, and security for delay-tolerant data in M2M communications networks based on dueling deep Q-network (DQN). According to the dynamic decision process by DQN, the optimal selection and decision of caching servers, computing servers, and blockchain systems can be made to achieve maximum system rewards, which includes higher efficiency of data processing, lower network costs, and better security of data interaction. Extensive simulation results with different system parameters show that our proposed framework can effectively improve the system performance for blockchain-enabled M2M communications compared to the existing schemes.

In this paper, we investigate a metamaterial- inspired small rectenna (rectifying antenna) system for microwave energy harvesting and wireless power transfer at 5.8 GHz. We also study the received-power maximization technique using an array of antennas connected to a single load by an optimal RF power combiner. This investigation is done both in simulation and experiment. The power combiner is embedded between the metamaterial- inspired antennas and the load (including rectifier) for maximizing the power, harvested by the input antennas and to deliver it to the load in the most optimal way. Moreover, we design and test proof-of-concept prototypes of the main components of the investigated energy harvesting systems. The similarity between the simulation and the experimental results also confirms our method of investigation.

In most cases, the batteries of sensor nodes in the Internet of Things (IoT) are usually constrained by size and weight, and are difficult to recharge or replace. In traditional wireless sensor networks, data is transmitted in a multi-hop manner, which may cause the high data transmission delay and unbalanced traffic load. In this paper, an Unmanned Aerial Vehicle (UAV)-assisted IoT architecture is introduced, in which UAV is utilized to achieve low-latency and seamless-coverage acquisition of the sensing data. Furthermore, based on the recent advances on deep reinforcement learning algorithms, considering both data delay requirements and network energy consumption, a real-time flight path planning scheme of the UAV in the dynamic IoT sensor networks has been proposed based on dueling deep Q-network (DQN). Besides, the grid-based method is used to handle the network state modeling, which effectively reduces the complexity of the proposed scheme. Simulation results show that the proposed scheme significantly improves the network performance.

With an increasing number of machine-type communication devices (MTCDs), machine-to-machine (M2M) communications have attracted great attentions from both academia and industry. Different from traditional communication networks, the data connections with M2M communications are typically small-sized but with high frequency, necessitating the efficiency optimization of both energy consumption and computation. In this paper, we introduce mobile edge computing (MEC) into virtualized cellular networks with M2M communications, to decrease the energy consumption and optimize the computing resource allocation as well as improve computing capability. Moreover, based on different functions and quality of service (QoS) requirements, the physical network can be virtualized into several virtual networks, and then each MTCD selects the corresponding virtual network to access through the embedded-SIM (eSIM) technology. Meanwhile, the random access process of MTCDs is formulated as a partially observable Markov decision process (POMDP) to minimize the system cost, which consists of both the energy consumption and execution time of computing tasks. Furthermore, to facilitate the network architecture integration, software-defined networking (SDN) is introduced to deal with the diverse protocols and standards in the networks. Extensive simulation results with different system parameters reveal that the proposed scheme could significantly improve the system performance compared to the existing schemes.

With the explosion in the number of connected devices and Internet of Things (IoT) services in smart city, the challenges to meet the demands from both data traffic delivery and information processing are increasingly prominent. Meanwhile, the connected vehicle networks have become an essential part in smart city, bringing massive data traffic as well as significant networking, caching and computing resources. In this paper, we propose a novel vehicle network architecture, mitigating the network congestion with the joint optimization of networking, caching and computing. Cloud computing at the data centers as well as mobile edge computing (MEC) at the evolved node Bs (eNodeBs) and on-board units (OBUs) are taken as the paradigms to provide caching and computing resources. The programmable control principle originated from software-defined networking (SDN) paradigm has been introduced to facilitate the system architecture and resource integration. With the careful modeling of the services, the vehicle mobility and the system state, a joint resource management scheme is proposed and formulated as a partially observable Markov decision process (POMDP) to minimize system cost, which consists of both network overhead and execution time of computing tasks. Extensive simulation results with different system parameters reveal that the proposed scheme could significantly improve the system performance compared to the existing schemes.

The Fourth Industrial Revolution we are experiencing currently is reshaping the world by facilitating factories with intelligence and significantly improved manufacturing efficiency and flexibility. Among the key technologies to achieve Industrie 4.0, industrial wireless networking enables convenient and reliable connections among the machines, network devices, cloud servers and humans for both delay-sensitive traffic and delay-tolerant data delivery. In this paper, the Cognitive radio and Edge computing based Industrial wireless Network (CEIN) is introduced. In CEIN, edge computing handles the processing requirements of the data during its transmission, and is deployed close to the machines for immediate response to delay-sensitive industrial data that requires real-time processing. Cognitive radio technologies are also adopted to ensure efficient spectrum resource utilization for big delay- tolerant data transmission that contributes mostly to the industrial data traffic. Besides, we propose an optimal networking and computing resource management scheme for CEIN. The harvested spectrum bands are allocated to the network devices taking into account of the computing requirements of industrial data. Stochastic optimization is adopted to find the optimal allocation actions with low on-line computational complexity. Extensive simulation results are also presented to demonstrate the significant system performance improvement.

As an important part of the Internet-of-Things (IoT), machine-to-machine (M2M) communications have attracted great attention. In this paper, we introduce mobile edge computing (MEC) into virtualized cellular networks with M2M communications, to decrease the energy consumption and optimize the computing resource allocation as well as improve computing capability. Moreover, based on different functions and quality of service (QoS) requirements, the physical network can be virtualized into several virtual networks, and then each MTCD selects the corresponding virtual network to access. Meanwhile, the random access process of MTCDs is formulated as a partially observable Markov decision process (POMDP) to minimize the system cost, which consists of both the energy consumption and execution time of computing tasks. Furthermore, to facilitate the network architecture integration, software-defined networking (SDN) is introduced to deal with the diverse protocols and standards in the networks. Extensive simulation results with different system parameters reveal that the proposed scheme could significantly improve the system performance compared to the existing schemes.

Indoor localization has attracted much attention recently, due to its wide applications in location-based services(LBSs). Localization accuracy and system costs are the key issues while designing indoor localization schemes. In this paper, an edge big data-enabled indoor localization scheme is proposed. We use the radio signal strength (RSS) information that is always available wherever WiFi coverage is available, to avoid the costs on deploying and maintaining specific devices for indoor localization. Bayesian theory and edge computing are adopted in our system, so that big localization data is collected and utilized to update the prior location probabilities. A testbed, BJUTLocate, is built to evaluate the performance of the proposed scheme, and the evaluation results show its significant performance improvement.

Machine-to-machine (M2M) communications have attracted great attention from both academia and industry. In this paper, with recent advances in wireless network virtualization and software- defined networking (SDN), we propose a novel framework for M2M communications in software- defined cellular networks with wireless network virtualization. In the proposed framework, according to different functions and quality of service (QoS) requirements of machine-type communication devices (MTCDs), a hypervisor enables the virtualization of the physical M2M network, which is abstracted and sliced into multiple virtual M2M networks. Moreover, we formulate a decision-theoretic approach to optimize the random access process of M2M communications. In addition, we develop a feedback and control loop to dynamically adjust the number of resource blocks (RBs) that are used in the random access phase in a virtual M2M network by the SDN controller. Extensive simulation results with different system parameters are presented to show the performance of the proposed scheme.

To deal with the rapid growth of mobile data traffic and the user interest shift from peer-to-peer communications to content dissemination-based services, such as video streaming, information-centric networking has emerged as a promising architecture and has been increasingly used for wireless and mobile networks. In this paper, we focus on video dissemination in information-centric cognitive radio networks (IC-CRNs) and investigate the use of harvested bands for proactively caching video contents at the locations close to the interested users to improve the performance of video distribution. With consideration of the dynamic and unobservable nature of some parameters, we formulate the allocation of harvested bands as a Markov decision process with hidden and dynamic parameters and transform it into a partially observable Markov decision process and a multi-armed bandit formulation. Based on them, we develop a new spectrum management mechanism, which maximizes the benefit of proactive video caching as well as the efficiency of spectrum utilization in the IC-CRNs. Extensive simulation results demonstrate the significant performance improvement of the proposed scheme for video streaming.

Abstract Cyber‐physical system has been widely used in various areas as the integration of computing and physical system. As a typical application of cyber‐physical system, cyber‐physical surveillance system for transportation (CPSST) allows real‐time video monitoring to facilitate the deploying of smart transportation systems. For video streaming in CPSST, dynamic radio spectrum management is a key technology dealing with the current situation that the spectrum resource is almost used up. In this paper, taking into account the application layer quality of service and wireless link security, a novel dynamic spectrum management scheme has been proposed to minimize the system cost of CPSST. Video distortion is considered as the application layer quality of service metric, and the system cost is defined as a combination of distortion and security cost. We use intra‐refreshing rate in video coding to minimize the distortion. Furthermore, the problem is formulated as a restless bandit system, which uses current and historical information to optimize the action, with the objective of maximizing the total discounted system reward. We also describe the two spectrum management processes. Extensive simulation results are presented to demonstrate the significant performance improvement of the proposed scheme compared with the existing one that ignores video distortion and subband security optimization. Copyright © 2014 John Wiley & Sons, Ltd.

When equipped with multiple antennas, secondary users in cognitive radio networks are able to communicate even when neighboring primary users are active by transmitting in the null space of the communication channel occupied by primary users. In this case, the throughput of a secondary link is limited by the transmission power and the dimension of the null space, i.e., the number of active primary users nearby. Since the number of active primary users is time-varying, the required transmission power to support certain data rate changes from time to time. Thus, secondary users could adapt their transmission to the variation of the primary traffic to improve energy efficiency. In view of that, we develop an energy-efficient traffic scheduling scheme for secondary users equipped with multiple antennas. By formulating the traffic scheduling problem as a Markov decision problem, an energy-efficient transmission scheme is derived from linear programming. The analytical results are verified by simulations and the impacts of various parameters are discussed. The superiority of the derived scheme is also shown by comparing with a randomized scheme.

In cognitive radio networks, primary users (PUs) can leverage secondary users (SUs) as cooperative relays to increase their transmission rates, and SUs will in turn obtain more spectrum access opportunities. While most existing works assume that SUs are passively selected by PUs regardless of SUs' willingness, in this paper, we propose a cooperative strategy for SUs to actively decide whether to cooperate or not. Basically, due to PUs' time-varying traffic demands, it is essential for SUs to firstly observe the channels and then select a specific PU to cooperate with in order to save the energy. In our paper, this decision related problem is formulated based on optimal stopping theory where SUs observe PUs in time sequence and then make decisions whether to stop observation and cooperate right away or wait till next time slot to repeat the same process. We address this problem by using backward induction and derive the energy-efficient strategy for SUs. To validate the feasibility of our proposed scheme, extensive simulations are conducted to show the impact of PUs' traffic demands on SUs' decisions. The results also reveal that the proposed optimal rule outperforms the greedy selection strategy and is thus more energy- efficient to be applied to the cooperative cognitive radio networks.

Recently, unmanned aerial vehicle (UAV) cooperation networks have attached much attention due to their successful applications in complex military and civilian missions. In this paper, we propose a navigation data-assisted optimal opportunistic spectrum access scheme for wireless communications in heterogeneous UAV networks, to achieve maximized data rate by flexibly scheduling the spectrum subbands. The system architecture is introduced, and thanks to the navigation data that is always available locally at the entities in the network, prediction of wireless link quality and routing information can be obtained to assist subband allocations. Furthermore, the spectrum allocation process is formulated as an optimization problem. Simulation results are also presented to demonstrate the significant performance improvement of the proposed scheme compared to the existing one.

This paper is concerned with the challenging problem of joint channel estimation and data detection for high mobility multiple-input multiple-output orthogonal frequency division multiplexing systems. We propose a new iterative channel estimation and detection scheme, which reduces the unexpected effects of both detection errors and channel estimation errors. Detection errors in channel estimation are analyzed and transformed as part of the noise. To filter this equivalent noise by Kalman estimator, we derive the covariance of both the channels and data errors in detection. Besides, we propose a new detection algorithm with an optimized weight to minimize the detection error caused by channel estimation errors. To obtain this optimized weight, the error covariance of the estimated channels is derived from the error estimate covariance matrix in Kalman estimator. Simulation results are presented to demonstrate the significant performance improvement in joint channel estimation and data detection with the proposed iterative scheme.

In mobile cloud computing (MCC) systems, the resource in both the cloud and the mobile network should be carefully managed. Cloud resource management and radio resource management have traditionally been addressed separately in previous works. In this paper, we propose to jointly study dynamic cloud and radio resource management so as to improve end-to-end performance of adaptive video transmissions in MCC systems. Video application quality of service performance, distortion, is adopted as the performance measure. An important video application layer parameter, intra-refreshing rate, is optimized to improve the video distortion performance. We formulate the problem as a stochastic restless bandits optimization problem, which facilitates the distributed MCC architecture and simplifies the computation and implementation due to its “indexibility” property. Simulation results are presented to show the effectivenes of the proposed scheme.

Cyber-physical system (CPS) has been widely used in various areas as the integration of computing and physical system. As a typical application of CPS, cyber-physical surveillance system (CPSS) allows real-time video monitoring for various fields such as smart transportation and warehouse management systems. For video streaming in CPSS, dynamic radio spectrum management is a key technology dealing with the current situation that the spectrum resource is almost used up. In this paper, taking into account the application layer quality-of-service (QoS) and wireless link security, a novel dynamic spectrum management scheme is proposed to minimize the system cost of CPSS. Video distortion is considered as the application layer QoS metric, and the system cost is defined as a combination of distortion and security cost. We use intra-refreshing rate in video coding to minimize the distortion. Furthermore, the problem is formulated as a restless bandit system, which uses current and historical information to optimize the action, with the objective of maximizing the total discounted system reward. We also describe the spectrum management operation processes. Extensive simulation results are presented to demonstrate the significant performance improvement of the proposed scheme compared with the existing one that ignores video distortion and subband security optimization.

In cognitive radio (CR) networks, there is a common assumption that the secondary devices always obey the spectrum access rules and are under full control. However, this may become unrealistic for future CR networks composed of intelligent, complicated and autonomous devices. To solve this problem, the concept of “proactive-optimization” cognitive radio (POCR) is proposed in this paper, in which the highly-intelligent secondary users proactively optimize their own behavior decisions according to the available information including device state and network condition to maximize their long-term reward. Furthermore, we propose an optimal transmission behavior decision scheme for secondary users in POCR networks considering imperfect spectrum channel sensing results. Specifically, we formulate the system as a partially-observable Markov decision process (POMDP) problem. With this formulation, a low complexity dynamic programming framework is introduced to obtain the optimal behavior policy. Extensive simulation results are presented to illustrate the significant performance improvement of the proposed scheme compared with the existing one that ignores the secondary user behavior optimization.

In Cognitive Radio (CR) networks, there could be a spectrum market which operates in real time with primary users (PU) as manager, where the secondary users (SU) pay the PUs for spectrum resource usage. Multi-carrier systems such as OFDM are best candidates for applying in CR networks because of the spectrum shaping and high adaptive capabilities. In this paper, an optimal resource allocation scheme aims at maximizing PU's reward in OFDM-based CR networks is proposed. Both the interference limit and BER requirements of SUs are considered by the PU to allocate its spectrum resources. The scheme is modeled as restless bandits problem, which can dramatically simplify the computation and implementation. Furthermore, extensive simulation results show that our proposed scheme can improve the reward of PU significantly compared to the existing random scheme and greedy scheme.

In conventional WLANs, stations (STAs) select access points (APs) using the existing scheme based on the current quality of links. However, some novel improved architecture of WLANs is proposed with cognitive radio (CR) to negotiate spectrum usage, where the existing AP selection scheme might not be suitable for it. Thus in this paper, a new optimal AP Selection based on Restless Bandits (APSRB) scheme with the "indexability" property is proposed for WLANs with CR to maximize the throughput and to minimize the energy consumption. The AP selection problem is firstly established as a restless bandit problem, which is solved by the primal-dual index heuristic algorithm based on the first order relaxation with low complexity to yield APSRB scheme. Additionally, the APSRB scheme is divided into offline computation and online selection, where main work will be finished in former one so as to decrease the complexity further. Finally, extensive simulation results illustrate the significant performance improvement of the APSRB scheme compared to the existing one in different scenarios.

In this paper, we take both the centralized and distributed architectures into account in heterogenous cognitive radio networks, and study the problem of the joint transmission time and power allocation. The problem is formulated as a dual optimization problem with the optimization objective to maximize the total capacity of the secondary users (SUs) with the constraint of fairness. We first optimize the joint transmission time and power allocation for centralized SUs and propose a corresponding resource allocation scheme in the time-frequency domain. Then for the heterogeneous case with both the centralized and distributed network architecture considered, we formulate the resource allocation problem as a cooperative game and propose an iterative power water-filling scheme to get to the Nash Equilibrium (NE). Based on the dual optimization, a dynamic optimal joint transmission time and power allocation scheme for heterogenous cognitive radio networks is proposed. Extensive simulation results are presented to illustrate the performance of the proposed scheme.

Spectrum sensing is one of the key issues for spectrum sharing in cognitive radio networks to deal with the more and more serious problem on exhaustive spectrum resource. In this paper, we study the problem of wideband spectrum sensing in cognitive radio networks with multiple primary networks. By dividing the total wideband spectrum into many groups with relatively small number of narrowbands, a novel wideband spectrum sensing scheme is proposed. In the proposed scheme, the number and corresponding probability of free bands in each group are predicted by Lempel-Ziv based prediction algorithm. Then the optimal number of secondary users that will sense the narrowbands in each group is found to maximize the defined reward. Not only the sensing speed and accuracy but also the system reward are improved in this paper. Finally, extensive simulation results are provided to show the effectiveness of our proposed wideband spectrum sensing scheme by comparing with the existing ones.

Dynamic spectrum access is one of the most important premises of spectrum reuse based on cognitive radio technologies, which are considered to be the best way to alleviate the controversy on spectrum scarcity and low efficiency. However, most of previous work focuses on the increasing of system throughput, ignoring the QoS requirement of secondary users. In this paper, dynamic spectrum access with QoS provisioning is studied. We adopt discrete-time Markov chain to analyze and model the spectrum usage in time-slotted cognitive radio networks. Furthermore, three admission control schemes are proposed to minimize the forced termination probability of secondary users. Simulation results show that our proposed schemes can significantly improve the forced termination probability of secondary users, though slightly increase the blocking probability.

Spectrum pooling, which allows the secondary networks to utilize the available spectrum bands from different licensed networks, is one of the most promising technologies for dynamic spectrum sharing in cognitive radio systems. Most previous work on spectrum pooling concentrates on the system architecture and the design of flexible access algorithms and schemes. In this paper, a distributed scheme for optimal internetwork spectrum sharing among multiple cognitive radio systems is proposed. Besides, the spectrum access price and spectrum efficiency are considered as the design criteria in the proposed scheme. The spectrum sharing problem is formulated as a restless bandits system, which dramatically reduces the computational complexity by simply allocating the new available band to the secondary network with the lowest index. Furthermore, extensive simulation results illustrate the significant performance improvement of the proposed scheme improves compared to the existing scheme.

Resource allocation problem in cognitive radio networks (CRN) is one of the key issues to improve the efficiency of spectrum utilization. Most of previous work on resource allocation mainly concentrates on the secondary users (SUs) with only one type of service requirement, without considering the scenario with heterogenous services requirement. In this paper, we study the dynamic channel and power allocation for SUs supporting heterogenous services in CRN. Firstly we classify the SUs by service requirement, i.e., SUs with minimum rate guarantee and SUs with best-effort services. Then we introduce the minimum rate constraints and proportional fairness constraints for SUs respectively. Under this setup, we formulate the problem of dynamic channel and power allocation for SUs as a mixed integer programming problem. And the heuristic optimal algorithm and suboptimal algorithm are proposed to realize the dynamic channel and power allocation. Extensive simulation results are presented to demonstrate the performance of the proposed scheme.

In cognitive radio networks (CRN), underlay spectrum sharing allows secondary users (SUs) to utilize the spectrum on which the primary users (PUs) in primary radio networks (PRN) are working at the same time, without introducing intolerant interferences. In this paper, we study the joint transmission power and time allocation in CRN with underlay spectrum sharing technology. To maximize the total capacity of CRN and maintain the fairness for SUs, the system is modeled as an optimization problem with the constraints of interference and transmission power and time. Furthermore, we prove that the problem of joint optimization resource allocation under the constraints condition can be implemented independently from the time-frequency domain. Based on this study, we propose the optimal joint transmission power and time allocation method. Extensive simulation results show that the proposed optimal method can significantly improve the system capacity and maintain the fairness compared to the existing methods.

In wireless mobile peer-to-peer (P2P) networks, multiple sources can provide multimedia file sharing at the same time. The selection of multiple sources is one of the key issues in the design of a multi-source transmission system in wireless mobile P2P networks. In this paper, we propose a distributed multi-source sender selection scheme to maximize the receiving data rate and minimize the energy consumption. Our scheme is based on recent advances in restless bandit algorithms. The proposed sender selection scheme has an indexability property that dramatically simplifies the computation and implementation of the policy. In addition, there is no need for a centralized control point, and senders can join and leave from the wireless mobile P2P network freely. Simulation results show that the proposed scheme improves the receiving data rate and energy consumption performance significantly compared to the existing scheme.

The complementary characteristics of different wireless networks make it attractive to integrate a wide range of radio access technologies. Most of previous work on integrating heterogeneous wireless networks concentrates on network layer quality of service (QoS), such as blocking probability and utilization, as design criteria. However, from a user's point of view, application layer QoS, such as multimedia distortion, is an important issue. In this paper, we propose an optimal distributed network selection scheme in heterogeneous wireless networks considering multimedia application layer QoS. Specifically, we formulate the integrated network as a restless bandit system. With this stochastic optimization formulation, the optimal network selection policy is indexable, meaning that the network with the lowest index should be selected. The proposed scheme can be applicable to both tight coupling and loose coupling scenarios in the integration of heterogeneous wireless networks. Simulation results are presented to illustrate the performance of the proposed scheme.

The network selection problem in next generation heterogeneous wireless networks is one of the key issues to integrate a wide range of radio access technologies. In most previous work, the network selection problem is modeled as a centralized system, without considering the distributed nature of heterogeneous wireless networks. In this paper, we propose a distributed network selection scheme in heterogeneous wireless networks. The the heterogeneous wireless networks are formulated as a restless bandit system, which has an "indexability" property that dramatically reduces the complexity. Extensive simulation results are presented to illustrate the performance improvement of the proposed scheme.

Multi-source multimedia transmission is a popular architecture in wireless mobile peer-to-peer (P2P) networks. Most of previous work on wireless mobile P2P networks concentrates on the protocols and network structures, and consequently ignores the multiple senders scheduling problem. In this paper, we present a distributed algorithm for scheduling the multiple senders for multi-source transmission in wireless mobile P2P networks, which can maximize the data rate and minimize the power consumption. Specifically, we formulate the wireless mobile P2P network as a multi-armed bandit system. The optimal distributed sender scheduling policy can be found according to the Gittins indices of the senders. Extensive simulation examples illustrate the effectiveness of the proposed scheme. It is shown that the data rate and power consumption in the proposed scheme can be improved significantly compared to existing schemes.

Multi-source multimedia transmission is a popular approach in wireless peer-to-peer networks. Most previous related work concentrates on the protocols and network structures, and consequently ignores the multiple senders scheduling problem. In this paper, we present a distributed sender scheduling algorithm for multi-source transmission in wireless P2P networks, which can maximize the data rate and minimize the power consumption. Specifically, we formulate the network as a multi-arm bandit system. The optimal distributed scheduling policy can be found according to the Gittins indices of the senders. Numerical examples illustrate that the data rate and power consumption can be improved significantly compared to existing schemes.

Chord is one of the best known lookup protocols for structured peer-to-peer (P2P) networks. Nodes in Chord can be viewed as being placed on a one-dimensional ring. In this paper, we present a novel concept of bi-dimensional P2P, in which all nodes are placed onto a square rather than a ring to enable the flexible configuration of ring(s). Diverse ring configuration schemes form a uniform protocol family called multi-ring bi-dimensional (MRBD) protocols, and Chord can be considered as MRBD-1. Different configurations of rings provide various performance to satisfy diverse user requirements. We study examples of MRBD from both theoretical analysis and simulations. The results validate the effectiveness of MRBD in improving P2P lookup performance.

In this paper, we focus on the Markov model of IEEE 802.11 distributed coordination function with power saving mode. The throughput evaluation is based on the model, with the comparison with the simulation result for validation. Furthermore, we notice that after each beacon transmission, there're more stations in contention than usual. Excessive contention results in high collision probability and low throughput. To solve this problem, we propose a downlink access scheme which can be used in both the basic access scheme and the access scheme with RTS/CTS. This novel scheme enables AP to constrain the number of stations in contention to be an optimal value. Simulation results prove the performance improvement declared. The proposed scheme can be extended for multi-rate or multi-service WLANs.