This paper investigates the optimal channel exploration in multiple spectrum band covert communication. Different from most existing covert communication works that only consider the static wardens, we also take the reactive wardens that release real-time suppression tracking jamming based on their receiving power into consideration. In channel exploration period, the covert transmitter Alice aims to find a channel with high channel gain while guaranteeing its transmission covertness. However, obtaining channel gains on different bands before transmission takes Alice time and Alice has to choose the right time to stop exploring for throughput maximization. Firstly, to address the threats of the inactive and reactive wardens, the strategies of channel inversion power control and feedback based anti-jamming are respectively adopted. Then, we formulate the channel exploration problem with optimal stopping theory after analyzing Alice’s transmission covertness performance. Furthermore, since the conventional approach to this problem requires intensively computation, the one stage look ahead (1-SLA) rule is adopted to reduce the computation complexity. In particular, we mathematically prove that this rule is optimal in maximizing Alice’s expected throughput. At last, the simulation results are provided to validate the analytical results and the superiority of the proposed scheme compared with the benchmarks.

This paper focuses on distributed resource management and task scheduling for multi-access MEC networks against the intelligent eavesdropping jammer (IEJ). Due to the lack of a central controller, the problem of joint task scheduling and network resource allocation is formulated as a distributed multi-user hybrid-integer non-convex model.The optimization objective is to maximize users’ satisfaction while meeting the Quality of Service (QoS) requirements of tasks and ensuring the high-reliable demands of data offloading. To overcome the challenge of partial observability for users, the channel observation matrix and Gramian Angular Field (GAF) are utilized to preprocess the limited channel state information and to mine the potential time-frequency characteristics of the external environment. Moreover, the hierarchical architecture and parallel networks are introduced for a parameterized redesign of the Multi-Agent Twin Delayed Deep Deterministic Policy Gradient (MATD3) method to improve the decision accuracy. Finally, simulation results demonstrate the superiority of the proposed algorithm over existing methods in terms of delay, energy consumption, and security.

Neighbor discovery is a prerequisite to establish self-organizing IoT networks. IoT devices are mostly energy-limited, so neighbor discovery is usually implemented in a duty-cycled manner. A fundamental problem of optimizing energy efficiency of duty-cycled neighbor discovery is to minimize neighbor discovery latency for given duty cycle. We investigate purely interval (PI)-based neighbor discovery that adopts pseudo-random delay to avoid blocking problem, which is also adopted by pseudo-random delay advertising mode of Bluetooth low energy (BLE). In this article, we proposed an additional scan windows-based assisted protocol, which achieves early discovery with the help of assistance mechanism. Based on the assistance mechanism, we model and optimize the neighbor discovery latency of the PI-based protocol. Moreover, to analyze the impact of assistance failure caused by contention problem in a clique scenario, we proposed a mutual exclusive drawer model. Simulation results show that the proposed protocol using the derived near-optimal parameters achieves 26% lower average latency in sparse networks.

The low earth orbit (LEO) satellites have received extensive attention as an essential supplement to the terrestrial network for supporting global Internet of Things (IoT) services. Considering the rapid growth of IoT devices and the significant satellite-to-ground latency, proposing low-latency, low-overhead access protocols for LEO satellite IoT systems is challenging. In this article, we propose a multibeam random access (RA) framework and deploy the deep reinforcement learning (DRL) algorithm to control the nonorthogonal multiple access (NOMA) aided RA strategy. First, we divide the satellite coverage region into multiple beams and assume that the adjacent beams share parts of regions. Hence, the devices in the sharing region are allowed to transmit packets in two periods allocated for the two beams. Then, packets in multiple beams can be decoded jointly by an interslot successive interference cancelation (SIC) decoder. In addition, we consider the heterogeneity among devices and assign different power levels for heterogeneous types of devices, which enables power-domain NOMA and the intraslot SIC decoder in this system to mitigate the collision resolution. To maximize the average throughput, the deep deterministic policy gradient (DDPG) algorithm is adopted to achieve an online decision to optimize the RA protocol where the packet repetition strategies of devices are adjusted dynamically. The simulation results show that the proposed scheme outperforms the traditional benchmark schemes with significant throughput gain.

The primary objective of anti-jamming techniques is to ensure that the transmitted data arrives at the intended receiver without being disturbed or jammed with by any jamming signal or other hostile activities to ensuring the security of the communication system. Deep reinforcement learning (DRL) has been extensively utilized in solving the dynamic spectrum anti-jamming problem. However, most of existing DRL-based algorithms require lots of training time, which fails to adapt the fast-channging jamming environment. Our objective is to find a practical and fast-convergence anti-jamming learning solution. To achieve this, we redesign the DRL algorithm in the following two ways. First, we split the cycle of reinforcement learning into two parts: applying process and training process. Second, we use soft labels instead of rewards which bring more information. We further theoretically show that the information gain can help our proposed algorithm converge faster. Moreover, we also show that our labeled DRL algorithm is better than the idealized DRL-based scheme which can obtain the same information as the soft labels. Simulation results demonstrate that compared with existing DRL-based algorithms, our proposed algorithm reduces the number of iterations by up to 90%.

Abstract This paper investigates the joint relay and channel selection problem using a deep reinforcement learning (DRL) algorithm for cooperative communications in a dynamic jamming environment. The latest types of jammers include the mobile and smart jammer that contains multiple jamming patterns. This new type of jammer poses serious challenges to reliable communications such as huge environment states, tightly coupled joint action selections and real‐time decision requirements. To cope with these challenges, a DRL‐based relay‐assisted cooperative communication scheme is proposed. In this scheme, the joint selection problem is constructed as a Markov decision process (MDP) and a double deep Q network (DDQN) based anti‐jamming scheme is proposed to address the unknown and dynamic jamming behaviors. Concretely, a joint decision‐making network composed of three sub‐networks is designed and the independent learning method of each sub‐network is proposed. The simulation results show that the user agent is able to anticipate the jammer behaviors and elude the jamming in advance. Furthermore, compared with the sensing‐based algorithm, the Q learning‐based algorithm and the existing DRL‐based anti‐jamming approaches, the proposed algorithm maintains a higher average normalized throughput.

In this study, the air–ground integrated deployment method is studied for the unmanned aerial vehicle (UAV)‐enabled mobile edge computing (MEC) system. The UAV can help to reduce the delay and energy consumption of MEC. However, the limited coverage range of UAV limits the quality of data offloading. To improve the efficiency of data transmission, a hierarchical game model is designed. Ground nodes form multiple coalitions actively according to the position of UAV and the UAV adjusts the position based on the data distribution of ground networks. The relationship between the UAV and ground nodes is modelled as a Stackelberg game. A coalition formation game (CFG) is constructed for the data gathering among ground nodes. It is proved that the proposed CFG is an exact potential game with at least one Nash equilibrium. Moreover, the property of Stackelberg equilibrium is proven for the air–ground cooperative relationship. Based on the hierarchical model, a distributed air–ground integrated deployment algorithm is proposed to jointly optimise the position of the UAV and the coalition formation of ground nodes. The simulation results show that the proposed method promotes the efficiency of data transmission greatly and can converge to a stable state with reasonable iteration times.

Here the intelligent jammer issue is studied. With the rapid development of cognitive radio technology, current cognitive terminals can adaptively or intelligently switch channel by spectrum sensing and decision‐making. Most of the traditional jamming methods, such as swept jamming and comb jamming, generally work in a relatively fixed pattern, which are not able to effectively jam the terminals empowered with cognition and spectrum decision‐making capability. In view of this problem, the authors propose an intelligent jamming decision‐making system based on reinforcement learning. First, in order to jam a pair of transmitter and receiver with adaptive frequency hopping capability, a jammer with spectrum sensing, offline training and learning scheme is proposed. Second, a reinforcement learning‐based algorithm for jamming decision‐making is proposed and simulated. A special feature of the proposed scheme is that considering the reward is difficult to obtain in the actual communication system, a virtual jamming decision‐making method is used to enable the jammer to learn and jam efficiently without the user's prior information. Finally, the proposed jamming model and algorithm are implemented and verified on Universal software radio peripheral testbed.