With the rapid development of reinforcement learning (RL)-enhanced anti-jamming wireless communication technologies and jamming technologies, intelligent communication confrontation has become an urgent problem to be solved. Most existing work assumed that detailed information of jammer was known in advance, which hardly holds in practice. Besides, some work was sensitive to the changing of jamming policy, leading to limited adaptability and scalability. This article extends the research to scenarios with unknown jammer and diverse jamming policies, including fixed, reactive, and deep RL (DRL)-based proactive jamming policies. The interaction between communication party and jammer is formulated as a partially observable adversarial team stochastic game (POATSG). To cope with unknown and diverse jamming policies, a prioritized fictitious rival play (PFRP)-based robust anti-jamming spectrum access scheme (RASAS) is proposed. First, a fictitious jammer is designed to force the communication party to promote robustness via adversarial training. Then, a synchronized update mechanism is adopted to mitigate the nonstationary issue. Finally, the fictitious agent pool is introduced to create diverse fictitious opponents and avoid overfitting. Simulation results show that the PFRP-based scheme is robust to the jamming policy, switching cycle of the jamming policy, and jamming channel number.

This paper addresses the challenge of anti-jamming in moving reactive jamming scenarios. The moving reactive jammer initiates high-power tracking jamming upon detecting any transmission activity, and when unable to detect a signal, resorts to indiscriminate jamming. This presents dual imperatives: maintaining hiding to avoid the jammer’s detection and simultaneously evading indiscriminate jamming. Spread spectrum techniques effectively reduce transmitting power to elude detection but fall short in countering indiscriminate jamming. Conversely, changing communication frequencies can help evade indiscriminate jamming but makes the transmission vulnerable to tracking jamming without spread spectrum techniques to remain hidden. Current methodologies struggle with the complexity of simultaneously optimizing these two requirements due to the expansive joint action spaces and the dynamics of moving reactive jammers. To address these challenges, we propose a parallelized deep reinforcement learning (DRL) strategy. The approach includes a parallelized network architecture designed to decompose the action space. A parallel exploration-exploitation selection mechanism replaces the$\varepsilon $-greedy mechanism, accelerating convergence. Simulations demonstrate a nearly 90% increase in normalized throughput.

This paper focuses on distributed resource management and task scheduling for multi-access MEC networks against the intelligent eavesdropping jammer (IEJ). Due to the lack of a central controller, the problem of joint task scheduling and network resource allocation is formulated as a distributed multi-user hybrid-integer non-convex model.The optimization objective is to maximize users’ satisfaction while meeting the Quality of Service (QoS) requirements of tasks and ensuring the high-reliable demands of data offloading. To overcome the challenge of partial observability for users, the channel observation matrix and Gramian Angular Field (GAF) are utilized to preprocess the limited channel state information and to mine the potential time-frequency characteristics of the external environment. Moreover, the hierarchical architecture and parallel networks are introduced for a parameterized redesign of the Multi-Agent Twin Delayed Deep Deterministic Policy Gradient (MATD3) method to improve the decision accuracy. Finally, simulation results demonstrate the superiority of the proposed algorithm over existing methods in terms of delay, energy consumption, and security.

As a fundamental requirement for IoT communication systems the importance of highly reliable anti-jamming communication methods in safety use cases is growing. In this article, we propose a novel anti-intelligent jamming scheme called opponent awareness-based anti-jamming algorithm (OA3). The user-jammer-environment interaction is formulated as a two-player simultaneous action stochastic game where participators have the ability to update their strategies. The decision-making process of each agent is modeled as a Markov decision process (MDP). Begin with the intuition “learn how the jammer learns,” the opponent awareness-based iterative learning objective (OAL) of the user is presented by considering the learning awareness of the jammer to defeat the intelligent jamming. Finally, we introduce the framework, including offline policy learning and online policy exploiting to implement OAL and accelerate the learning. Simulations show that the OA3 outperforms the benchmark anti-jamming strategy in terms of packet success rate.

The primary objective of anti-jamming techniques is to ensure that the transmitted data arrives at the intended receiver without being disturbed or jammed with by any jamming signal or other hostile activities to ensuring the security of the communication system. Deep reinforcement learning (DRL) has been extensively utilized in solving the dynamic spectrum anti-jamming problem. However, most of existing DRL-based algorithms require lots of training time, which fails to adapt the fast-channging jamming environment. Our objective is to find a practical and fast-convergence anti-jamming learning solution. To achieve this, we redesign the DRL algorithm in the following two ways. First, we split the cycle of reinforcement learning into two parts: applying process and training process. Second, we use soft labels instead of rewards which bring more information. We further theoretically show that the information gain can help our proposed algorithm converge faster. Moreover, we also show that our labeled DRL algorithm is better than the idealized DRL-based scheme which can obtain the same information as the soft labels. Simulation results demonstrate that compared with existing DRL-based algorithms, our proposed algorithm reduces the number of iterations by up to 90%.

Abstract This paper investigates the joint relay and channel selection problem using a deep reinforcement learning (DRL) algorithm for cooperative communications in a dynamic jamming environment. The latest types of jammers include the mobile and smart jammer that contains multiple jamming patterns. This new type of jammer poses serious challenges to reliable communications such as huge environment states, tightly coupled joint action selections and real‐time decision requirements. To cope with these challenges, a DRL‐based relay‐assisted cooperative communication scheme is proposed. In this scheme, the joint selection problem is constructed as a Markov decision process (MDP) and a double deep Q network (DDQN) based anti‐jamming scheme is proposed to address the unknown and dynamic jamming behaviors. Concretely, a joint decision‐making network composed of three sub‐networks is designed and the independent learning method of each sub‐network is proposed. The simulation results show that the user agent is able to anticipate the jammer behaviors and elude the jamming in advance. Furthermore, compared with the sensing‐based algorithm, the Q learning‐based algorithm and the existing DRL‐based anti‐jamming approaches, the proposed algorithm maintains a higher average normalized throughput.