With the rapid development of reinforcement learning (RL)-enhanced anti-jamming wireless communication technologies and jamming technologies, intelligent communication confrontation has become an urgent problem to be solved. Most existing work assumed that detailed information of jammer was known in advance, which hardly holds in practice. Besides, some work was sensitive to the changing of jamming policy, leading to limited adaptability and scalability. This article extends the research to scenarios with unknown jammer and diverse jamming policies, including fixed, reactive, and deep RL (DRL)-based proactive jamming policies. The interaction between communication party and jammer is formulated as a partially observable adversarial team stochastic game (POATSG). To cope with unknown and diverse jamming policies, a prioritized fictitious rival play (PFRP)-based robust anti-jamming spectrum access scheme (RASAS) is proposed. First, a fictitious jammer is designed to force the communication party to promote robustness via adversarial training. Then, a synchronized update mechanism is adopted to mitigate the nonstationary issue. Finally, the fictitious agent pool is introduced to create diverse fictitious opponents and avoid overfitting. Simulation results show that the PFRP-based scheme is robust to the jamming policy, switching cycle of the jamming policy, and jamming channel number.

The environmental shelters such as public links can enable covert communication by covering covert transmission. To further exploit shelters, this paper focuses on a two-hop system where multiple pairs of public users and one pair of covert users communicate through a multi-antenna relay. We aim to improve covertness performance while satisfying the covertness constraints of two hops and quality of service (QoS) requirements of public users. The covert throughput maximization problem is formulated via jointly optimizing transmit power and beamformer, which is challenging to solve. We introduce successive convex approximation (SCA) and semidefinite relaxation (SDR) techniques to convert the problem into convex, where the joint optimization algorithm is developed. Considering the computational complexity, we further design a block diagonalization (BD) beamformer at the relay, which translates the beamformer optimization into a power allocation problem and derives the optimal solution in a closed form. We analytically show that the covert throughput first increases and then decreases as the number of public pairs increases in BD-based design, which has been verified numerically and can be generalized in other designs. Numerical results also evaluate the superiority of the joint optimization algorithm and the effectiveness of the BD-based efficient design. In particular, the BD-based design is very close to the joint optimization under small maximum transmit power of users or large maximum transmit power of relay.

This paper studies the anti-jamming channel selection problem in the unmanned aerial vehicle (UAV) communication scenario using machine learning. Recently, deep reinforcement learning (DRL) based anti-jamming approaches have drawn much attention, but most of them require lots of computing resources and power supply for training, which is impractical for the hardware-limited UAVs. What's more, the high complexity of DRL-based algorithms weakens their online learning ability, failing to rapidly adapt to the changing jamming environment. To be applicable to the hardware-limited UAVs, we propose a lightweight reinforcement learning algorithm based on the idea of spectrum state abstraction. We first assign similar spectrum states to clusters using the DRL and clustering algorithms. A state clustering network is deployed in the UAV to convert the large and redundant state space into a small number of state clusters. Based on the clustered states, the UAV uses a simple tabular Q-learning algorithm to online find the optimal anti-jamming policy. The simulation results show that, compared with the conventional DRL approach, the proposed algorithm can efficiently find the optimal anti-jamming policy and fast adapt to the change of jamming pattern in the complicated and dynamic jamming environment.

Neighbor discovery is a prerequisite to establish self-organizing IoT networks. IoT devices are mostly energy-limited, so neighbor discovery is usually implemented in a duty-cycled manner. A fundamental problem of optimizing energy efficiency of duty-cycled neighbor discovery is to minimize neighbor discovery latency for given duty cycle. We investigate purely interval (PI)-based neighbor discovery that adopts pseudo-random delay to avoid blocking problem, which is also adopted by pseudo-random delay advertising mode of Bluetooth low energy (BLE). In this article, we proposed an additional scan windows-based assisted protocol, which achieves early discovery with the help of assistance mechanism. Based on the assistance mechanism, we model and optimize the neighbor discovery latency of the PI-based protocol. Moreover, to analyze the impact of assistance failure caused by contention problem in a clique scenario, we proposed a mutual exclusive drawer model. Simulation results show that the proposed protocol using the derived near-optimal parameters achieves 26% lower average latency in sparse networks.

This paper investigates the problem of multidimensional resource management in multi-access mobile edge computing (MEC) networks against external dynamic jamming. The objective is to minimize the long-term computational cost of the MEC network while satisfying the task computation delay requirements of user equipment (UE) by jointly optimizing computing and communication resource allocation. To overcome challenges such as frequency conflict and dynamic jamming attacks, a distributed multi-agent hierarchical deep reinforcement learning (MAHDRL) MEC framework based on hybrid heterogeneous decision-making is proposed. Specifically, a hierarchical MEC anti-jamming data offloading optimization model is constructed, and the MEC resource management problem is formulated as a decentralized partially observable Markov decision process (Dec-POMDP). Based on this, a distributed MAHDRL algorithm based on the actor-critic (AC) model is designed to solve the multi-agent high-dimensional nonlinear hybrid integer programming NP-hard problem: the high-level network in the base station (BS) optimizes discrete channel access strategies, while the low-level network in UEs learns data offloading strategies. Additionally, the computational complexity is discussed and a theoretical proof of the algorithm convergence is presented. Simulation results demonstrate the superiority of the proposed algorithm, which reduces energy consumption and data processing delay across the network.

Intelligent reflective surfaces (IRSs) as low energy consumption and easy to attach devices have been widely applied in the field of anti-jamming recently. In particular, the combination of IRS and unmanned aerial vehicle (UAV), as IRS-UAV, further expands the scope of IRS services. In this article, the joint IRS selection and beamforming optimization problem has been investigated in multiple IRS-UAV-assisted anti-jamming D2D networks. To solve the above optimization problem, a distributed matching-based selection and$Q$-learning-based beamforming optimization algorithm (DMQ) was proposed. In detail, the optimization problem is decomposed into two subproblems, namely, the IRS selection subproblem is formulated as a noncommutative many-to-many matching game model to describe peer effects and uncertainty selection quotas, and the passive beamforming optimization subproblem is solved by a reinforcement algorithm to satisfy the complex environment. Numerical simulations confirm the convergence and near-optimal performance of the proposed scheme with lower latency and greater robustness.

The cover image is based on the Research Article Joint Channel and Power Optimization for Multi-user Antijamming Communications: A Dual Mode Q-learning Approach by Xiaobo Zhang et al., https://doi.org/10.1049/cmu2.12339

Abstract In view of the anti‐tracking‐jamming problem, traditional online learning methods usually cannot analyse the jamming behaviour, and find an effective way to prevent the jamming attacks. To cope with these challenges, a novel communication/deception dual mode mechanism is proposed in this paper. Deception users are selected to send high‐power signal for jamming attraction, and form collaborative relationships with communication users. The corresponding collaborative anti‐jamming model is then constructed as a Markov game to analyse the multi‐agent decision. Based on that, a joint channel and power optimisation for multi‐user anti‐jamming communications based on dual mode Q‐learning scheme is proposed. Compared with two traditional online learning algorithms, the proposed DCAJ‐QL algorithm effectively achieves 146.5% and 80.4% higher maximum communication rate under tracking jamming conditions, and achieves 40.7% and 53.6% higher maximum communication rate under fixed jamming conditions.

In this paper, we study the problem of network reconfiguration when unmanned aerial vehicle (UAV) swarms suffer damage. Multiple UAVs are divided into several groups to perform various tasks. Each master UAV is connected to the ground control station and provides network services for small UAVs that perform various tasks, ensuring that the information of small UAVs can be transmitted back in a timely manner. When master UAVs are destroyed due to factors such as jamming or attacks, the associated small UAVs must select new master UAVs for network service and cooperate with other small UAVs to execute tasks. Based on the heterogeneity and relevance of tasks, we model and analyze the task relationship among different UAVs. Since both master UAVs and small UAVs have respective optimization objectives in the network reconfiguration process, we construct a many-to-one bilateral matching market to model the interaction between master UAVs and small UAVs. To realize an efficient solution for UAV network reconfiguration in complex environments, we propose a distributed matching algorithm and prove that the algorithm can converge to two-sided stable matching. Simulation results indicate that the proposed algorithm can significantly improve the task completion degree of the network compared with three other algorithms.

This paper investigates the anti-jamming problems for multi-user scenarios. On the one hand, users in the networks should coordinate their channel selection strategies to avoid spectrum conflicts among different users. On the other hand, because of the openness characteristic of wireless communications, malicious jammers can disrupt the legitimate communications of legitimate users by sending jamming signals, thus users also need to fully consider how to defend against malicious jamming attacks. To cope with the internal coordination and external confrontation problem, a context-aware dynamic spectrum coordinated anti-jamming approach is proposed. In detail, the multiuser anti-jamming scenario is modeled as an anti-jamming local altruistic game, and the existence of Nash Equilibrium (NE) is demonstrated. Besides, the proposed game model is proved to be an exact potential game. To obtain NEs, a multi-pattern stochastic learning algorithm (MSLA) is designed. Through local information exchange and distributed learning, users can achieve global optimization under the dynamic jamming environment.

Abstract This paper investigates the problem of multi‐user anti‐jamming channel access with partially overlapping channels (POC). Compared with traditional anti‐jamming systems that use non‐overlapping channels, POC improve the spectral efficiency. However, the partial overlap of channels also brings more serious interference. For this, physical distance and channel separation on the interference intensity under partially overlapping channels are first considered and the malicious jamming and interference among users are formulated as a hierarchical binary model. Secondly, to cope with multi‐user decisions under dynamic jamming conditions, the Markov game framework is adopted to analyse the problem. Thirdly, a multi‐user collaborative anti‐jamming channel selection algorithm based on reinforcement learning is proposed as well as the optimal anti‐jamming strategy can be obtained. Finally, the simulation results validate that the proposed algorithm helps users cope with jamming and eliminate mutual interference. Compared with the non‐overlapping channel access scheme, the POC access scheme achieves higher network throughput.

This paper investigates the anti-jamming problems in wireless communication networks. In these networks consisted of multiple devices (users), there exist two critical problems. On the one hand, users with various transmission requirements should coordinate their channel selection strategies distributedly to avoid spectrum conflicts and satisfy transmission demands. On the other hand, they also need to fully consider how to eliminate the effects of malicious attacks. To cope with the internal coordination and external confrontation challenges and accommodate the dynamic changing jamming attacks, a context-aware distributed coordinated anti-jamming channel access mechanism is proposed, which means for different cases of jamming attacks, different access strategies are adopted. In detail, to reflect the heterogeneous communication demands of users, the transmission satisfaction function is firstly introduced. Then, the multi-user anti-jamming scenario is modeled as a context-aware multi-pattern dynamic anti-jamming game, which can be decomposed into two sub-games. Here, for the case that the control channel is available, a local altruistic sub-game is introduced. While for the case that the control channel has been jammed, an anti-jamming congestion sub-game is designed. Besides, the existence of Nash Equilibriums is demonstrated. To obtain NEs, a context-aware distributed channel access (CDCA) algorithm is designed. Through game-theoretic analysis and distributed learning, global transmission satisfaction can be improved under the dynamic jamming environment. Furthermore, the fairness of the network can also be guaranteed.

This paper mainly investigates the unmanned aerial vehicle (UAV) communication networks under the threat of a static malicious jammer. While taking the flying process of the user (UAV transmitter-receiver pair) into consideration, we propose a joint power and trajectory optimization method. Moreover, a Stackelberg framework is formulated to solve the proposed optimization problem. In addition, a joint power and trajectory optimization algorithm (JPTOA) based on best-response (BR) is designed to obtain the user's strategy as well as Stackelberg equilibrium (SE) in each time stage. Furthermore, as an extension of single stage optimization, the I-step exploration and one-step decision (IEOD) scheme is designed to enhance the user's cumulative utility. Finally, simulation results are presented to show the performance of the proposed JPTOA scheme.