Semantic-aware Vehicular Edge Computing (VEC) has emerged as a novel paradigm to significantly reduce transmission costs and edge resource consumption by offloading extracted task-driven semantic information. However, excessive semantic extraction may impose additional computational workload. In the face of unknown environmental dynamics, the semantic extraction ratio must be jointly designed with task offloading for resource-constrained VEC. To this end, we conceive a multiple-objective (MO) semantic-aware task offloading framework for VEC by jointly optimizing semantic extraction ratio, transmit power and task scheduling strategies aimed at minimizing both long-term age-of-information (AoI) and energy consumption while guaranteeing queue stability. Subsequently, we propose a Lyapunov-guided multi-objective reinforcement learning (MORL)-based semantic-aware joint task scheduling and resource allocation (SJTSRA) solution. Specifically, Lyapunov optimization method is first leveraged to transform the original problem into a multi-objective Markov decision process (MOMDP). Then, the concave-augmented Pareto Q-learning (CAPQL) algorithm is employed to find Pareto optimal solutions through introducing uniform weight sampling and entropy regularization, where the convergence can be guaranteed theoretically. Simulation results show that the proposed solution achieves the closest approximation to the Pareto front with the highest hypervolume, and superior energy-AoI trade-offs across varying environment parameters compared to all benchmarks.

The Unmanned Aerial Vehicles (UAVs) communication faces challenges arising from scarce spectrum resources and malicious jamming. This paper proposes a spectrum waterfall (SW)-assisted multi-agent anti-jamming framework for UAV swarms by designing joint resource allocation and trajectory optimization (JRATO) strategies. By formulating the problem as a decentralized partially observable parameterized-action Markov Decision Process (Dec-POPAMDP), we first employ a self-attention-based convolutional neural network (CNN) to extract spatiotemporal SW knowledge, and then propose a multiagent hybrid Proximal Policy Optimization (MA-HPPO) based anti-jamming scheme to maximize the long-term utility-cost trade-off. Simulation results show that the proposed scheme outperforms the benchmarks in terms of both the convergence and the long-term utility-cost trade-off, while achieving higher success rate with lower energy consumption with varying numbers of channels.

This paper considers utilizing the knowledge of age gains to reduce the average age of information (AoI) in random access with event-driven periodic updating for the first time. Built on the form of slotted ALOHA, we require each device to determine its age gain threshold and transmission probability in an easily implementable decentralized manner, so that the contention can be limited to devices with age gains as high as possible. For the basic case that each device utilizes its knowledge of age gain of only itself, we provide an analytical modeling by a multi-layer discrete-time Markov chains (DTMCs), where an external DTMC manages the jumps between the beginnings of frames and an internal DTMC manages the evolution during an arbitrary frame, for obtaining optimal fixed access parameters offline. For the enhanced case that each device utilizes its knowledge of age gains of all the devices, we require each device to adjust its access parameters for maximizing the estimated network expected AoI reduction per slot, through maintaining the a posteriori joint probability distribution of local age and age gain of an arbitrary device in a Bayesian manner. Numerical results validate our study and demonstrate the advantage of the proposed schemes over other schemes.

Vehicular edge computing (VEC) has driven the proliferation of computation-intensive and delay-sensitive vehicular services by deploying computing and energy resources at the edge. However, the exploitation of edge resources faces challenges due to unpredictable environmental dynamics and partial observability. To this end, this paper investigates the computing energy efficiency (CEE) problem in twin-timescale VEC scenarios by dynamically adjusting the offloading policy. Based upon modeling the problem as a decentralized partially observable Markov decision process (Dec-POMDP), a pair of non-cooperative and cooperative offloading solutions are proposed relying on multi-agent reinforcement learning (MARL), respectively. Specifically, the non-cooperative solution employs multi-agent independent proximal policy optimization (IPPO) to enable vehicular user equipments (VUEs) to learn their policies in a fully distributed manner without any information sharing. By contrast, the cooperative solution combines the multi-agent shared PPO with graph attention networks (MAPPO-GAT), where the relationship among agents is learned cooperatively and the historical learning experience is shared. Additionally, we compare the computational complexity and analyze the convergence. Simulation results show that in terms of the trade-off between offloading delay and offloading energy consumption, the proposed cooperative solution is superior to the non-cooperative counterpart with the cost of moderate training overhead for cooperative learning.

In the face of increasingly computing-intensive and delay-sensitive vehicular applications, vehicular edge computing (VEC) has become a promising computing paradigm by deploying computing resources at the edge. This paper investigates an age of information (AoI)-aware vehicular edge offloading problem by dynamically adjusting the edge offloading ratio and selecting the VEC server, taking into account the computing energy efficiency (CEE). To adapt to the time-varying network topology of VEC, we propose a multi-agent cooperative edge offloading solution relying on actor-attention-critic framework, where each vehicular user equipment (VUE) employs an attention mechanism to regulate its attention to other VUEs, facilitating selective focus on important information to enhance policy learning. The simulation results show that the proposed solution can achieve a more compelling trade-off between AoI and CEE compared with the baseline solutions.

The covert communication technology has emerged as a novel method for network authentication, copyright protection, and providing the evidence of cybercrimes. However, how to design the covert communication scheme in the physical layer of wireless networks and how to optimize the data rate for the covert communication channels are very challenging. In this article, we propose a wireless covert communication system (CCS), where the transmit antennas are selected and coded to generate a covert codebook. According to the covert codebook, the antennas can be dynamically combined to transmit different covert messages. In addition, we adopt a modulation scheme, named covert quadrature amplitude modulation (QAM), to modulate the covert messages, where the precoding method is designed to deviate the constellations for covert information bits from those for the public information bits. Furthermore, we derive the closed-form expressions of capacity and bit error ratio (BER) for the proposed CCS. To maximize the covert data rate of the CCS, we formulate an optimization problem of the covert data rate and solve the problem to find the optimal precoding matrix. To reduce the covert information leakage, artificial noise is introduced to the system to jam the communication between the transmitting and watching nodes. We design a beamforming scheme to maximize the secure rate for the CCS, where the leakage of covert information can be minimized while the covert communication is not influenced. Simulation results show that the proposed CCS can significantly improve the covert data rate and reduce the covert BER in comparison with the traditional CCSs.

Designing efficient random access is a vital problem for urgency-constrained packet delivery in uplink Internet of Things (IoT), which has not been investigated in depth so far. In this paper, we focus on unpredictable frame-synchronized traffic, which captures a number of scenarios in IoT communications, and generalize prior studies on this issue by considering a general ALOHA-like protocol, a general single-packet reception (SPR) channel, urgency-dependent throughput (UDT) based on a general urgency function, and the dynamic programming optimality. With a complete knowledge of the number of active users, we use the theory of Markov Decision Process (MDP) to explicitly obtain optimal policies for maximizing the UDT, and prove that a myopic policy is in general optimal. With an incomplete knowledge of the number of active users, we use the theory of Partially Observable MDP (POMDP) to seek optimal policies, and show that a myopic policy is in general not optimal by presenting a counterexample. Because of the prohibitive complexity to obtain optimal or near-optimal policies for this case, we propose two practical policies that utilize the inherent property of our MDP framework and channel model. Simulation results show that both outperform other alternatives. The robustness under relaxed system settings is also examined.

In this paper, we investigate the energy-efficiency (EE) problem in edge association for heterogeneous Internet of Vehicles (IoV), when the dynamic environmental information can not be known in advance. Aiming to maximize the long-term tradeoff between EE and handover (HO) overhead, we propose a cooperative multi-agent edge association solution, where vehicular user equipments (VUEs) make decisions cooperatively relying on their local observations under centralized training. Specifically, we first construct a multi-agent partially observable Markov decision process (MA-POMDP) problem and decompose the system value function into the local value functions for implicit individual learning. Next, through sharing learning experience and approximating the global state, each VUE is able to obtain its own optimal/suboptimal policy given its local observations and historical information. Simulation results show that the proposed solution outperforms the non-cooperative counterpart and other baselines in terms of improving EE with the most appropriate number of HOs.

In covert communication systems, covert messages can be transmitted without being noticed by the monitors or adversaries. Therefore, the covert communication technology has emerged as a novel method for network authentication, copyright protection, and the evidence of cybercrimes. However, how to design the covert communication in the physical layer of wireless networks and how to improve the channel capacity for the covert communication systems are very challenging. In this paper, we propose a wireless covert communication system, where data streams from the antennas of the transmitter are coded according to a code book to transmit covert and public messages. We adopt a modulation scheme, named covert quadrature amplitude modulation (QAM), to modulate the messages, where the constellation of covert information bits deviates from its normal coordinates. Moreover, the covert receiver can detect the covert information bits according to the constellation departure. Simulation results show that proposed covert communication system can significantly improve the covert data rate and reduce the covert bit error rate, in comparison with the traditional covert communication systems.

Vehicular edge computing (VEC) has become a promising enabler for ultrareliable and low-latency communications (URLLC) vehicular networks by providing computational resources for task offloading. In this article, we investigate an online task offloading problem for heterogeneous VEC (HVEC) network in the face of unknown environment dynamics. To overcome the unavailability of state information, we aim for minimizing the expectation of total offloading energy consumption while satisfying stringent delay requirements by learning the relationship between historical observations and rewards. Hence, this problem constitutes a contextual multiarmed bandit (MAB) problem. By grouping users according to their task preferences, we propose a contextual clustering of bandits-based online vehicular task offloading (CBTO) solution, which is aware of the task popularity. Simulation results reveal that the proposed solution outperforms other contextual and context-free benchmarkers in terms of both offloading energy consumption and delay performance.

In Internet of Vehicles (IoV), information freshness is a significant indicator to indemnify road traffic safety, which is measured by Age of Information (AoI). In this article, we consider the coexistence scenario of vehicular user pairs and cellular users, where the base station (BS) acts as an agent to allocate channels and transmit power for vehicular user pairs. With the goal of minimizing the sum of the average AoI of all links and the average power consumption of all vehicular user pairs, we formulate this optimization problem as a discrete-time Markov decision process (MDP) problem and adopt the trust region policy optimization (TRPO) algorithm, which has the advantage of fast convergence and high stability. Then, an AoI-aware joint spectrum and power dynamic allocation scheme based on the TRPO algorithm is proposed. Simulation results show that the TRPO-based scheme significantly outperforms both the deep$Q$network (DQN)-based scheme and the random scheme in terms of average cumulative reward, convergence speed, and stability.

In this article, we investigate the anti-jamming problem with joint channel and power allocation for unmanned aerial vehicle (UAV) networks. In particular, we focus on avoiding both mutual interference among UAVs and external malicious jamming to maximize the system Quality of Experience (QoE) relevant to the power consumption. To simultaneously capture the competition and coordination among UAVs, we first model the problem as a local interaction Markov game and then prove it as an exact potential game with at least one Nash equilibrium. Next, we propose a collaborative multiagent layered Q learning (MALQL)-based anti-jamming communication algorithm to reduce the high dimensionality of the action space and analyze the asymptotic convergence of the proposed algorithm. Simulation results show the effectiveness of the proposed algorithm, which outperforms the traditional multiagent$Q$learning algorithm when suffering from different jamming strategies.

Reconfigurable intelligent surfaces (RISs) are envisioned to be a disruptive wireless communication technique that is capable of reconfiguring the wireless propagation environment. In this paper, we study a free-space RIS-assisted multiple-input single-output (MISO) communication system in far-field operation. To maximize the received power from the physical and electromagnetic nature point of view, a comprehensive optimization, including beamforming of the transmitter, phase shifts of the RIS, orientation and position of the RIS is formulated and addressed. After exploiting the property of line-of-sight (LoS) links, we derive closed-form solutions of beamforming and phase shifts. For the non-trivial RIS position optimization problem in arbitrary three-dimensional space, a dimensional-reducing theory is proved. The simulation results show that the proposed closed-form beamforming and phase shifts approach the upper bound of the received power. The robustness of our proposed solutions in terms of the perturbation is also verified. Moreover, the RIS significantly enhances the performance of the mmWave/THz communication system.

Technically, the security performance of generalized spatial modulation (GenSM) networks can be enhanced by dynamically adjusting the precoder allocated to the legitimate signal as communication channel varies. For this purpose, our paper proposes a secure transmission strategy upon designing both digital and analog precoders for hybrid GenSM systems, where an eavesdropper is taken into account. The concept of the hybrid GenSM system has arose to improve the spatial multiplexing (SMX) gain for remedying the shortcoming of the limited number of radio frequency chains in traditional GenSM systems. However, this may lead to a great deal of security degradation since the SMX gain of the unintended receiver will be also improved. To this end, we develop a secrecy enhancement scheme by devising both analog and digital precoders for hybrid GenSM networks. Specifically, we derive an efficiently closed-form alternative to the original secrecy rate (SR) expression for reducing the excessive computational complexity of the joint optimization problem over the hybrid precoder. Then, by using this alternative as our cost function, an iterative algorithm is proposed. In particular, we elaborately conceive a pair of concave maximization problems in order to optimize the digital and analog precoders, respectively. Our proposed strategy not only utilizes semi-positive definite relaxing technique over the analog precoder but also invokes a lower bound of the alternative to further simplify the optimization over the vectored digital precoder. Subsequently, both the convergence and computational complexity of the proposed alternating iteration algorithm are analyzed. Compared to existing designs, our proposed strategy strikes a compelling role in balancing the SR performance and complexity. Finally, our simulation results confirm the efficiency of the proposed algorithm in terms of the SR performance achieved.

The security of spatial modulation (SM) aided networks can always be improved by reducing the desired link's power at the cost of degrading its bit error ratio performance and assuming the power consumed to artificial noise (AN) projection (ANP). We formulate the joint optimization problem of maximizing the secrecy rate (Max-SR) over the transmit antenna selection and ANP in the context of secure SM-aided networks. In order to solve this problem, we provide a pair of solutions, namely joint and separate solutions. Specifically, an accurate approximation of the SR is used for reducing the computational complexity, and the optimal AN covariance matrix (ANCM) is found by convex optimization for any given active antenna group (AAG). Then, given a large set of AAGs, simulated annealing mechanism is invoked for optimizing the choice of AAG, where the corresponding ANCM is recomputed by this optimization method as well when the AAG changes. To further reduce the complexity of the above-mentioned joint optimization, a low-complexity two-stage separate optimization method is also proposed. Moreover, when the number of transmit antennas tends to infinity, the Max-SR problem becomes equivalent to that of maximizing the ratio of the desired user's signal-to-interference-plus-noise ratio to the eavesdropper's. Thus, our original problem reduces to a fractional programming problem and a significant computational complexity reduction can be achieved. Finally, our simulation results verify the efficiency of the proposed methods in terms of the SR performance attained.

With an unprecedented amount of sensitive private data generated by mobile user equipment (UE), securing the emerging ultra-dense networks (UDNs) becomes critical. Although involving more access points (APs) is potentially capable of enhancing both the UE's throughput and security, the energy consumption becomes significant. In this paper, we investigate secure UDNs in the context of the user-centric clustering of UDNs from a secrecy energy efficiency perspective, while satisfying both the throughput and the security of each UE. We first propose a secure user-centric clustering architecture by introducing both a dedicated jamming strategy and an embedded jamming strategy, both of which degrade the overheard signals of the eavesdroppers and guarantee secure transmission relying on the different APs' involvement status. We formulate the secure user-centric clustering design for both known and unknown eavesdropper channel state information (CSI), whilst maximizing the secrecy energy efficiency with the aid of various secure transmission schemes. Since the problem formulated is a non-convex mixed integer non-linear programming problem, we develop a set of heuristic greedy secure user-centric clustering algorithms for diverse operating scenarios. Finally, our numerical results reveal the quantitative benefits of the proposed secure user-centric clustering architectures as a function of the network densities (i.e., AP, UE, and eavesdropper) and of both the throughput and the security constraints on the secrecy energy efficiency trade-off in different scenarios.

When massive MIMO technology is combined with dense heterogeneous networks (HetNets), the user association and power allocation problems are fundamentally different although the energy- efficiency benefits can be intensified. This paper aims to investigate the energy efficient joint user association and power allocation problem in downlink massive MIMO empowered dense HetNets under proportional fairness criterion. The joint optimization problem is a non-convex mixed-integer nonlinear program (MINLP) which is NP-hard, and hence it is difficult to efficiently obtain exact solution. In order to obtain the highquality suboptimal solution, the joint optimization problem is first decomposed into two subproblems with alternating iterative method. Then a two-layer iterative suboptimal algorithm is proposed to solve the joint optimization problem with guaranteed convergence. The involved association subproblem adopts dual decomposition to achieve the optimal association index, whilst the power allocation subproblem allocates the transmit power of each BS with Newton's method. Numerical results verify the effectiveness of our proposed algorithm and show that our proposed algorithm outperforms conventional association schemes in the enhancement of energy efficiency performance. Furthermore, it can be seen that the energy efficiency performance is enhanced by increasing the number of antennas at macro base station (MBS).