Sporadic ambient radio frequency signals can offer opportunistic spectrum and energy sources for backscatter communications, but they also induce unpredictable transmission interruptions in ambient backscatter wireless networks (AmBWNs). Integrating self-carrier-generative active transmissions with backscatter communications could significantly enhance transmission stability but require frequent mode switching to accommodate the ever-changing ambient radio frequency signals. However, this will result in frequent changes in network topology and link capacity, posing significant challenges in solving the network maximum flow problem in hybrid AmBWNs. To address this problem, we design a link-aware online maximum flow (OnMAXFlow) scheme to tackle agile and adaptive flow scheduling and communication mode selection. Specifically, we first employ an online learning framework to dynamically track changes in ambient signal strength and channel states, enabling real-time evaluation of link capacity. We then model the network maximum flow problem as a stochastic multi-armed bandit (MAB) problem and solve it with a Kullback-Leibler upper confidence bound (KL-UCB) algorithm. Our experimental evaluation results reveal that our OnMAXFlow scheme exhibits rapid convergence and superior adaptability against the varying network states, while maintaining spectrum efficiency and latency performance comparable to the Oracle scheme, which always selects the optimal transmission modes and paths.

In this paper, we consider a large-scale Wi-Fi HaLow heterogeneous network, where numerous stations (STAs) are distributed around an access point (AP) to collect and transmit data using the restricted access window (RAW) mechanism. The AP manages channel access by adjusting and broadcasting RAW Parameter Set (RPS) and grouping messages, which include the duration and slot allocation for each RAW group. We aim to maximize the overall network throughput while ensuring fairness among STAs. Traditional methods struggle with real-time RAW optimization in practical networks. To overcome this challenge, we first divide the RAW groups heuristically according to the STAs' task type and formulate the throughput optimization problem regarding various RPS. Then, we construct a virtual twin network environment to estimate the throughput performance used for RAW optimization. Specifically, the twin environment is built on neural networks and trained by both synthetic data generated from the classic Markov model and the NS-3 simulator. Given the throughput estimate, we devise the reward function of the proximal policy optimization (PPO) algorithm to adapt the optimal RPS, without frequent interaction with the real network environment. Numerical results indicate that the twin-enhanced PPO (TE-PPO) algorithm achieves a comparable performance with the classic PPO algorithm built on the real trace of the NS3 simulator. Particularly, TE-PPO can reduce the time overhead for RPS adaptation to 1/180.

Energy constraints significantly limit the long-term operation of underwater wireless sensor networks (UWSNs) due to their battery-powered sensor nodes (SNs). In this paper, we investigate an autonomous underwater vehicle (AUV)assisted UWSN where the AUV simultaneously collects data and recharges multiple independently located SNs. We employ the value of information (VoI) to evaluate the importance of the sensing data. We propose a novel Lyapunov-guided deep reinforcement learning (LDRL) algorithm to maximize the long-term average VoI while guaranteeing the battery energy constraints of SNs. We first employ a Lyapunov optimization to decompose the multi-stage stochastic VoI maximization problem into a series of single-stage deterministic subproblems. The Lyapunov drift-pluspenalty function is designed to deal with the long-term energy queue equilibrium. Then, we employ a deep neural network (DNN) to optimize the AUV's path and integrate a sequential convex approximation (SCA) optimization module for optimal charging time allocation. Experimental results demonstrate that our algorithm significantly enhances the long-term average VoI while ensuring the SNs' battery energy constraints.

This paper focuses on secure communications in UAV-assisted wireless sensing systems in the presence of a mobile UAV eavesdropper (MUE), without prior information about the traffic demands of ground users (GUs) in the sensing environment. To maximize secrecy performance, we propose a digital-twin enhanced proximal policy optimization (DTEPPO) algorithm that optimizes the GUs' sensing scheduling and the UAVs' trajectory planning and network formation. Unlike traditional reinforcement learning methods that require complete environment-interacted information, we build a digital twin (DT) model using Gaussian process regression (GPR) based on the historical sensing information collected by the UAVs. Moreover, DT model can be dynamically updated by exploiting the PPO algorithm to lean the UAVs' interactions with the environment, continuously enhancing its accuracy and providing a reliable virtual learning environment for fast network adaptation. Numerical simulations demonstrate that the proposed DTEPPO algorithm achieves rapid convergence and improved secure throughput with reduced communication overhead compared to conventional approaches. Moreover, the proposed framework enables simultaneous learning and modeling (SLAM) in an unknown environment, providing a general framework to solve complex network control problems in wireless and mobile systems with high costs of environmental interactions.

LoRa-based backscatter communication technology is promising in enabling ubiquitous connectivity for the Internet of Things (IoT) over large distances with extremely low power consumption. In this paper, we design and implement CPLoRa, a high-throughput parallel LoRa backscatter communication system compatible with commodity LoRa receivers. The core idea of CPLoRa is to enable multiple backscatter tags to communicate with remote LoRa receivers simultaneously by generating standard LoRa packets from a common single-tone RF emitter, which can be extracted from ambient LoRa transmitters or generated from dedicated mobile devices. CPLoRa employs a modified low-power direct digital synthesizer (DDS) scheme for precise frequency synthesis, ensuring compatibility with commodity LoRa receivers and enhancing data rates for long-range backscatter transmissions. Each tag is assigned a unique frequency offset in the synthesizer, allowing parallel transmissions and creating orthogonal, independent LoRa channels. Moreover, we design a harmonic-canceling switch network at the RF front end to reduce mutual interference among different tags. Finally, we implement the CPLoRa tag prototype using low-cost circuit components and rigorously tested in outdoor and indoor environments, demonstrating that CPLoRa supports long-range transmissions of up to 1000 meters while achieving a throughput of 9.6 kbps with 10 parallel tags compatible with commodity LoRa receivers.

Clustering user devices (UDs) with similar traffic flows enables more effective transmission scheduling and resource allocation in wireless networks, especially for Internet of Things (IoT) with dominant demands for machine-to-machine (M2M) data communications. In this paper, we propose an experience-driven spatial-temporal graph attention network (Exp-STGAN) for UDs’ clustering, by exploiting the spatial-temporal correlations from UDs’ historical traffic flows. Considering the UDs’ heterogeneity, we first characterize each UD’s out-going traffic flows by a dynamic radiation pattern, which reflects the UD’s spatial distribution of traffic demands and its targeted receivers in different directions. We aim to explore UDs’ clustering based on traffic flows’ radiation patterns and propose a spatial-temporal graph attention to aggregate information from different UDs correlated in space and time domains. Without true labels for the UDs’ clustering, we formulate a flow similarity metric based on Kullback-Leibler (KL) divergence to quantify the clustering performance. Moreover, to improve the learning efficiency, we integrate salient human experience into the graph attention module and also continuously update the experience during the training process. Experiments demonstrate that the Exp-STGAN framework can effectively cluster similar UDs by their dynamic traffic flows, highlighting the potential for flow-aware network performance maximization in large-scale IoT systems.

This paper investigates an unmanned aerial vehicle (UAV)-assisted semantic communication network. The energy-limited ground users (GUs) provide semantic services to periodically generated raw data and a UAV relays the extracted semantic information to a base station (BS). Semantic extraction enhances data responsiveness and reduces the age-of-information (AoI) by transmitting only the most essential information. However, more complex semantic extraction increases energy consumption, making it easier for the GUs to deplete their energy. Therefore, we introduce a novel energy-efficient AoI (EAoI) metric to capture both information freshness and energy consumption of the GUs. We formulate a time-averaged EAoI minimization problem by jointly optimizing the GUs' scheduling, pre-extraction strategy, semantic control, computing resource allocation, and the UAV's trajectory. We further propose a semantic-aware joint pre-extraction and trajectory planning (Sem-JPT) algorithm to decompose the complex optimization problem into three subproblems, which are solved by a series of approximation methods. Simulation results demonstrate that semantic communication can reduce the overall EAoI by more than 18% compared with conventional bit-based communication. Moreover, the proposed Sem-JPT algorithm can maintain information freshness and prolong the GUs' lifetimes, outperforming existing baselines.

In this paper, we investigate hybrid passive and active intelligent reflecting surface (IRS)-assisted secure non-orthogonal multiple access (NOMA) networks. Multiple users concurrently transmit sensitive data to an access point (AP) in the presence of an eavesdropper (Eve). The hybrid IRS is employed to enhance the NOMA users’ sum rates while simultaneously performing jamming beamforming against the Eve by optimizing the communication channels of NOMA users and injecting controllable noise into the Eve’s channel. We formulate a sum secrecy rate maximization problem by jointly optimizing the users’ scheduling policy, the hybrid IRS’s working mode and beamforming, and the AP’s receiving beamforming. To address combinatorial user scheduling and high-dimensional beamforming design, we develop a dual-cycling deep reinforcement learning (DRL) framework. We first determine the NOMA users’ scheduling strategy and the hybrid IRS’s working mode using a proximal policy optimization (PPO)-based learning algorithm. Then, we optimize the AP’s receiving beamforming and hybrid IRS’s beamforming strategies using an alternating optimization (AO) algorithm. The joint beamforming optimization can significantly enhance the DRL’s learning efficiency by limiting its action space. Moreover, we propose a lightweight two-phase algorithm with approximation techniques to reduce computational complexity by eliminating double-nested loops in AO, while maintaining secrecy performance close to optimum. Numerical results demonstrate that the proposed dual-cycling DRL scheme achieves 54.85% gains in the secrecy rate compared to traditional DRL schemes.

Millimeter wave (mmWave) communication has emerged as a key technology enabling ultra-low latency and high throughput in vehicular communication. Usually, an appropriate decision on user association requires timely channel information between vehicles and base stations (BSs), which is challenging given a fast-fading mmWave vehicular channel. In this paper, we propose a low-complexity semi-distributed contextual correlated upper confidence bound (SD-CC-UCB) algorithm to establish an up-to-date user association between vehicles and BSs without explicit measurement of channel state information (CSI). Under a contextual multi-arm bandits framework, SD-CC-UCB learns and predicts the transmission rate given the location and velocity of the vehicle, which can adequately capture the intricate channel condition for a prompt decision on user association. Further, SD-CC-UCB efficiently identifies the set of candidate BSs which probably support supreme transmission rates by leveraging the correlated distributions of transmission rates on different locations. To further refine the learning transmission rate to candidate BSs, each vehicle deploys the Thompson Sampling algorithm by taking the interference among vehicles and handover into consideration. Numerical results show that our proposed algorithm achieves the network throughput within 100%–103% of a benchmark algorithm which requires perfect instantaneous CSI, demonstrating the effectiveness of SD-CC-UCB in vehicular communications.

This paper investigates an unmanned aerial vehicle (UAV) assisted semantic network where the ground users (GUs) periodically capture and upload the sensing information to a base station (BS) via UAVs’ relaying. Both the GUs and the UAVs can extract semantic information from large-size raw data and transmit it to the BS for recovery. Smaller-size semantic information reduces latency and improves information freshness, while larger-size semantic information enables more accurate data reconstruction at the BS, preserving the value of original information. We introduce a novel semantic-aware age-of-information (SAoI) metric to capture both information freshness and semantic importance, and then formulate a time-averaged SAoI minimization problem by jointly optimizing the UAV-GU association, the semantic extraction, and the UAVs’ trajectories. We decouple the original problem into a series of subproblems via the Lyapunov framework and then use hierarchical deep reinforcement learning (DRL) to solve each subproblem. Specifically, the UAV-GU association is determined by DRL, followed by the optimization module updating the semantic extraction strategy and UAVs’ deployment. Simulation results show that the hierarchical structure improves learning efficiency. Moreover, it achieves low AoI through semantic extraction while ensuring minimal loss of original information, outperforming the existing baselines.

In this paper, we consider an aerial reconfigurable intelligent surface (ARIS)-assisted wireless network, where multiple unmanned aerial vehicles (UAVs) collect data from ground users (GUs) by using the non-orthogonal multiple access (NOMA) method. The ARIS provides enhanced channel controllability to improve the NOMA transmissions and reduce the co-channel interference among UAVs. We also propose a novel dual-mode switching scheme, where each UAV equipped with both an ARIS and a radio frequency (RF) transceiver can adaptively perform passive reflection or active transmission. We aim to maximize the overall network throughput by jointly optimizing the UAVs’ trajectory planning and operating modes, the ARIS’s passive beamforming, and the GUs’ transmission control strategies. We propose an optimization-driven hierarchical deep reinforcement learning (O-HDRL) method to decompose it into a series of subproblems. Specifically, the multi-agent deep deterministic policy gradient (MADDPG) adjusts the UAVs’ trajectory planning and mode switching strategies, while the passive beamforming and transmission control strategies are tackled by the optimization methods. Numerical results reveal that the O-HDRL efficiently improves the learning stability and reward performance compared to the benchmark methods. Meanwhile, the dual-mode switching scheme is verified to achieve a higher throughput performance compared to the fixed ARIS scheme.

LoRa is considered one of the most promising low-power wide-area techniques. Given that end devices (EDs) are typically battery-powered, energy efficiency (EE) is a critical factor to consider. In this paper, we aim to improve the system EE of the LoRa network by jointly allocating transmission parameters such as the channel (CH), transmission power (TP) and spreading factor (SF) for each ED. Owing to the low duty cycle and sporadic traffic of LoRa networks, evaluating the system EE under various parameter settings proves to be time-consuming. Consequently, we propose an analytical model aimed at calculating the system EE while fully considering the impact of multiple gateways, quasi-orthogonal SFs and capture effects. On this basis, we investigate a joint CH, SF and TP allocation problem to optimize the system EE for uplink transmissions. Given the NP-hard complexity of the problem, the original problem is decomposed into two subproblems: CH assignment and SF/TP assignment. First, a matching-based algorithm is introduced to tackle the CH assignment subproblem. Then, an attention-based multiagent reinforcement learning technique is employed to address the SF/TP assignment subproblem for EDs allocated to the same CH. The simulation outcomes indicate that the proposed approach converges quickly and obtains significantly better system EE than baseline algorithms.

Physical layer security issues have attracted significant attention in wireless networks to protect information leakage from illegitimate eavesdroppers. In this paper, we focus on an intelligent reflecting surface (IRS)-assisted secure non-orthogonal multiple access (NOMA) uplink system. Multiple users intend to transmit sensitive data to an access point (AP) considering the existence of a nearby eavesdropper (Eve). The IRS can be used to enhance the NOMA users’ sum rates while concurrently weakening the Eve’s channel condition, suppressing information leakage to the Eve without resorting to a cooperative jammer or the power injection of artificial noise in the system. The users’ scheduling, the IRS’s passive beamforming, and the AP’s receive beamforming are jointly optimized to maximize the secure rate of the IRS-assisted NOMA system. We develop a hierarchical deep reinforcement learning (DRL) framework to iteratively search for an optimal solution considering the combinatorial nature of the NOMA users’ scheduling and the high-dimensional beamforming design. Firstly, we search for the NOMA users’ scheduling strategy by using the PPO-based DRL algorithm. Given the NOMA scheduling strategy, we then optimize the active and passive beamforming strategies by the alternating optimization (AO) algorithm. The inner optimization helps evaluate the quality of the scheduling strategy and thus guides the outer-PPO algorithm to update a better scheduling strategy. Simulation results demonstrate the superiority of the proposed scheme over existing benchmarks, resulting in significant gains in secure rate.

This paper exploits multiple unmanned aerial vehicles (UAVs) to assist energy transfer, data uploading, and transmission in wireless networks, aiming to maximize the network's energy efficiency (EE). The inherent challenge of inaccessible or energy-intensive real-time information exchanges among UAVs results in undesirable delays in acquiring global network information. Such delayed information significantly hinders the transmission control and trajectory planning of the UAV s in multi-UAV-assisted wireless networks. To address this challenge, we propose a delay-tolerant multi-agent deep reinforcement learning (DT-MADRL) algorithm to jointly optimize the UAVs' trajectories and transmission control strategies based on randomly delayed information. In particular, we integrate a delay penalty term in the reward function that forces each UAV to have more regular information exchanges with the base station (BS). This ensures that each UAV can understand the real-time network environment, thereby reducing information delay and fostering more effective multi-agent collaboration. The simulation results reveal that our proposed algorithm reduces the UAVs' average information delay by 68% and improves overall EE by 28% compared to traditional MADRL algorithms.

Recently, ambient backscatter communication has become a promising technology to support the low-power and low-cost Internet-of-Things (IoT). However, the nondeterministic and sporadic nature of ambient signals makes it a great challenge when designing multiple access in spectrum opportunistic ambient backscatter wireless networks (AmBWNs). Moreover, the stringent energy supply and ultra-low-cost design of the backscatter transmitter make most multiple access schemes no longer suitable for AmBWNs. To effectively share carrier frequency resources for backscattering, we propose a hierarchical multiple access scheme, which allows beamforming based spatial division multiple access among groups, and non-orthogonal multiple access (NOMA) for multiple users access within a group. Consequently, we formulate a multi-objective optimization problem to balance the sum rate and the fairness by exploiting grouping, beamforming, and reflection coefficients. To solve this problem, we employ the matching theory to tackle the grouping problem and achieve the corresponding beamforming. We then reformulate the non-convex reflection coefficient optimization and solve it with successive convex approximation and geometric programming. Our extensive evaluation results demonstrate that the spectrum and energy efficiency, latency, and fairness can be significantly improved with minimal overhead at the transmitter.

Recently, energy-efficient ambient backscatter communication has emerged as a promising technology to build up self-sustainable wireless networks. However, the limited transmission range and rate restrict the communication capability of a single backscatter node. To ensure network coverage and connectivity, the nodes in ambient backscatter-aided wireless networks (AmBWNs) have to be densely deployed. Unfortunately, the fickle and sporadic nature of the energy source and carrier in AmBWNs would lead to unreliable and unstable transmissions. We argue that the network throughput can be significantly improved if we can take full advantages of both stable active transmission and energy-saving backscattering. However, it is challenging to jointly determine nodes' transmission mode to enhance the throughput and energy efficiency. Moreover, the fairness in terms of traffic load at each node of AmBWNs is not thoroughly considered in the existing research. The improper load allocation would hinder the network throughput and even cause network partition if some bottleneck nodes deplete the energy. In this article, a novel metric is defined to measure the aggregate energy efficiency of neighboring nodes, capturing the beneficial interaction of active links and backscatter links. Subsequently, we address the aggregate energy efficiency maximization problem constrained by Gini threshold. The Gini coefficient is used to adjust the load in proportion to the delivery capacity of each node, achieving proportional fairness in the dense AmBWN. The simulation result shows that the well-designed metric can improve the energy efficiency by 11 times and the throughput by 86%.

Ambient backscatter communication (AmBC) enables information delivery over an ambient RF signal without carrier generation and has emerged as a promising technology to build up the self- sustainable Internet-of-Things (IoT). However, when a strong ambient signal appears, multiple backscatter nodes may initiate data transmission simultaneously, causing severe contention and wasting the precious transmission opportunity. The nondeterministic and sporadic nature of ambient signals makes it a great challenge for efficient multiple access design in ambient backscatter aided wireless network (AmBWN). Moreover, the stringent energy supply and ultra-low-cost design of the backscatter transmitter makes most multiple access schemes no longer suitable for AmBWN. To fully share carrier resources for backscattering, we resort to the non-orthogonal multiple access (NOMA) to allow multiple devices in the same regime to transmit over an ambient signal with low latency. Moreover, we propose a hierarchical multiple access scheme, which allows beamforming based spatial division multiple access among groups, and NOMA for multiple users access within a group. The evaluation result shows latency and SINR can be significantly improved with minimal overhead at the transmitter.

Due to the extremely low power consumption, ambient backscatter communications has attracted great interest from both industry and research communities. However, the short communication range and unpleasant reliability are the two major challenges which prevent ambient backscatter from wide deployment in WSNs. In this work, we propose the design of a robust cooperative routing protocol (RCRP) for ambient backscatter based wireless sensor network (AmB-WSN), to account for the volatility of ambient RF environment. In RCRP, the backscatter sensor nodes (BSNs) work cooperatively to reduce the probability of routing path failure. To ensure robustness, we propose novel routing metrics to construct a robust counter-part for each nominal routing path, which are related to the strength of ambient RF signals and the BSNs' residual energy. Extensive simulations reveal that RCRP can achieve enhanced routing stability, improved throughput performance, and reduced end-to-end delay, which make it preferable and scalable for multi-hop AmB-WSN.