In this paper, we consider a large-scale Wi-Fi HaLow heterogeneous network, where numerous stations (STAs) are distributed around an access point (AP) to collect and transmit data using the restricted access window (RAW) mechanism. The AP manages channel access by adjusting and broadcasting RAW Parameter Set (RPS) and grouping messages, which include the duration and slot allocation for each RAW group. We aim to maximize the overall network throughput while ensuring fairness among STAs. Traditional methods struggle with real-time RAW optimization in practical networks. To overcome this challenge, we first divide the RAW groups heuristically according to the STAs' task type and formulate the throughput optimization problem regarding various RPS. Then, we construct a virtual twin network environment to estimate the throughput performance used for RAW optimization. Specifically, the twin environment is built on neural networks and trained by both synthetic data generated from the classic Markov model and the NS-3 simulator. Given the throughput estimate, we devise the reward function of the proximal policy optimization (PPO) algorithm to adapt the optimal RPS, without frequent interaction with the real network environment. Numerical results indicate that the twin-enhanced PPO (TE-PPO) algorithm achieves a comparable performance with the classic PPO algorithm built on the real trace of the NS3 simulator. Particularly, TE-PPO can reduce the time overhead for RPS adaptation to 1/180.

In this paper, we consider a large-scale Wi-Fi HaLow heterogeneous network in a real-world Internet of Things (IoT) environment, where numerous devices are distributed around an access point (AP). These devices collect and transmit data using the restricted access window (RAW) mechanism. They exhibit varying traffic characteristics. Moreover, dependencies among the devices often exist. We aim to maximize the overall throughput by adjusting the RAW grouping decision. The heterogeneity of real network data and dependencies between devices make the RAW grouping process more complicated. To overcome this challenge, we propose a novel traffic-driven RAW grouping approach. Specifically, we build a simulation environment based on real IoT data and NS-3. Then, we analyze the traffic characteristics of each IoT device. This analysis allows us to fully explore the dependencies and cooperation relationships among these devices. Hence, we can aggregate devices with these relationships into clusters. Each cluster is then treated as a supernode which is used as a basic unit for RAW grouping. Then, we use proximal policy optimization (PPO) algorithm to optimize the RAW grouping process via interacting with the environment. Numerical results indicate that the proposed traffic-driven algorithm significantly achieves faster convergence and improves grouping efficiency in large-scale heterogeneous networks compared to baselines.

This paper investigates an unmanned aerial vehicle (UAV)-assisted semantic communication network. The energy-limited ground users (GUs) provide semantic services to periodically generated raw data and a UAV relays the extracted semantic information to a base station (BS). Semantic extraction enhances data responsiveness and reduces the age-of-information (AoI) by transmitting only the most essential information. However, more complex semantic extraction increases energy consumption, making it easier for the GUs to deplete their energy. Therefore, we introduce a novel energy-efficient AoI (EAoI) metric to capture both information freshness and energy consumption of the GUs. We formulate a time-averaged EAoI minimization problem by jointly optimizing the GUs' scheduling, pre-extraction strategy, semantic control, computing resource allocation, and the UAV's trajectory. We further propose a semantic-aware joint pre-extraction and trajectory planning (Sem-JPT) algorithm to decompose the complex optimization problem into three subproblems, which are solved by a series of approximation methods. Simulation results demonstrate that semantic communication can reduce the overall EAoI by more than 18% compared with conventional bit-based communication. Moreover, the proposed Sem-JPT algorithm can maintain information freshness and prolong the GUs' lifetimes, outperforming existing baselines.

This paper investigates an unmanned aerial vehicle (UAV) assisted semantic network where the ground users (GUs) periodically capture and upload the sensing information to a base station (BS) via UAVs’ relaying. Both the GUs and the UAVs can extract semantic information from large-size raw data and transmit it to the BS for recovery. Smaller-size semantic information reduces latency and improves information freshness, while larger-size semantic information enables more accurate data reconstruction at the BS, preserving the value of original information. We introduce a novel semantic-aware age-of-information (SAoI) metric to capture both information freshness and semantic importance, and then formulate a time-averaged SAoI minimization problem by jointly optimizing the UAV-GU association, the semantic extraction, and the UAVs’ trajectories. We decouple the original problem into a series of subproblems via the Lyapunov framework and then use hierarchical deep reinforcement learning (DRL) to solve each subproblem. Specifically, the UAV-GU association is determined by DRL, followed by the optimization module updating the semantic extraction strategy and UAVs’ deployment. Simulation results show that the hierarchical structure improves learning efficiency. Moreover, it achieves low AoI through semantic extraction while ensuring minimal loss of original information, outperforming the existing baselines.

This paper exploits multiple unmanned aerial vehicles (UAVs) to assist energy transfer, data uploading, and transmission in wireless networks, aiming to maximize the network's energy efficiency (EE). The inherent challenge of inaccessible or energy-intensive real-time information exchanges among UAVs results in undesirable delays in acquiring global network information. Such delayed information significantly hinders the transmission control and trajectory planning of the UAV s in multi-UAV-assisted wireless networks. To address this challenge, we propose a delay-tolerant multi-agent deep reinforcement learning (DT-MADRL) algorithm to jointly optimize the UAVs' trajectories and transmission control strategies based on randomly delayed information. In particular, we integrate a delay penalty term in the reward function that forces each UAV to have more regular information exchanges with the base station (BS). This ensures that each UAV can understand the real-time network environment, thereby reducing information delay and fostering more effective multi-agent collaboration. The simulation results reveal that our proposed algorithm reduces the UAVs' average information delay by 68% and improves overall EE by 28% compared to traditional MADRL algorithms.

In this paper, we consider a multiple unmanned aerial vehicles (UAVs)-assisted wireless sensing network, where low-power ground users (GUs) periodically sense the environmental information and upload the recent sensing information to a base station (BS). The GUs firstly backscatter their information to the UAVs and then the UAVs transmit the information to the BS by the non-orthogonal multiple access (NOMA) transmissions. Our goal is to minimize the long-term age-of-information (AoI) by jointly optimizing the UAV's sensing scheduling, transmission control, and trajectories. To solve this problem, we propose the Lyapunov-driven hierarchical proximal policy optimization framework, named Lya-HPPO, to decouple the multi-stage AoI minimization problem into several control subproblems. In each control subproblem, the UAVs' sensing scheduling and transmission control are firstly determined by the outer-loop deep reinforcement learning (DRL) approach, and then the inner-loop optimization module is to update the UAVs' trajectories. Simulation results verify that the proposed Lya-HPPO framework converges very fast to a stable value and can make online decisions in real time, while guaranteeing the long-term data buffer and AoI stability.

In this paper, we employ dual-mode unmanned aerial vehicles (UAVs) equipped with both the active radio frequency (RF) module and aerial reconfigurable intelligent surface (ARIS) to assist ground users (GUs) for both the downlink energy transfer and uplink data transmission in a wireless-powered network. To maximize the GUs' minimum throughput, we propose a collaborative mode switching scheme for the dual-mode UAVs to dynamically switch between the active RF and passive ARIS modes according to the time-varying channel conditions. Besides, we jointly optimize the GUs' transmission control, the UAVs' beamforming, and the trajectory planning strategies. This optimization problem is intractable directly due to the non-convexity in both the objective and constraints. We design an iterative algorithm to first decompose the original problem into several subproblems, and then solve each subproblem individually by approximate optimization methods. Numerical results verify that the UAVs' collaborative mode switching along with their trajectories efficiently improves the transmission performance compared to the benchmarks in which both UAVs are operating in one fixed mode.

In this paper, we employ multiple unmanned aerial vehicles (UAVs) to assist sensing data transmission from the ground users (GUs) to the remote base station (BS). Each UAV can first cache the sensing data and then report the cached data to the BS. We consider a time-slotted protocol to coordinate the UAVs' data collection and reporting. Only one UAV is allowed to forward its data to the BS in each time slot. We formulate a multi-stage stochastic optimization problem to minimize the longterm age-of-information (AoI) by jointly optimizing the UAVs' trajectories and scheduling strategies. To simplify this problem, we model the dynamics of the UAVs' data buffer and AoI statuses by queueing systems, and propose a novel AoI-aware Adaptation scheme. This scheme allows us to transform the multistage dynamic programming problem into per-slot scheduling and trajectory planning sub-problems by using the Lyapunov optimization framework. Then, in each time slot, we can update the UAVs' scheduling and flying strategies in an iterative manner according to the instant buffer and AoI statuses. Simulation results show that the proposed scheme outperforms baseline schemes in terms of reducing AoI while stabilizing and balancing the UAVs' data queues.

In this paper, we consider an IRS-assisted and wireless-powered mobile edge computing (MEC) system that allows both edge users and the IRS to harvest energy from the hybrid access point (HAP), co-located with the MEC server. Each edge user uses the harvested energy to offload its data to the MEC server. The IRS not only assists downlink energy transfer to the edge users, but also improves the users' uplink offloading rates. To minimize the overall energy consumption, we jointly optimize the users' offloading decisions, the HAP's active beamforming, as well as the IRS's energy harvesting and passive beamforming strategies. The energy minimization problem is intractable due to complicated couplings in both the objective function and constraints. We decompose this problem into the downlink energy transfer and the uplink data offloading phases. The uplink phase can be efficiently optimized by the conventional semi-definite relaxation (SDR) method, while the downlink phase depends on the alternating optimization between the users' offloading decisions and the joint active and passive beamforming strategies. Numerical results demonstrate that the proposed offloading scheme can significantly reduce the HAP's energy consumption compared with typical benchmarks.

In this paper, we consider a hybrid computation offloading scheme that allows edge users to offload workloads to the edge servers by using active RF communications and backscatter communications. We aim to maximize the overall energy efficiency by jointly optimizing the beamforming of access point (AP) and the users’ offloading decisions. Considering a dynamic environment, we propose a hierarchical multi-agent deep reinforcement learning (H-MADRL) framework to solve this problem. The high-level agent resides in the AP and optimizes the beamforming strategy, while the low-level user agents learn and adapt individuals’ offloading strategies. To further improve the learning efficiency, we propose a novel optimization-driven learning algorithm that allows the AP to estimate the low-level users’ actions by solving an approximate problem efficiently. Then, the action estimation can be shared with all users and drive them to update individuals’ actions independently. Simulation results reveal that our algorithm can improve the reward performance by 50%. The learning efficiency and reliability are also enhanced comparing to the conventional model-free learning methods.

In this paper, we focus on a multi-user wireless network coordinated by a multi-antenna access point (AP). Each user can generate the sensing information randomly and report it to the AP. The freshness of information is measured by the age of information (AoI). We formulate the AoI minimization problem by jointly optimizing the users’ scheduling and transmission control strategies. Moreover, we employ the intelligent reflecting surface (IRS) to enhance the channel conditions and thus reduce the transmission delay by controlling the AP’s beamforming vector and the IRS’s phase shifting matrices. The resulting AoI minimization becomes a mixed-integer program and difficult to solve due to uncertain information of the sensing data arrivals at individual users. By exploiting the problem structure, we devised a hierarchical deep reinforcement learning (DRL) framework to search for optimal solution in two iterative steps. Specifically, the users’ scheduling strategy is firstly determined by the outer-loop DRL approach, and then the inner-loop optimization adapts either the uplink information transmission or downlink energy transfer to all users. Our numerical results verify that the proposed algorithm can outperform typical baselines in terms of the average AoI performance.

In this article, a wireless multirelay network in which the relays are energy constrained is studied. Especially, in order to consume the harvested energy efficiently at the relays so as to improve the network throughput, a new hybrid relaying protocol is first proposed. In the proposed protocol, each relay can flexibly switch its operation among energy harvesting (EH), information receiving (IR), active information transmission (IT), and two passive backscatter communication (BC) modes according to the channel states as well as its data buffer states and energy states in each transmission block, by which the harvested energy can be efficiently utilized and superior throughput performance can be achieved. However, under the hybrid relaying protocol, it is challenging to achieve a strategy to optimally determine the operation mode for each relay, and the energy and information scheduling at the relays that operate in the IR, IT, and BC modes. To address this issue, the involved optimization problem is formulated as a stochastic optimization problem, which cannot be tackled directly. To make it tractable, the stochastic optimization problem is transformed into a Markov decision process (MDP) with finite state and action spaces. By solving the MDP via a dynamic programming (DP) algorithm, the optimal strategy for the multirelay network is achieved. Furthermore, to reduce the computational complexity in the DP algorithm, an efficient algorithm with low complexity is developed by using a Lyapunov optimization framework. Numerical simulations show that our proposed hybrid relaying strategy can achieve superior throughput performance in wireless multirelay networks.

This article studies a network where a transmitter communicates with a receiver by hybrid communications that consist of passive information transmission (IT) via backscatter communication (BC) and active IT via wireless-powered communication (WPC). Because the circuit energy consumption in the passive IT of BC is much lower than that in the active IT of WPC, BC usually achieves a higher data transmission rate than WPC. Thus, it was suggested in the literature that the network throughput performance could not be improved by hybrid communications. However, our work in this article demonstrates that the throughput can be enhanced by a newly designed hybrid communication strategy. To demonstrate this, we develop a novel protocol that enables the transmitter to adaptively switch its operation between BC, active IT, and energy harvesting in one time block while scheduling energy consumption flexibly among multiple time blocks. Under the developed protocol, we formulate an optimization problem to jointly optimize the operation mode and resource allocation at the transmitter. The formulated problem is difficult to solve because the energy scheduling at the transmitter is coupled across multiple time blocks, and noncausal channel state information (CSI) is required. To address this problem, we first solve a simplified optimization problem via dynamic programming (DP) and a layered optimization method by assuming that the noncausal CSI is known. Then, we employ an approximate DP approach to solve the original problem with causal CSI. Finally, we verify by simulations that the proposed scheme can achieve superior throughput performance.

This paper considers the distributed beamforming design for a simultaneous wireless information and power transfer (SWIPT) in two-way relay network, which consists of two sources, K relay nodes and one energy harvesting (EH) node. For such a network, assuming perfect channel state information (CSI) is available, and we study two different beamforming design schemes. As the first scheme, we design the beamformer through minimization of the average mean squared error (MSE) subject to the total transmit power constraint at the relays and the energy harvesting constraint at the EH receiver. Due to the intractable expression of the objective function, an upper bound of MSE is derived via the approximation of the signal-to-noise ratio (SNR). Based on the minimization of this upper bound, this problem can be turned into a convex feasibility semidefinite programming (SDP) and, therefore, can be efficiently solved using interior point method. To reduce the computational complexity, a suboptimal beamforming scheme is proposed in the second scheme, for which the optimization problem could be recast to the form of the Rayleigh–Ritz ratio and a closed-form solution is obtained. Numerical results are provided and analyzed to demonstrate the efficiency of our proposed beamforming schemes.