Underwater Acoustic Sensor Networks (UASNs) have garnered considerable attention in recent years due to their widespread applications in both industrial and civilian domains, such as ocean exploration and environmental monitoring. This paper introduces an intelligent Multi-Agent Reinforcement Learning (MARL) algorithm to determine routing in UASNs with dynamic underwater environments adaptively. Initially, we model ocean currents and acoustic signal loss in underwater communication to perform the real-world characteristics of underwater routing. Based on software-defined networking (SDN) principles, we redefine the architecture of UASNs and propose an Adaptive Routing scheme for Software-Defined UASNs (ARSDU). Leveraging ARSDU, we propose the Reward Cooperative Distribution and Tracing Mechanism-enabled Multi-Agent Reinforcement Learning (RCDTM-MARL) algorithm to optimize routing decisions. The proposed RCDTM-MARL algorithm enhances the convergence speed of MARL by decomposing the reward function into independent-reward and interactive-reward, while incorporating an experience replay buffer to further accelerate convergence. Ultimately, the adaptive routing decision algorithm, based on RCDTM-MARL, adaptively determines the optimal routing path for UASNs. The evaluation results demonstrate that the proposed routing scheme outperforms recent research approaches, achieving superior underwater data routing decisions on multiple key performance metrics.

This paper investigates a symbiotic radio (SR)-assisted mobile edge computing (MEC) network for railway Internet of Things (RIoT) services, where IoT devices parasitize in a train-ground primary network for passively modulating their computation tasks over computation offloading by associating a mobile relay (MR) on the high-speed railway (HSR). The multi-antenna base station (BS) integrated with the MEC server recovers computation task data from MRs and IoT devices through joint decoding. With the objective of maximizing the total computation efficiency (CE) of all MRs while satisfying the computation requirements of IoT devices, we formulate a computation offloading and resource allocation problem that jointly optimizes the association strategy between MRs and IoT devices, the received beamforming of the BS, the transmission power and computation frequency of MRs. However, since the rapid variation of channel conditions in HSRs poses difficulties to centralized optimization methods in terms of both accurate model acquisition and computation overhead, we utilize a model-free deep reinforcement learning (DRL) approach to propose a fingerprint-based distributed dueling double deep Q-network (FD4QN)-based computation offloading and resource allocation scheme to solve the above problem. In particular, this scheme describes the original problem as a partially observable Markov decision process (POMDP), and then incorporates low-dimensional fingerprint markers in each computing agent to stabilize the experience replay mechanism in a multi-agent environment, thereby enhancing training robustness. Moreover, each agent makes a decision for each MR at each time frame by using a dueling double deep Q-network (D3QN) framework based on the local observation state. Simulation results show that the proposed scheme achieves superior performance compared to other benchmark schemes.

In this paper, we investigate a novel symbiotic radio (SR)-aided high-speed railway (HSR) wireless network, in which the Internet of Things (IoT) device, operating as a secondary transmitter, transmits its own information to the mobile relay (MR) on the HSR by backscattering radio frequency (RF) signals from the base station (BS). With the assistance of SR, the designed network facilitates the transmission of locally collected environmental sensing messages from the IoT network to the HSR, simultaneously enhancing the primary communication between the BS and MRs. Aiming to maximize the sum transmission rate of the primary and the IoT network, we focus on a joint power control and device access (JPCDA) problem. Specifically, each IoT device accesses the network through appropriate time slot selection and appropriate power control, thereby achieving satisfactory overall network performance. However, since the fast channel variations arising from the high mobility of HSRs make it impractical to acquire accurate channel state information (CSI), it is challenging to achieve an optimal resource allocation scheme. To address this challenge, we develop a distributed deep reinforcement learning (DRL)-based algorithm that utilizes historical CSI to infer real-time CSI for decision making. In particular, each computing unit of the agent performs action selection for only one IoT device at one time based on the current local observation information. Numerical results illustrate that our proposed algorithm outperforms other baselines, and still works effectively when the environment changes.

This paper focuses on a symbiotic radio (SR)-aided high-speed railway (HSR) wireless network, where the base station (BS) in the primary network serves the mobile relays (MRs) on the HSR via orthogonal frequency division multiple access (OFDMA) and the Internet of Things (IoT) devices deployed around the HSR serve as secondary transmitters for information transmission by selecting appropriate time slots. By using the SR technique, the proposed network not only facilitates the transmission of locally collected environmental messages from the IoT network to MRs, but also enhances the primary communications from the BS to MRs. With the aim of maximizing the sum transmission rate of the primary and the IoT network, we formulate a device access problem under time slot allocation constraints. However, the time-varying channel due to the high mobility of HSR makes it challenging to obtain an optimal policy for the problem. To overcome this challenge, we develop a distributed deep reinforcement learning (DRL) algorithm, which utilizes historical knowledge to infer real-time information to make decisions. Particularly, the proposed algorithm performs action selection for only one IoT device at one time based on the current local observation information. Numerical results demonstrate that the performance of the proposed distributed DRL algorithm closely approximates the optimal strategy that requires perfect instantaneous information.

To improve the reliability of dynamic multiple wireless body area networks (WBANs) system, it is indispensable to comprehensively consider the interference mitigation and user data differences. In this paper, we study a multi-WBAN system, where sensors receive radio frequency (RF) signals from the access point (AP), then transmit the monitoring sign to the sink node. Considering the dynamic network topology and the individuality of users, we propose a data-driven wasser-stein distributionally robust weight-based joint power allocation (DW-JPA) scheme. In particular, we formulate a sum-weighted transmission rate maximization problem by optimizing dynamic weight and transmit power ratio subject to the data transmission and energy limitation constraints. We divide the problem into dynamic weight subproblem and transmission power control subproblem. We utilize the collected physiological data to predict the optimal actual weight assignment. Then, we quantify the criticality of sensors and build an ambiguity set based on wasserstein distance for probability distributions of the critically. In essence, the optimal weight is obtained by using the distributionally robust optimization (DRO) method. Furthermore, due to the non-convexity of the power control subproblem, we convert the subproblem to a difference of convex (DC) problem and use an iterative algorithm to alternately optimize the power ratio. The results reveal that the proposed scheme achieves a significantly higher weighted transmission rate with physiological data compared with traditional schemes.

This paper investigates a high-speed railway (HSR) network with device-to-device (D2D)-aided underlaying cellular communications, where cellular-based train-to-infrastructure (T2I) and D2D-supported train-to-train (T2T) transmissions co-exist. Considering the diverse quality-of-service (QoS) requirements of different types of links, age of information (AoI) is adopted as a new metric to evaluate the information freshness performance of T2T links. With the objective to maximize the sum data rate of T2I links, we formulate a resource allocation problem under the minimum data rate constraints of T2I links and the maximum average AoI constraint of T2T links. As the problem with a set of binary variables, it is intractable to be solved directly. Thus, we propose an AoI-aware power control and subcarrier assignment (AoI-PCSA) scheme, which decomposes the optimization problem into a power control subproblem and a subcarrier assignment subproblem. More specifically, we derive the optimal analytical solutions of power control for each T2I-T2T subcarrier reusing pair with algebraic methods. Then, we transform the subcarrier assignment subproblem into a weighted bipartite matching problem and obtain the optimal reusing pattern based on the Kuhn-Munkres algorithm. Simulation results demonstrate that our proposed scheme can achieve a sum data rate gain of up to 19.94% on average for T2I links as compared with other benchmark schemes.