Due to the high dynamic of unmanned aerial vehicle (UAV), the beam of UAV-mounted aerial base station (ABS) is difficult to align with ground users (GUs) and macro-cell base stations (MBSs), thereby reducing the communication rate. Towards this end, the channel state information of communication is used to assist onboard radar of ABS to sense the locations of GUs and MBSs for beam alignment to increase communication rate. To clarify the mechanism of mutual assistance between sensing and communication, we first derive the fundamental communication rate lower bound of integrated sensing and communication by utilizing the Cramér-Rao Bound. We find that the sensing power, sensing time, and transmit power between GU-ABS and ABS-MBS mutually influence the bounds of their communication rates with the shared frequency between sensing and communication. Accordingly, the maximizing communication rate problem is established by jointly optimizing transmit power, sensing power, and sensing dwell time allocation, which is decoupled into GU-ABS and ABS-MBS resource allocation subproblems. To reduce the computation complexity, a deep reinforcement learning based algorithm is proposed to solve this problem to replace the successive convex approximation technique. The simulation results demonstrate that the proposed approach is effective in maximizing the communication rate.

This letter is concerned with a three-dimensional target motion analysis issue using azimuth and elevation measurements. The nonlinear relationship between these measurements and target dynamics often poses challenges for conventional methods, especially in high-noise environments. To address this challenge, a novel multi-agent deep reinforcement learning (MADRL)-based estimator is proposed for target motion parameter estimation. Specifically, by modeling each component of the target motion parameter as an individual agent, the target motion parameter estimation process is framed as a cooperative Markov game. An MADRL framework is then introduced to solve this problem. Simulation results demonstrate that the proposed algorithm achieves higher estimation accuracy than existing estimators.

The localization of underwater acoustic sensor networks (UASNs) has emerged as a critical research area in the marine information fusion field. Generally, the convex optimization method is adopted to solve the localization problem. However, this method has limitations in complex underwater environments, since it is difficult to transform the nonconvex optimization problem into a convex optimization problem under such conditions. Recently, deep reinforcement learning (DRL) has shown great potential and promise in solving intricate optimization tasks. Motivated by this, we propose to adopt DRL for UASNs localization to improve accuracy and robustness. The key challenge is that existing DRL-based methods require discretization of the environment, which leads to a compromise between search time and localization precision. To address this challenge, we first model the localization problem as a Markov decision process (MDP) with continuous state and action spaces and subsequently introduce a continuous control DRL framework to solve the localization problem. Within this framework, we develop three continuous control DRL-based localization estimators to address the localization problem in unsupervised, supervised, and semisupervised scenarios. Comprehensive simulations demonstrate the effectiveness of our approach, as the proposed solutions exhibit several advantageous features compared to traditional methods, such as: 1) compared with the convex optimization-based method, the convex relaxation is not required; 2) compared with the least squares method, the proposed estimators are capable of converging to a global optimal state; and 3) compared with the discrete control DRL method, the proposed estimators reduce localization time and enhance localization accuracy significantly.

Unmanned aerial vehicles mounted aerial base stations (ABSs) are capable of providing on-demand coverage in next-generation mobile communication system. However, resource allocation for ABSs to provide continuous coverage is challenging, since the high dynamic of ABSs and time-varying air-to-ground channel would result in channel state information (CSI) mismatch between resource allocation decision and implementation. In consequence, the coverage of ABSs is discontinuous in spatial-temporal dimensions, i.e., the variance of user rate between adjacent time slots is large. To ensure the coverage continuity, we design a resource allocation method based on deep reinforcement learning (RDRL). Capable of adaptively tuning neural network structures, RDRL could satisfy coverage requirements by jointly allocating subchannels and power for ground users. Meanwhile, the temporal channel correlation is taken into account in the design of reward function in RDRL, which aims to alleviate the influence of CSI mismatch between method decision and implementation. Moreover, RDRL can apply a pre-trained model of previous coverage requirement to current requirement to reduce computation complexity. Experimental results show that the rate variance of RDRL can be reduced by 66.7% and spectral efficiency of RDRL can be increased by 34.7% compared with benchmark algorithms, which ensures the coverage continuity.

In this paper, we consider an unmanned aerial vehicle (UAV)-assisted post-disaster rescue scenario, where UAV-mounted aerial base stations (ABSs) compute tasks related to post-disaster rescue operations while also providing communication services to ground users (GUs). With the limited computation capacity of ABSs, we aim to minimize the task computation queuing delay and ensure the GU communication rate by jointly optimizing ABS-GU association, task offloading, and ABS trajectory. The problem is formulated as a mixed-integer nonlinear program, and a solution is proposed by integrating Lyapunov optimization and actor-critic based deep reinforcement learning. We utilize a model-based successive convex approximation technique in a critic module to acquire an accurate evaluation of actor module output. Simulation results demonstrate the effectiveness of the proposed approach in reducing the task computation queuing delay.

In this work, we investigate the location error of a fingerprint-based indoor system with the application of hybrid received signal strength (RSS) and channel state information (CSI) fingerprints. It manifests that exploiting correlation between RSS and CSI could effectively reduce location error. On this basis, we propose a hybrid RSS/CSI localization algorithm (HRCL), which is designed based on the deep learning. The HRCL fully exploits quick construction of fingerprint database with the coarse-grained RSS and rich multipath information of the fine-grained CSI. The RSS and CSI with high correlation are selected to construct fingerprint database, aiming to improve localization accuracy. Moreover, the deep neural network is trained for location estimation. Especially, experimental results validate that the location error of HRCL can be reduced by 64.4%, compared with the existing localization method. Moreover, the location error of HRCL can be reduced by 29.1 %, compared with HRCL without RSS/CSI selection by correlation coefficient.