This paper studies the influence of logarithmic functions in policy gradient deep reinforcement learning (RL) methods, a subject previously under-emphasized in the field.It delves into how different logarithmic bases, specifically loge, log2, and log10, influence the efficiency of policy gradient methods.We analyze the role of these logarithmic functions and their gradients when using Stochastic Gradient Descent as the optimizer in deep RL.Experimental results show that the choice of the logarithmic base influences the policy gradient methods.loge and log2 are generally more effective than log10 in deep RL algorithms, but log10 exhibits more stability.Based on analysis and experimental results, we propose the Logarithmic Basis Policy Gradient (LBPG) and Adaptive LBPG algorithms to enhance policy gradient methods.LBPG denotes policy gradient methods with loge, log2, and log10 as the logarithmic base, respectively.Furthermore, Adaptive LBPG dynamically selects a logarithmic basis among loge, log2, and log10 based on the variance-mean * Both authors contributed equally to this research.

Navigation methods based on deep reinforcement learning (RL) have recently exhibited superior performance, particularly for navigation in dynamic environments. However, most existing methods solely rely on deep neural network feature encoders to extract features from raw LiDAR data, lacking an explicit representation of environmental structure. This limitation hinders effective environmental representation and interpretability, constraining navigation performance improvement. To solve this problem, we propose two quantitative metrics based on laser scans, which explicitly represent environmental complexity and show great interpretability. Furthermore, we propose an environmental-complexity-based navigation method based on hierarchical deep RL with the proposed metrics. Experimental results show that the proposed method achieves better navigation performance than baselines, especially in challenging scenarios with corners and dynamic obstacles.

This paper considers a wireless-energy-transfer (WET)-enabled massive multiple-input-multiple-output (MIMO) system based on superimposed pilot (SP). With the aid of SP, the uplink (UL) channel estimation and wireless information transmission (WIT) that powered by the downlink (DL) WET can be operated simultaneously, and thus provide the potential for improving the UL achievable rate. The impact that SP has on the performance of such a WET-enabled massive MIMO system is mathematically characterized, and the UL achievable throughput is maximized by optimizing the variables, including the SP power-allocation factor and the time-allocation factor between the duration of WET and WIT. Numerical results validate the effectiveness of the proposed scheme.