In offline actor-critic (AC) algorithms, the distributional shift between the training data and target policy causes optimistic value estimates for out-of-distribution (OOD) actions. This leads to learned policies skewed toward OOD actions with falsely high values. The existing value-regularized offline AC algorithms address this issue by learning a conservative value function, leading to a performance drop. In this article, we propose a mild policy evaluation (MPE) by constraining the difference between the values of actions supported by the target policy and those of actions contained within the offline dataset. The convergence of the proposed MPE, the gap between the learned value function and the true one, and the suboptimality of the offline AC with MPE are analyzed, respectively. A mild offline AC (MOAC) algorithm is developed by integrating MPE into off-policy AC. Compared with existing offline AC algorithms, the value function gap of MOAC is bounded by the existence of sampling errors. Moreover, in the absence of sampling errors, the true state value function can be obtained. Experimental results on the D4RL benchmark dataset demonstrate the effectiveness of MPE and the performance superiority of MOAC compared to the state-of-the-art offline reinforcement learning (RL) algorithms.

In this paper, the problem of content caching in 5G cellular networks relying on social-aware device-to-device communications (DTD) is investigated. Our focus is on how to efficiently select important users (IUs) and how to allocate content files to the storage of these selected IUs to form a distributed caching system. We aim at proposing a novel approach for minimizing the downloading latency and maximizing the social welfare simultaneously. In particular, we first model the problem of maximizing the social welfare as a many-to-one matching game based on the social property of mobile users. We study this game by exploiting users' social properties to generate the utility functions of the two-side players, i.e., content providers (CPs) and IUs. Then we model the problem of minimizing the downloading latency as a many-to-many matching problem. For solving these games, we design a many-to-one IU selection (MOIS) matching algorithm and a many-to-many file allocation (MMFA) matching algorithm, respectively. Simulation and analytical results show that the proposed mechanisms are stable, and are capable of offering a better performance than other benchmarks in terms of social welfare and network downloading latency.

With the burst of mobile data, it is necessary to make use of idle mobile equipment for caching space. Caching in the femto-cells is proposed for reducing transmission latency between the WiFi points and its mobile users (MU) and better user service. In this paper, we firstly take the copyright of the files as the allocation resource and the WiFi points with caching space want to rent these copyrights. We propose a multi-rounds double auction mechanism for this problem and take the popularity parameter of the files as the quality weight. This game can help multiple content providers (CP) lease copyrights of the files to multiple WiFi points effectively. Different from traditional double auctions, it will take the failed buyers and sellers into consideration and they are allowed to change their requests in the next auction process. This mechanism can largely improve efficiency of the game and is budget balanced. We also prove that the allocation process is monotone and with the set of the critical payment rule, we prove the truthfulness of the mechanism. Additionally, we prove that the mechanism can form a conditional equilibrium. Simulation results verify the effectiveness of the proposed mechanism and compare with the traditional one-round double auction.

OFDM data detection in doubly-selective fading channels requires high complexity due to intercarrier interferences (ICI). We present a low-complexity receiver consisting of a semidefinite relaxation (SDR) based detector and parallel interference cancellation (PIC). The entire band is divided into clusters of adjacent subcarriers. SDR is applied on each cluster while PIC tackles ICI from other clusters. An upper bound of ICI power is derived and used to omit far-away clusters in performing PIC. Finally, an adaptive detector based on PIC, PIC-based SDR and the snap-shot SNR in channel is proposed to achieve a better tradeoff between complexity and performance.