This research examines quantized uplink multi-user MIMO communication systems with low-resolution quantizers at users and base stations (BS). In such a system, we employ the non-orthogonal multiple access (NOMA) technique for communication between users and the BS to enhance communication performance. To maximize the number of users that satisfy the quality of service (QoS) requirement while minimizing the user’s transmit power, we jointly optimize the transmit power and precoding matrices at the users and the digital beamforming matrix at the BS. Owing to the non-convexity of the objective function, we transform the problem into a reinforcement learning-based problem and propose a deep reinforcement learning (DRL) framework named QNOMA-DRLPA to overcome the challenge. Because the nature of the action decided by the DRL algorithm may not satisfy the problem constraints, we propose a postactor process to redesign the actions to meet all the problem constraints. In the simulation, we assess the proposed framework’s performance in training convergence and demonstrate its superior performance under various environmental parameters compared with other benchmark schemes.

The proliferation of Internet-of-Things applications has given rise to several challenges, including network congestion and high energy consumption. Among the promising technologies for beyond-5G networks, nonorthogonal multiple access (NOMA) and ambient backscatter communications (BackComs) stand out. These technologies enhance wireless access capacity and enable energy-efficient data sharing. In this study, we propose a novel energy-efficient resource allocation scheme for 6G NOMA-assisted BackCom networks. Our network model comprises a central reader (RD) and distributed backscatter devices (BDs) that harvest energy from incident signals to modulate useful data and reflect it toward the RD. To maximize energy efficiency (EE), we formulated a joint optimization problem of channel resource allocation and BDs’ reflection coefficients. However, solving this problem is challenging because of its nonconvexity and system dynamics. To address this issue, we developed a novel deep-learning-based algorithm that leverages the advantages of deep reinforcement learning. During training, we estimated the state components without relying on exact channel state information (CSI), which is computationally expensive. This estimation reduces the communication overhead raised in collecting CSI data. Extensive simulations were conducted to demonstrate the superiority of the proposed scheme. Simulation results show that the proposed scheme notably enhances EE compared to existing benchmarks. Specifically, improvements of approximately 30.3%, 41.7%, 6.0%, and 4.4% were observed when compared to the greedy approach, random approach, deep Q-Network, and successive convex approximation approach, respectively.

Unmanned aerial vehicles (UAVs) have been widely applied in wireless communications because of its high flexibility and line-of-sight transmission. In this study, we develop low-complexity and robust device-to-device (D2D) link scheduling in UAV-assisted industrial-Internet-of-Things (IIoT) networks. First, we propose a sparse convolutional neural network (SCNN) model that uses the geographical map of transmission links as input. The model consists of three main blocks: 1) generic feature filtering, 2) speed–accuracy balancing, and 3) deep feature processing. Unlike other state-of-the-art methods, the proposed SCNN directly processes the geographical map collected using a connected UAV. Second, we propose a deep deterministic policy gradient-based reinforcement learning model that processes the output feature map from the SCNN to optimize the D2D scheduling decision and maximize the achievable system rate in the long run. Extensive simulations revealed that the proposed scheme significantly improved the achievable rate over other benchmark comparison schemes, such as transmitters and receivers density-based deep learning (DL), ResNet-based DL, VGGNet-based DL, random scheduling, and all-active schemes, respectively. The simulations also demonstrated that the proposed scheme reduces computational complexity. With reduced complexity and nearly optimal performance, the proposed solution can be more efficiently applied to large-scale and dense IIoT networks.

Compared to static time-division duplexing (TDD), dynamic TDD (D-TDD) has significantly increased the spectral efficiency of cellular networks. However, conventional systems operate based on exact channel state information, resulting in high communication overhead and delay. Spatial deep learning refers to using spatial geographical information as the training data. This study investigates a spatial deep learning-based D-TDD scheme for 6G hotspot networks. First, we represent geographical location information in forms of traffic demand density grid matrices. Second, we use spatial convolution filters to extract discriminative features of uplink and downlink service gains and harms, taking the traffic demand density grid matrices as the input. Subsequently, extracted feature matrices are processed with sparse convolution blocks to reduce computation cost for the classification. Finally, we develop novel deep dueling neural networks, leveraging the extracted features to efficiently learn the near-optimal radio slot configurations for all base stations. Numerical results show that the proposed approach improves average rate per user by 2.5%, 6%, and 523.3% over those achieved in state-of-the-art centralized D-TDD, the competitive reinforcement learning, and greedy approaches, respectively. In addition, the proposed approach achieves up to 98.7% of the data rate performance of the optimum scheme with an exhaustive search algorithm.

Aerial access infrastructures have been considered a compulsory component of the sixth-generation (6G) networks, where airborne vehicles play the role of mobile access points to service ground users (GUs) from the sky. In this scenario, intelligent reflecting surface (IRS) is one of the promising technologies associated with airborne vehicles for coverage extensions and throughput improvements, a.k.a., flying IRS (F-IRS). This study considers a multiuser multiple-input single-output (MISO) F-IRS system, where the F-IRS reflects downlink signals from ground base stations (BSs) to users located at underserved areas where direct communications are unavailable. To achieve the system sum-rate maximization, we proposed a deep reinforcement learning (DRL) algorithm namedFlyReflectto jointly optimize the flying trajectory and IRS phase shift matrix. First, end-to-end communications from a BS to its GUs via the F-IRS are analyzed to identify environmental and operational factors that impact achievable system sum rate. Subsequently, the system is transformed into a DRL model, which is resolvable by the deep deterministic policy gradient (DDPG) algorithm. To improve the action decision accuracy of the DDPG algorithm, we proposed a mapping function to guarantee that all constraints are satisfied regardless of noise additions in the exploration process. Simulation results showed that our proposed algorithm outperforms state-of-the-art algorithms in multiple scenarios.

To meet the stringent demands of emerging Internet-of-Things (IoT) applications, such as smart home, smart city, and virtual reality in 5G/6G IoT networks, edge content caching for mobile/multiaccess edge computing (MEC) has been identified as a promising approach to improve the quality of services in terms of latency and energy consumption. However, the limitations of cache capacity make it difficult to develop an effective common caching framework that satisfies diverse user preferences. In this article, we propose a new content caching strategy that maximizes the cache hit ratio through flexible prediction in dynamically changing network and user environments. It is based on a hierarchical deep learning architecture: long short-term memory (LSTM)-based local learning and ensemble-based meta-learning. First, as a local learning model, we employ an LSTM method with seasonal-trend decomposition using loess (STL)-based preprocessing. It identifies the attributes for demand prediction on the contents in various demographic user groups. Second, as a metalearning model, we employ a regression-based ensemble learning method, which uses an online convex optimization framework and exhibits sublinear “regret” performance. It orchestrates the obtained multiple demographic user preferences into a unified caching strategy in real time. Extensive experiments were conducted on the popular MovieLens data sets. It was shown that the proposed control provides up to a 30% higher cache hit ratio than conventional representative algorithms and a near-optimal cache hit ratio within approximately 9% of the optimal caching scheme with perfect prior knowledge of content popularity. The proposed learning and caching control can be implemented as a core function of the 5G/6G standard’s network data analytic function (NWDAF) module.

Mobile edge computing and nonorthogonal multiple access (NOMA) have been considered as promising technologies that can satisfy rigorous requirements of industrial Internet of Things systems. However, system dynamics, including channel states and computation task requests, may continuously change NOMA decoding order and computation uploading time, making it difficult to reduce latency using conventional highly complex optimization methods. In this article, we investigate a novel scheme that effectively reduces the average task delay to improve the quality of service for all users by jointly optimizing subchannel assignment (SA), offloading decision (OD), and computation resource allocation (CRA). To deal with the high complexity, the original multiserver problem is first decomposed into multiple single-server problems. Subsequently, each single-server problem is decoupled into CRA and SA/OD subproblems. Using convex optimization, a closed-form solution is derived for the optimal CRA action. Concurrently, the optimal SA/OD action is obtained using a distributed multiagent deep reinforcement learning algorithm. Simulation results reveal that the proposed scheme significantly outperforms the state-of-the-art schemes. In particular, it reduces the action decision duration by 30 times while achieving a near-optimal performance of up to 97% of the optimum under the exhaustive search scheme.

Mobile-edge computing (MEC) and nonorthogonal multiple access (NOMA) have been regarded as promising technologies for beyond fifth-generation (B5G) and sixth-generation (6G) networks. This study aims to reduce the computational overhead (weighted sum of consumed energy and latency) in a NOMA-assisted MEC network by jointly optimizing the computation offloading policy and channel resource allocation under dynamic network environments with time-varying channels. To this end, we propose a deep reinforcement learning algorithm named ACDQN that utilizes the advantages of both actor-critic and deep Q-network methods and provides low complexity. The proposed algorithm considers partial computation offloading, where users can split computation tasks so that some are performed on the local terminal while some are offloaded to the MEC server. It also considers a hybrid multiple access scheme that combines the advantages of NOMA and orthogonal multiple access to serve diverse user requirements. Through extensive simulations, it is shown that the proposed algorithm stably converges to its optimal value, provides approximately 10%, 27%, and 69% lower computational overhead than the prevalent schemes, such as full offloading with NOMA, random offloading with NOMA, and fully local execution, and achieves near-optimal performance.

Future wireless and mobile network services must accommodate highly dynamic downlink and uplink traffic asymmetry. To fulfill this requirement, the third-generation partnership project (3GPP) introduced the enhanced interference mitigation and traffic adaptation strategy in addition to dynamic time division duplexing (TDD). In this study, we develop a reinforcement learning (RL)-based dynamic TDD framework that effectively controls interference and serves various traffic demands. First, we introduce an interference-penalty model that evaluates interference indirectly based on the duplexing policy. This can significantly reduce overhead for measuring and exchanging channel information in a dense network. Second, we design a new mixed-reward model that consists of the achievable data rate and the implicit interference penalty. Third, we implement deep RL algorithms that base station (BSs) use to train their radio frame configurations (RFCs). The training process at each BS takes into account the traffic demand and the RFCs of the surrounding BSs. The BSs are coordinated in a single-leader multi-follower Stackelberg game, which achieves a global RFC setup that maximizes the data rate and minimizes the interference. Extensive simulations show that the proposed framework stably converges in various environments and provides near-optimal performance equivalent to 95% or more of the full-search-based optimal performance, which is 48.84%, 41.92%, and 62.11% higher than the currently utilized random RFC, fixed RFC, and traffic-matched RFC approaches.