This work explores combining the rate-splitting multiple-access (RSMA) technique with an active reconfigurable intelligent surface (RIS) to improve the quantized multiuser multiple-input single-output network. The active RIS facilitates communication between the base station (BS) and users equipped with low-resolution quantizers, whereas RSMA improves downlink transmission efficiency. By maximizing the spectral efficiency while minimizing the power consumption at the transmitter and active RIS, we formulate an energy efficiency maximization problem by jointly designing the BS precoding matrix and active RIS reflecting matrix. The optimization problem presents nonconvexity, which makes finding the optimal solution challenging. Therefore, we reformulate the problem into a reinforcement learning-based problem that is solvable by applying deep reinforcement learning (DRL) algorithms. To ensure action accuracy, we design a constraint-matching function that integrates with the DRL algorithm, forming a DRL framework securing all problem constraints. To assess the proposed DRL algorithm, we propose an alternating-based solution that decomposes the problem into precoding matrix optimization and active reflecting matrix optimization sub-problems, which are solvable using the successive convex approximation-based method. The performance evaluations demonstrate the convergence and effectiveness of the proposed approaches in various scenarios.

This research examines quantized uplink multi-user MIMO communication systems with low-resolution quantizers at users and base stations (BS). In such a system, we employ the non-orthogonal multiple access (NOMA) technique for communication between users and the BS to enhance communication performance. To maximize the number of users that satisfy the quality of service (QoS) requirement while minimizing the user’s transmit power, we jointly optimize the transmit power and precoding matrices at the users and the digital beamforming matrix at the BS. Owing to the non-convexity of the objective function, we transform the problem into a reinforcement learning-based problem and propose a deep reinforcement learning (DRL) framework named QNOMA-DRLPA to overcome the challenge. Because the nature of the action decided by the DRL algorithm may not satisfy the problem constraints, we propose a postactor process to redesign the actions to meet all the problem constraints. In the simulation, we assess the proposed framework’s performance in training convergence and demonstrate its superior performance under various environmental parameters compared with other benchmark schemes.

The use of directional antennas in high-frequency bands (e.g., millimeter-wave) is essential to support applications requiring high throughput and low latency. However, communications using directional antennas require intricate scheduling by a central coordinator to avoid collision and deafness problems. Thus, in this study, we propose a directional medium access control (DMAC) protocol based on a deep$Q$-network (DQN) framework wireless ad hoc networks (WANETs) for Internet of Things (IoT). In our model, even though there is no central coordinating unit (e.g., edge/cloud server), each IoT device can intelligently avoid the collision and deafness through its learning agent. In addition, to maximize the throughput, we design a reinforcement learning (RL) architecture and propose a DQN-based DMAC such that each IoT device intelligently selects the time-slot and transmitting beam without any central coordinator. The proposed schemes are evaluated using carrier-sense multiple access (CSMA) and adaptive learning-based DMAC (AL-DMAC) protocols. The evaluation results reveal that the proposed double DQN scheme outperforms the existing schemes by approximately 54.1% and 57.2% in terms of the throughput.

The exponential growth of the Internet of Multimedia Things (IoMT) traffic has posed a threat of service quality degradation due to the limitation of current communication, networking, and computing advances in mobile networks. In this regard, managing the Quality-of-Experience (QoE) for IoMT services is a vital challenge to meet user satisfaction. To cope with this problem, we investigate the joint optimization of video quality variation and latency in multiuser downlink rate-splitting multiple-access (RSMA) networks, especially within imperfect network conditions and state information. To accomplish this, we first formulated the joint optimization problem into a Markov decision process framework, then exploited a deep reinforcement learning approach to adaptively calculate the optimal configuration of the RSMA against environment dynamics. As a result, the proposed deep deterministic policy gradient on RSMA-based video streaming system (DDPG-RMAVS) provides QoE maintenance by minimizing video resolution reduction and latency. Extensive simulation results revealed that the proposed DDPG-RMAVS algorithm surpasses existing algorithms by achieving higher video quality, lower delay, larger buffer capacity, and limited stalling events, representing a significant breakthrough in IoMT streaming optimization.

This paper proposes a flexible and efficient access control scheme that combines the orthogonal frequency division multiple access and rate-splitting multiple-access techniques for enhancing the uplink transmission in a digital twin edge network system. We formulate a non-convex mixed integer optimization problem that minimizes the energy consumption of all Internet of Things devices (IoTDs) and maximizes the number of successful IoTD tasks. To this end, we propose a deep reinforcement learning (DRL) framework by normalizing a DRL training algorithm named deep deterministic policy gradient for efficiently designing the variables while ensuring the problem constraints. However, in the inference stage, the proposed DRL method may encounter different devices and services. Therefore, we design an exhaustive-improved DRL method that can improve the proposed DRL effectively using information from a digital-twin module. We also propose a mathematical approximation-based solution employing two convexification approach: Dinkelbach’s method and relaxed Linear Matrix Inequality (LMI). Through extensive simulations over different parameters and scenarios, we identify the polynomial complexity, stable convergence, and operating regime of the proposed solutions. It is also confirmed that the proposed approaches work well even with digital twin defects and provide improved performance in terms of energy consumption and number of successful tasks in comparison with benchmark schemes.

Rate-splitting multiple access (RSMA) and reconfigurable intelligent surface (RIS) techniques show promise in enhancing spectral efficiency in sixth-generation Internet of Things (IoT) networks. However, optimizing the synergy between these two methods is challenging due to the complex and dynamic environment. This study focuses on maximizing the sum-rate metric in RIS-assisted uplink multiantenna RSMA IoT networks to address this problem. We jointly optimized the base station beamforming design, power allocation, and RIS phase shifts to enhance the spectral efficiency with multiple mobile IoT devices present. The controlled parameters are continuous variables and the mathematical problem is nonconcave. Therefore, we formulated the problem as a Markov decision process and used the deep deterministic policy gradient (DDPG) to determine the optimal joint actions. We proposed a safe action shaping process for the decision-making actor network to address constraint violations. Through a rigorous performance evaluation, we demonstrated that the DDPG approach with action shaping outperforms the current DDPG algorithm regarding the maximum achievable sum rate.

Aerial access networks, comprising a hierarchical model of high-altitude platforms (HAPs) and multiple unmanned aerial vehicles (UAVs), are considered a promising technology to enhance the service experience of Internet of Things Devices (IoTD), especially in underserved areas where terrestrial base stations (TBSs) do not exist. In such scenarios, optimally orchestrating the limited computation, communication, and energy resources in both HAPs and UAVs is crucial toward for an efficient aerial networking infrastructure. Thus, in this study, we investigate and formulate the joint IoTDs association, partial offloading, and communication resource allocations (JAPORAs) decisions problem in heterogeneous Aerial Access IoT (AAIoT) networks to maximize service satisfaction for IoTDs, while minimizing their total energy consumption. In particular, the formulated problem is transformed into a multiagent Markov decision process (MAMDP) to deal with its nonconvexity and environmental dynamicity. To solve the problem, we propose a multiagent policy-gradient-based deep actor–critic algorithm, named MADDPG-JAPORA, with centralized training and decentralized execution. Our extensive numerical experiments demonstrated that MADDPG-JAPORA reliably converges and provides superior performance compared with other state-of-the-art schemes.

Aerial access infrastructures have been considered a compulsory component of the sixth-generation (6G) networks, where airborne vehicles play the role of mobile access points to service ground users (GUs) from the sky. In this scenario, intelligent reflecting surface (IRS) is one of the promising technologies associated with airborne vehicles for coverage extensions and throughput improvements, a.k.a., flying IRS (F-IRS). This study considers a multiuser multiple-input single-output (MISO) F-IRS system, where the F-IRS reflects downlink signals from ground base stations (BSs) to users located at underserved areas where direct communications are unavailable. To achieve the system sum-rate maximization, we proposed a deep reinforcement learning (DRL) algorithm namedFlyReflectto jointly optimize the flying trajectory and IRS phase shift matrix. First, end-to-end communications from a BS to its GUs via the F-IRS are analyzed to identify environmental and operational factors that impact achievable system sum rate. Subsequently, the system is transformed into a DRL model, which is resolvable by the deep deterministic policy gradient (DDPG) algorithm. To improve the action decision accuracy of the DDPG algorithm, we proposed a mapping function to guarantee that all constraints are satisfied regardless of noise additions in the exploration process. Simulation results showed that our proposed algorithm outperforms state-of-the-art algorithms in multiple scenarios.

The recent expansion of mobile IoT devices (MIoTDs) along with the exposure of many compute-intensive and latency-critical applications, have given a step rise to the mobile edge computing (MEC) platform to process computational microservices at the edge. The paramount importance of designing an effective incentive mechanism is a very important topic for such systems to get a fair amount of resources and provide incentives to MIoDs. Hence, we design a MEC platform with heterogeneous MIoTDs participating in a computational microservice offloading scheme. Here, we propose an incentive approach applying a double auction mechanism to incentivize the involvement of MIoTDs. In practice, the incentive mechanism typically interacts with the demand estimation scheme that estimates the demand profile of MIoTDs. As a result, we design a novel mechanism for microservices –microservice Incentive Service Offloading (mISO), which comprises an incentive approach and a demand estimation scheme. The mISO mechanism holds truthfulness, rationality, and low computational complexity while guaranteeing positive social welfare and generating the optimal demand profiles for MIoTDs. Simulation results showed that mISO provides 18–21$\%$and 25–30$\%$improvements in terms of average latency and resource utilization compared to existing works.

Recent studies have shown that rate splitting multiple access (RSMA), which depends on multi-antenna rate splitting (RS) at the transmitter and successive interference cancellation (SIC) at the receivers, successfully controls interference in multi-antenna communication networks. This paper examines RSMA's applicability to video streaming applications in cloud radio access networks (C-RAN). We aim to address a practical challenge to maximize the perceived quality of end users while keeping the delay constraints remained satisfied using RSMA. We propose a learning-based framework to select appropriate video quality together with beamforming vectors according to current defined system state. The simulation figure confirms that the learning behavior of proposed learning scheme is stable.

To meet the stringent demands of emerging Internet-of-Things (IoT) applications, such as smart home, smart city, and virtual reality in 5G/6G IoT networks, edge content caching for mobile/multiaccess edge computing (MEC) has been identified as a promising approach to improve the quality of services in terms of latency and energy consumption. However, the limitations of cache capacity make it difficult to develop an effective common caching framework that satisfies diverse user preferences. In this article, we propose a new content caching strategy that maximizes the cache hit ratio through flexible prediction in dynamically changing network and user environments. It is based on a hierarchical deep learning architecture: long short-term memory (LSTM)-based local learning and ensemble-based meta-learning. First, as a local learning model, we employ an LSTM method with seasonal-trend decomposition using loess (STL)-based preprocessing. It identifies the attributes for demand prediction on the contents in various demographic user groups. Second, as a metalearning model, we employ a regression-based ensemble learning method, which uses an online convex optimization framework and exhibits sublinear “regret” performance. It orchestrates the obtained multiple demographic user preferences into a unified caching strategy in real time. Extensive experiments were conducted on the popular MovieLens data sets. It was shown that the proposed control provides up to a 30% higher cache hit ratio than conventional representative algorithms and a near-optimal cache hit ratio within approximately 9% of the optimal caching scheme with perfect prior knowledge of content popularity. The proposed learning and caching control can be implemented as a core function of the 5G/6G standard’s network data analytic function (NWDAF) module.

Technological advances in fog computing are precipitating an evolution in conventional vehicle networks to a new paradigm called fog-enabled connected vehicle networks (FCVNs). FCVNs provide communication efficiency for ensuring safe transportation through the massive Internet of vehicles. In FCVNs, massive vehicles tend to associate with roadside units and high power nodes, which act as fog nodes (FNs), when they have a good channel quality and/or popular contents. This circumstance may lead to a load imbalance among the FNs. This problem significantly decreases the resource utilization efficiency and service capability of the networks. In this article, we propose a dynamic resource orchestration (DRO) scheme to harmonize resource allocation for connected vehicles by migrating the offloaded services among FNs. A graph-theoretic approach is utilized to transform the FCVN into a directed graph model, where the maximum resource reduction obtained by service migrations is considered the weight of the link between every two FNs. Subsequently, the maximum weight matching solution is used to determine optimal pairs of FNs for migrating services to maximize network resource utilization. Our simulation results reveal that the proposed DRO scheme achieves significant improvements in terms of service capability, throughput, and resource utilization efficiency as compared with existing algorithms.

Future wireless and mobile network services must accommodate highly dynamic downlink and uplink traffic asymmetry. To fulfill this requirement, the third-generation partnership project (3GPP) introduced the enhanced interference mitigation and traffic adaptation strategy in addition to dynamic time division duplexing (TDD). In this study, we develop a reinforcement learning (RL)-based dynamic TDD framework that effectively controls interference and serves various traffic demands. First, we introduce an interference-penalty model that evaluates interference indirectly based on the duplexing policy. This can significantly reduce overhead for measuring and exchanging channel information in a dense network. Second, we design a new mixed-reward model that consists of the achievable data rate and the implicit interference penalty. Third, we implement deep RL algorithms that base station (BSs) use to train their radio frame configurations (RFCs). The training process at each BS takes into account the traffic demand and the RFCs of the surrounding BSs. The BSs are coordinated in a single-leader multi-follower Stackelberg game, which achieves a global RFC setup that maximizes the data rate and minimizes the interference. Extensive simulations show that the proposed framework stably converges in various environments and provides near-optimal performance equivalent to 95% or more of the full-search-based optimal performance, which is 48.84%, 41.92%, and 62.11% higher than the currently utilized random RFC, fixed RFC, and traffic-matched RFC approaches.

The explosive growth of end devices and mobile applications calls for novel schemes that can enable computation‐hungry applications at small end‐devices and meet the massive connectivity requirement. Cognitive radio (CR), non‐orthogonal multiple access (NOMA) and multi‐access edge computing (MEC) are envisioned as the key technologies in fifth‐generation and beyond. In this work, the authors introduce the concept of CR‐NOMA in MEC offloading, where a secondary user (SU) can utilise the spectrum allocated to a primary user (PU) to offload its computation task to the MEC server for remote execution. For the spectrum utilisation, an equation to specify the minimum transmit power that must be allocated to the PU is derived. The authors also develop an algorithm to determine the offloading decision (i.e. offloading or not) and the paired PU (i.e. subcarrier used for computation offloading) for SUs, using one‐to‐one matching game. Moreover, through numerical simulations, the authors demonstrate the superior performance of the proposed algorithm compared with several baseline schemes.

Software-Defined Networking (SDN) leverages the implementation of reliable, flexible and efficient network security mechanisms which make use of novel techniques such as artificial intelligence (AI) and machine learning (ML). In particular, these techniques - together with SDN - are the key enablers for the design of anomaly detection methods which are based on efficient traffic flow monitoring. In this paper, we tackle this problem by proposing an efficient anomaly detection framework, denoted as DeepGuard, which improves the detection performance of cyberattacks in SDN based networks by adopting a fine-grained traffic flow monitoring mechanism. Specifically, the proposed framework utilizes a deep reinforcement learning technique, i.e., Double Deep${Q}$-Network (DDQN), to learn traffic flow matching strategies maximizing the traffic flow granularity while proactively protecting the SDN data plane from being overloaded. Afterwards, by implementing the learned optimal traffic flow matching control policy, the most beneficial traffic information for anomaly detection is acquired at runtime—thereby improving the cyberattack detection performance. The performance of the proposed framework is validated by extensive experiments, and the results show that DeepGuard yields significant performance improvements compared to existing traffic flow matching mechanisms regarding the level of traffic flow granularity. In the case of distributed denial-of-service (DDoS) attacks, DeepGuard achieves a remarkable attack detection performance while effectively preventing forwarding performance degradation in the SDN data plane.

Internet of Things (IoT) systems are characterized by highly automated operating environments, which comprise several IoT end devices (IDs) that generate vast amounts of data with strict real-time communication and high data rate requirements. Edge computing facilities are an alternative to traditional cloud computing and support massive data processing in IoT systems while reducing the burden on data centers. In this paper, we consider an edge-based IoT system that comprises an edge server (ES), edge gateways (EGs), and IDs that communicate wirelessly. The EGs reduce the load on the ES by preprocessing data received from ID. However, it may not be possible for a few EGs to accommodate a sheer number of IDs, given the limited computing power and communication coverage of the EGs. Therefore, it is necessary for a few IDs to directly connect to the ES without the support of EGs. Thus, we propose a resource orchestration scheme between EGs and ES and/or among EGs based on a Lagrangian and the Karush-Kuhn-Tucker condition. The scheme allocates optimal resources by considering the computing capacities of EGs and ES and manages interference among the EGs to maximize the efficiency of IoT systems. The performance evaluation indicates that the proposed scheme outperforms the existing schemes in terms of aggregate throughput, latency, data reception rate, and workload fairness among EGs by 42%, 59%, 37%, and 40%, respectively.

Internet of Things (IoT) technology has accelerated various industries through digital transformation. In an edge computing-based smart factory, a significant number of IoT devices generate large volumes of real-time data. This big data requires efficient routing among edge gateways (EGs) and an edge server for real-time data processing. Existing industrial wireless communication systems provide relatively low data rates and network capacity for real-time sensor data and control information over a wireless channel. This calls for the use of the very large bandwidth available at the mmWave spectrum for real-time data transmission. Existing data routing techniques for the mmWave band are based on traditional mobile ad hoc routing techniques and do not reduce the transmission delay for real-time sensory data in smart manufacturing systems. Therefore, to alleviate the real-time data processing requirement, we propose a new directional routing and link scheduling algorithm based on maximum weight independent set (MWIS). The proposed algorithm solves complicated MWIS problems efficiently and computes backhaul link scheduling results in a relatively short time by lowering the deafness problem among EGs. For transmission fairness, we used a Jain's fairness index method with numerical analysis of the transmission fairness constraint. We measured the efficiency of our proposed scheme in terms of throughput, delay, packet loss rate, and transmission fairness. Our simulation results show that the proposed scheme outperforms existing mmWave routing techniques. Moreover, we investigated the performance difference between the proposed algorithm and the optimal solution.

The next-generation mobile network anticipates integrated heterogeneous fronthaul and backhaul technologies referred to as a unified crosshaul architecture. The crosshaul enables a flexible and cost-efficient infrastructure for handling mobile data tsunami from dense Internet of things (IoT). However, stabilization, energy efficiency, and latency have not been jointly considered in the optimization of crosshaul performance. To overcome these issues, we propose an orchestration scheme referred to as the stabilized green crosshaul orchestration (SGCO). SGCO utilizes a Lyapunov-theory-based drift-plus-penalty policy to determine the optimal amount of offloaded data that should be processed either at the eastbound or westbound computing platforms to minimize energy consumption. To achieve system stability, the cache buffer is considered as the main constraint in developing the optimization process. Moreover, the amount of offloaded data transmitted via crosshaul links is selected by adopting the binary min-knapsack problem. Accordingly, a lightweight heuristic algorithm is proposed. As the cache buffer is stabilized and the computations are controlled, the SGCO ensures adjustable computing latency threshold for various IoT services. The performance analysis shows that the proposed SGCO scheme exposes effective energy consumption compared to other existing schemes while maintaining system stability considering latency.