The routing protocol for low-power and lossy networks (RPL) is a specialized routing protocol designed for optimized data routing, specifically for resource-constrained Internet of Things (IoT) networks with unreliable links and high packet loss. However, RPL is highly vulnerable to significant security challenges, particularly the decrease rank attack (DRA), in which malicious nodes attract child nodes by falsely advertising lower ranks, leading to routing inefficiencies, unnecessary retransmissions, and increased energy consumption. To address this problem, we propose a novel intent-based networking-driven centralized real-time reinforced detection scheme (CRRDS), which translates high-level security intents into policy-driven automated control strategies for DRA detection. In the proposed CRRDS, a resource-rich root node acts as a deep reinforcement learning agent that collects critical information from the child nodes, including the node ID, end-to-end delay, received signal strength indicator, and hop count, to detect suspicious behavior accurately and intelligently. Initially, we implemented a deep Q-network (DQN)-assisted CRRDS in detecting DRA. Subsequently, we utilized double DQN (DDQN) and dueling DDQN due to their enhanced capabilities in value estimation and policy learning. The dueling DDQN performed optimally because of its deeper architecture. Simulation results demonstrate that the proposed dueling DDQN-assisted CRRDS achieves the highest detection accuracy of 98% with notable gains in true positive and false positive rates, even in complex scenarios with up to 30% malicious nodes.

In this paper, we design a resource block (RB) oriented power pool (PP) for semi-grant-free non-orthogonal multiple access (SGF-NOMA) in the presence of residual errors resulting from imperfect successive interference cancellation (SIC). In the proposed method, the BS allocates one orthogonal RB to each grant-based (GB) user, and determines the acceptable received power from grant-free (GF) users and calculates a threshold against this RB for broadcasting. Each GF user as an agent, tries to find the optimal transmit power and RB without affecting the quality-of-service (QoS) and ongoing transmission of the GB user. To this end, we formulate the transmit power and RB allocation problem as a stochastic Markov game to design the desired PPs and maximize the long-term system throughput. The problem is then solved using multi-agent (MA) deep reinforcement learning algorithms, such as double deep Q networks (DDQN) and Dueling DDQN due to their enhanced capabilities in value estimation and policy learning, with the latter performing optimally in environments characterized by extensive states and action spaces. The agents (GF users) undertake actions, specifically adjusting power levels and selecting RBs, in pursuit of maximizing cumulative rewards (throughput). Simulation results indicate computational scalability and minimal signaling overhead of the proposed algorithm with notable gains in system throughput compared to existing SGF-NOMA systems. We examine the effect of SIC error levels on sum rate and user transmit power, revealing a decrease in sum rate and an increase in user transmit power as QoS requirements and error variance escalate. We demonstrate that PPs can benefit new (untrained) users joining the network and outperform conventional SGF-NOMA without PPs in spectral efficiency.

Semi-grant-free non-orthogonal multiple access (SGF-NOMA) is a potential paradigm to support massive connec-tivity for the short packets Internet of things (IoT) applications while satisfying the undistracted transmission requirements of primary IoT users. However, resource allocation in SGF-NOMA is more challenging due to the sporadic traffic of grant-free (GF) users and the need to satisfy the quality of service (QoS) requirements of grant-based (GB) users. The GF users access and choose resources at random, resulting in frequent power collisions and decoding failures at the base station (BS). This paper develops a general learning framework that enables GF users to learn from historical information to avoid power collisions. We utilize a hybrid multi-agent deep reinforcement learning (hMA-DRL) framework to maximize the connectivity and enhance the number of successful decoded users at the BS. The numerical results show that the proposed scheme achieves a solution near to the optimal one and increases the successful decoded users by 42.38% as compared to the benchmark scheme. The considered algorithm performs well with an increasing number of users as compared to the competitive and cooperative MA-DRL algorithms.

In this paper, we propose a novel distributed resource allocation mechanism for semi-grant-free non-orthogonal multiple access (SGF-NOMA) transmission to maximize the network throughput, where multi-agent deep reinforcement learning with prioritized experience replay (PER) is employed. We design a centralized training framework and decentralized decision making to increase the flexibility of the proposed scheme. More specifically, each grant-free user as an "agent" learns the dynamics of the environment and makes its decisions independently in a decentralized manner. No heavy information exchange is needed to find the optimal transmit power and sub-channel that maximize the throughput. Numerical results show that the proposed algorithm with PER enhances the learning efficiency compared to the algorithm with conventional replay buffer and outperforms the existing scheme with a 12% throughput increase.

Grant-free non-orthogonal multiple access (GF-NOMA) is a potential multiple access framework for internet-of-things (IoT) networks to enhance connectivity. However, the resource allocation problem in GF-NOMA is challenging and the effectiveness of such a solution is limited due to the absence of closed-loop power control. In this paper, we design a prototype of layer-based transmit power pool by utilizing multi-agent reinforcement learning to provide open-loop power control and offload the computing tasks at the base station (BS) side. IoT users in each layer decide their own transmit power level from this layer-based power pool, instead of transmitting on the allocated sub-channel with allocated transmit power level. The proposed algorithm does not require any information exchange between IoT users and does not rely on any assistance from the BS. Numerical results confirm that the double deep Q network based GF-NOMA algorithm achieves high accuracy and finds out an accurate transmit power level for each layer. Moreover, the proposed GF-NOMA system outperforms the traditional GF with orthogonal multiple access techniques in terms of throughput.

Grant-free non-orthogonal multiple access (GF-NOMA) is a potential multiple access framework for short-packet internet-of-things (IoT) networks to enhance connectivity. However, the resource allocation problem in GF-NOMA is challenging due to the absence of closed-loop power control. We design a prototype of transmit power pool (PP) to provide open-loop power control. IoT users acquire their transmit power in advance from this prototype PP solely according to their communication distances. Firstly, a multi-agent deep Q-network (DQN) aided GF-NOMA algorithm is proposed to determine the optimal transmit power levels for the prototype PP. More specifically, each IoT user acts as an agent and learns a policy by interacting with the wireless environment that guides them to select optimal actions. Secondly, to prevent the Q-learning model overestimation problem, double DQN (DDQN) based GF-NOMA algorithm is proposed. Numerical results confirm that the DDQN based algorithm finds out the optimal transmit power levels that form the PP. Comparing with the conventional online learning approach, the proposed algorithm with the prototype PP converges faster under changing environments due to limiting the action space based on previous learning. The considered GF-NOMA system outperforms the networks with fixed transmission power, namely all the users have the same transmit power and the traditional GF with orthogonal multiple access techniques, in terms of throughput.