In large-scale Internet of Things (IoT) deployments, efficiently allocating computing resources to IoT devices, while preserving the integrity and utility of their data, remains a critical challenge. This paper introduces a novel online probabilistic model designed to handle uncertainties in both demand and resource availability within IoT networks, where the computing tasks of requesting devices (RDs) are fulfilled by serving devices (SDs). The proposed model integrates stochastic elements and formulates an optimization problem that aims to minimize the number of active serving devices required for task offloading, subject to the constraints of available computing resources. To further enhance decision-making, the model incorporates the concept ofValue of Information (VoI)to ensure that the informational utility of each device’s data remains above a predefined threshold during task processing. The optimization problem is addressed using a heuristic algorithm. In scenarios where no serving device is immediately available, tasks are temporarily stored in a buffer and deferred to the next time slot, with their waiting time being tracked. This task allocation process is inspired by bin-packing algorithms, which are known for their efficiency in resource management and task scheduling. Moreover, the paper evaluates the performance of the proposed solution under worst-case conditions through feasibility analysis, thereby demonstrating its robustness. Two buffering strategies, First-In First-Out (FIFO) and Last-In First-Out (LIFO), are also examined to model task retrieval and execution behavior. Results show that adopting the FIFO strategy can reduce the average waiting time by approximately 50%. Overall, the proposed framework provides a reliable and scalable task computing service, with each serving device capable of supporting, on average, four requesting devices under typical operating conditions.

In this paper, we propose a novel cooperative resource sharing technique in multi-tier edge slicing networks which is robust to imperfect channel state information (CSI) caused by user equipments’ (UEs) mobility. Due to the mobility of UEs, the dynamic requirements of their tasks, and the limited resources of the network, we propose a smart joint dynamic pricing and resources sharing (SJDPRS) scheme that can incentivize the infrastructure provider (InP) and mobile network operators (MNOs). Aiming to maximize the profits of UEs, MNOs and the InP under the task fulfillment constraints, we formulate an optimization problem by deploying the multi-objective optimization method where in addition to the resource allocation variables, the price values are also the optimization variables. To solve the problem, we adopt a new deep reinforcement learning (DRL) method based on a carefully designed reward function. The simulation results indicate that the proposed resource sharing scenario can increase total profits for the UEs, MNOs, and InP in comparison to non-cooperative case, while also providing almost complete fairness among the players. In particular, as compared to the baselines and benchmarks, the profits for each network component (MNO, InP, and UEs), under fairness considerations, are enhanced by 75%, 79%, and 76%, respectively.

In this paper, we design a new flexible smart software-defined radio access network (Soft-RAN) architecture with traffic awareness for sixth generation (6G) wireless networks. In particular, we consider a hierarchical resource allocation model for the proposed smart soft-RAN model where the software-defined network (SDN) controller is the first and foremost layer of the framework. This unit dynamically monitors the network to select a network operation type on the basis of distributed or centralized resource allocation procedures to intelligently perform decision-making. In this paper, our aim is to make the network more scalable and more flexible in terms of conflicting performance indicators such as achievable data rate, overhead, and complexity indicators. To this end, we introduce a new metric, i.e, throughput-overhead-complexity (TOC), for the proposed machine learning-based algorithm, which supports a trade-off between these performance indicators. In particular, the decision making based on TOC is solved via deep reinforcement learning (DRL) which determines an appropriate resource allocation policy. Furthermore, for the selected algorithm, we employ the soft actor-critic (SAC) method which is more accurate, scalable, and robust than other learning methods. Simulation results demonstrate that the proposed smart network achieves better performance in terms of TOC compared to fixed centralized or distributed resource management schemes that lack dynamism. Moreover, our proposed algorithm outperforms conventional learning methods employed in recent state-of-the-art network designs.

Network slicing (NwS) is one of the main technologies in the fifth-generation of mobile communication and beyond (5G+). One of the important challenges in the NwS is information uncertainty which mainly involves demand and channel state information (CSI). Demand uncertainty is divided into three types: number of users requests, amount of bandwidth, and requested virtual network functions workloads. Moreover, the CSI uncertainty is modeled by three methods: worst-case, probabilistic, and hybrid. In this paper, our goal is to maximize the utility of the infrastructure provider by exploiting deep reinforcement learning (DRL) algorithms in end-to-end NwS resource allocation under demand and CSI uncertainties. Enhanced mobile broadband (eMBB) requires high data rates. The uncertainties we argued above have a direct negative impact on the data rate and our objective function. Therefore, we focus primarily on eMBB. Additionally, we also consider ultra-reliable low latency communications (uRLLC) and massive machine-type communication (mMTC). The proposed formulation is a non-convex mixed-integer non-linear programming problem. To perform resource allocation in problems that involve uncertainty, we need a history of previous information. To this end, we use a recurrent deterministic policy gradient (RDPG) algorithm, a recurrent and memory-based approach in DRL. Then, we compare the RDPG method in different scenarios with soft actor-critic (SAC), deep deterministic policy gradient (DDPG), distributed, and greedy algorithms. The simulation results show that the SAC method is better than the DDPG, distributed, and greedy methods, respectively. Moreover, the RDPG method out performs the SAC approach on average by 70%.

In this paper, we study a Deep Reinforcement Learning (DRL) based framework for an online end-user service provisioning in a Network Function Virtualization (NFV)-enabled network. We formulate an optimization problem aiming to minimize the cost of network resource utilization. The main challenge is provisioning the online service requests by fulfilling their Quality of Service (QoS) under limited resource availability. Moreover, fulfilling the stochastic service requests in a large network is another challenge that is evaluated in this paper. To solve the formulated optimization problem in an efficient and intelligent manner, we propose a Deep Q-Network for Adaptive Resource allocation (DQN-AR) in NFV-enabled network for function placement and dynamic routing which considers the available network resources as DQN states. Moreover, the service’s characteristics, including the service life time and number of the arrival requests, are modeled by the Uniform and Exponential distribution, respectively. In addition, we evaluate the computational complexity of the proposed method. Numerical results carried out for different ranges of parameters reveal the effectiveness of our framework. In specific, the obtained results show that the average number of admitted requests of the network increases by 7 up to 14% and the network utilization cost decreases by 5 and 20%.