Network slices (NSs) are managed through a hierarchical architecture in recent 5G standards. Each NS is formed by connecting autonomously managed network slice subnets (NSSs) across the 5G network domains. To provision a new NS, users specify NS requirements as a network slice request (NSR). The NSR is decomposed into requirements for each NSS, and resources in each domain are allocated. This NSR decomposition is crucial as the selected decomposition affects resource usage and, ultimately, the total number of successfully provisioned NSs. Although several methods address NSR decomposition, most rely either on (i) detailed knowledge of domain-specific resource allocation mechanisms or (ii) extensive historical NS operational data. In practice, however, each domain’s internal processes act as "black boxes" in the hierarchical NS management architecture, making it infeasible to acquire detailed resource allocation algorithms. Additionally, historical data may be insufficient because network slicing remains an emerging technology. These limitations hinder the direct application of existing methods to real operational 5G networks. In this paper, we propose a multi-armed bandits (MABs)-based optimization method that formulates the NSR decomposition as a linear contextual bandits with knapsacks (linCBwK) problem and sequentially acquires the optimal decomposition policy. A MABs-based optimization approach enables us to avoid the need for domain-internal knowledge or extensive pre-collected training data. Simulations demonstrate that our method increases the total number of successfully provisioned NSs by 16.0% compared to the baseline method.

Cloud computing (CC) and edge computing (EC) enhance the performance of end devices (EDs) with limited computational power by offloading tasks to cloud and edge servers, respectively. Multi-access edge computing (MEC) further advances EC by integrating wireless network resources, thus improving mobile service efficiency. While CC is well-suited for intensive computational tasks, it may face latency issues due to geographical distances. EC and MEC aim to minimize this latency by deploying server resources closer to EDs, but they encounter challenges due to the limited resources of edge servers. Cooperative task offloading emerges as a solution to address the above challenges of optimizing resource allocation across cloud and edge based on task characteristics. Despite numerous research, existing methods often cover only a portion of the networks and servers, leading to sub-optimal task allocation. Therefore, we propose a cooperative task-offloading method for multi-access edge-cloud networks, simultaneously considering server and link resources, base station (BS), and wireless channel allocation. This method improves task-offloading efficiency by utilizing cooperative multi-group multi-agent deep reinforcement learning (CMG-MADRL) with different agent groups for BS and server allocation. Simulations have demonstrated that our method effectively reduces resource utilization and task latency while minimizing constraint violations.

Network services provisioning with Service Function Chaining (SFC) consists of various controls such as Virtual Network Function (VNF) placement and traffic flow routing, executed dynamically. These controls take time to execute from start to completion (reconfiguration delay), and the reconfiguration delay varies depending on the type of control. When control is executed with a long reconfiguration delay to eliminate Quality of Service (QoS) degradation, it will take longer to complete, resulting in continued degradation. To optimize the network performance, a control method needs to consider the difference in the reconfiguration delay. However, most of existing works do not assume the difference, and aim to optimize QoS only at the completion of controls. In this paper, we assume different reconfiguration delays for each control in SFC provisioning, and propose a control scheduling that optimizes QoS during control execution on the basis of them. We first propose a network model in which the reconfiguration delay of each type of control is different and formulate the scheduling problem of network controls that minimizes the duration of QoS degradation. Then we propose a control scheduling method to solve the problem. Our method develops a stochastic search on the basis of the load degree of VNF instances to obtain a sub-optimal solution with low computational complexity. We also provide a packet-level simulation to verify the performance of our method when QoS degrades due to traffic demand rapidly increasing. Simulation results show that our method reduces the delay degradation and its duration compared with the control scheduling method that optimizes performance at the completion of controls.

Mobile networks require high availability to provide reliable connectivity for various mobile services. Therefore, when a service outage occurs due to mobile base station (BS) failures, mobile network operators need to immediately resolve the effects of the outage. Cell Outage Compensation (COC) is the critical technology that resolves the outage. The COC method is composed of two steps: antenna selection from antennas of neighboring BSs and optimization of the tilts of selected antennas to provide coverage in the outage area. Although most existing works on COC methods focus on the tilt optimization algorithm, the antenna selection algorithm has not been fully discussed. Since the COC method obtains a solution by optimizing the tilts of selected antennas, a poor antenna selection causes performance degradation of the COC solution. This paper proposes an antenna selection algorithm considering the positional relationship between the outage area and neighboring antennas. Moreover, we use deep reinforcement learning (DRL) in our algorithm to find the optimal antenna selection policy. The simulation results show that the COC method with our algorithm finds a practical solution within one minute and outperforms existing selection algorithms in terms of coverage in the outage area and overlap of coverage areas.

With the 5th Generation mobile communication system (5G), ultrahigh capacity and ultralow latency communication are realized. Toward the next generation, i.e., 6th Generation mobile communication system (6G), even higher capacity or lower latency is needed. This paper focuses on the significant processing delay in the control plane when a User Equipment (UE) state transitions from Radio Resource Control (RRC)-IDLE to RRC-CONNECTED. On the transition, many interactions exist between a UE and a Base Station (BS) or a mobile core system, such as a connection establishment or context data exchange. We propose eliminating the effect of processing delay by introducing an Always-Connected Enablement BS (ACE-BS). We conduct experiments in an actual local 5G environment with UEs, BSs, and mobile cores, demonstrating the low latency communication by using the ACE-BS.

Edge computing is a new paradigm to provide computing capability at the edge servers close to end devices. A significant research challenge in edge computing is finding efficient task offloading to edge and cloud servers considering various task characteristics and limited network and server resources. Several reinforcement learning (RL)-based task-offloading methods have been developed, because RL can immediately output efficient offloading by pre-learning. However, these methods do not take into account clouds or focus only on a single cloud. They also do not take into account the bandwidth and topology of the backbone network. Such shortcomings strongly limit the range of applicable networks and degrade task-offloading performance. Therefore, we formulate a task-offloading problem for multi-cloud and multi-edge networks considering network topology and bandwidth constraints. We also propose a task-offloading method that is based on cooperative multi-agent deep RL (Coop-MADRL). This method introduces a cooperative multi-agent technique through centralized training and decentralized execution, improving task-offloading efficiency. Simulations revealed that the proposed method can minimize network utilization and task latency while minimizing constraint violations in less than one millisecond in various network topologies. It also shows that cooperative learning improves the efficiency of task offloading. We demonstrated that the proposed method has generalization performance for various task types by pre-training with many resource-consuming tasks.

Edge computing is a new paradigm to provide computing capability at the edges close to end devices. A significant research challenge in edge computing is finding an efficient task offloading to edge and cloud servers, considering various task characteristics and limited network and server resources. Several studies have proposed the reinforcement learning (RL) based task offloading method, because RL can immediately output the efficient offloading by pre-learning. However, due to the performance problem of RL, these previous studies do not consider clouds or focus only on a single cloud. They also do not consider the bandwidth and topology of the backbone network. Such shortcomings could lead to degrading the performance of task offloading. Therefore, we formulated a task offloading problem for multi-cloud and multi-edge networks, considering network topology and bandwidth constraints. Moreover, we proposed a task offloading method based on cooperative multi-agent deep reinforcement learning (Coop-MADRL) to solve the performance problem of RL. This method introduces a cooperative multi-agent technique through centralized training and decentralized execution, improving the efficiency of task offloading. Simulations revealed that the proposed method drastically reduces the average latency while satisfying all constraints, compared with the greedy approach. It also revealed that the proposed cooperative learning method improves the efficiency of task offloading.

Network traffic and computing demand have been changing dramatically due to the growth of various types of network services, e.g., high-quality video delivery and operating system (OS) updates. To maximize the utilization efficiency of limited network resources, network resource control technology is required for smooth and quick operation when network demands change. Therefore, we propose a dynamic virtual network (VN) allocation method based on cooperative multi-agent deep reinforcement learning (Coop-MADRL). This method can quickly optimize network resources even while network demands are drastically changing by learning the relationship between network demand patterns and optimal allocation by using deep reinforcement learning (DRL) in advance. The key idea is to use a multi-agent technique for a reinforcement learning (RL) based dynamic VN allocation method, which can reduce the number of candidate actions per agent and can improve the performance for VN allocation. Moreover, a cooperation technique improves the efficiency of VN allocation. From results of a simulation evaluation, Coop-MADRL can calculate effective allocation within 1 s, which reduces the maximum server and link utilization and drastically reduces the constraint violations compared with that of the static VN allocation method. Furthermore, we revealed that the learning with various mixed traffic models could achieve a high generalization performance for all traffic patterns.

Network traffic and computing demand have been changing dramatically due to the growth of various types of network services, e.g., high-quality video delivery and operating system (OS) updates. To maximize the utilization efficiency of limited network resources, network resource control technology is required for smooth and quick operation when network demands change. We propose a dynamic virtual network (VN) allocation method based on cooperative multi-agent deep reinforcement learning (Coop-MADRL). This method can quickly optimize network resources even while network demands are drastically changing by learning the relationship between network demand patterns and optimal allocation by using deep reinforcement learning (DRL) in advance. The key idea is to use a multi-agent technique for a reinforcement learning (RL) based dynamic VN allocation method, which can reduce the number of candidate actions per agent and can improve the performance for VN allocation. Moreover, a cooperation technique improves the efficiency of VN allocation. From results of a simulation evaluation, Coop-MADRL can calculate effective allocation within 1 s, which reduces the maximum server and link utilization and drastically reduces the average constraint violation compared with that of the static VN allocation method.

Network traffic and computing demand have been changing dramatically due to the growth of various types of network services, e.g., high-quality video delivery and OS update. To maximize the utilization efficiency of limited network resources, network resource control technology is required for smooth and quick operation when the network demands change. We propose a dynamic virtual network allocation method based on safe multi-agent deep reinforcement learning (safe MA-DRL). This method can quickly optimize network resources even while network demands are drastically changing by learning the relationship between network demand patterns and optimal allocation by using the DRL algorithm in advance. We developed two techniques to be used with our method; safety-considerations and multi-agent. Our safety-considerations technique reduces the degree of constraint violations, such as network congestion and server overload, and our multi-agent technique improves the scalability of virtual network allocation by dividing demands into groups and assigning each group's allocation to each agent. As a result of a simulation evaluation, safe MA-DRL can calculate effective allocation within 1 s that doubles the link utilization efficiency without any constraint violations compared to the static virtual network allocation method.

Network functions virtualization (NFV) enables telecommunications service providers to provide various network services by flexibly combining multiple virtual network functions (VNFs). To provide such services with carrier-grade quality, an NFV controller must optimally allocate such VNFs into physical networks and servers, taking into account combination(s) of objective functions and constraints for each metric defined for each VNF type. The NFV controller should also be extendable, i.e., new metrics should be able to be added. One approach for NFV control to optimize allocations is to construct an algorithm that simultaneously solves the combined optimization problem. However, this algorithm is not extendable because the problem formulation needs to be rebuilt every time, e.g., a new metric is added. Another approach involves using an extendable network-control architecture that coordinates multiple control algorithms specified for individual metrics. However, to the best of our knowledge, no method has been developed to optimize allocations through this kind of coordination. In this paper, we propose an extendable NFV-integrated control method by coordinating multiple control algorithms. We also propose an efficient coordination algorithm based on reinforcement learning. Finally, we evaluate the effectiveness of the proposed method through simulations.