In-Network Computing (INC) is a currently emerging paradigm. Realizing INC in 6G networks could mean that user plane entities (UPEs) carry out computations on packets while transmitting them. These computations may have specific requirements in terms of their completion time. In case of high compute pressure at one UPE, migrating computations to another UPE may be beneficial, in order to avoid exceeding the completion time requirement. Centralized migration approaches suffer from increased signaling and are prone to react too slow. Therefore, this paper investigates the applicability of distributed intelligence to tackle the problem of compute task migration in the 6G User Plane. Each UPE is equipped with an intelligent agent, enabling autonomous decisions on whether computations should be migrated to another UPE. To enable the intelligent agents to learn and apply an optimal task migration policy, we investigate and compare two state-of-the-art Deep Reinforcement Learning (DRL) approaches: Advantage Actor-Critic (A2C) and Double Deep Q-Network (DDQN). We show, via simulations, that the performance of both solutions, in terms of the percentage of tasks exceeding their completion time requirement, is near-optimal and training A2C is at least 60% faster than DDQN.

The deployment of network elements in 6G is expected to be significantly more distributed than the existing 5G deployments. Distributed management paradigms are compatible with such distributed network deployments. Further, owing to their ability to solve complex problems by evaluating the impact of actions on the environment, intelligent solutions based on Reinforcement Learning (RL) for distributed management are promising. However, there are still several unsolved challenges before distributed intelligence could be seamlessly integrated in 6G. This work defines relevant research questions, reports on the progress made in the PhD project and presents the next steps and future directions for the advancement of this topic.

In future 6G networks, the deployment of network elements is expected to be highly distributed, going beyond the level of distribution of existing 5G deployments. To fully exploit the benefits of such a distributed architecture, there needs to be a paradigm shift from centralized to distributed management. To enable distributed management, Reinforcement Learning (RL) is a promising choice, due to its ability to learn dynamic changes in environments and to deal with complex problems. However, the deployment of highly distributed RL – termed massive distribution of intelligence – still faces a few unsolved challenges. Existing RL solutions, based on Q-Learning (QL) and Deep Q-Network (DQN) do not scale with the number of agents. Therefore, current limitations, i.e., convergence, system performance and training stability, need to be addressed, to facilitate a practical deployment of massive distribution. To this end, we propose improved Double Deep Q-Network (IDDQN), addressing the long-term stability of the agents’ training behavior. We evaluate the effectiveness of IDDQN for a beyond 5G/6G use case: auto-scaling virtual resources in a network slice. Simulation results show that IDDQN improves the training stability over DQN and converges at least 2 times sooner than QL. In terms of the number of users served by a slice, IDDQN shows good performance and only deviates on average 8% from the optimal solution. Further, IDDQN is robust and resource-efficient after convergence. We argue that IDDQN is a better alternative than QL and DQN, and holds immense potential for efficiently managing 6G networks.

In future 6G networks, distributed management of network elements is expected to be a promising paradigm. Recent research progress in Artificial Intelligence (AI) is rapidly driving the adoption of distributed management. However, distributed management using intelligence or distributed AI inherently suffers from a number of issues - potential conflicts, signaling required to ensure cooperation and the convergence time of the algorithm. To this end, an early understanding and analysis of the overall effort to implement distributed AI in 6G, is still unexplored. This work, therefore, examines the impact of distributed AI, by analyzing its performance and how the existing 5G architecture could be enhanced to support it in 6G. We aim to understand the impact of distributed AI in 6G by selecting a relevant beyond 5G use case - auto-scaling virtual resources in a network slice. We present the performance and architecture analysis for two distributed algorithms from the domain of Reinforcement Learning - Q-Learning and Deep Q-Networks. We argue that despite its aforementioned issues, distributed AI brings benefits such as dynamic and adaptive decision-making, making it highly applicable for certain use cases in 6G.

IT systems monitoring is a crucial process for managing and orchestrating network resources, allowing network providers to rapidly detect and react to most impediment causing network degradation. However, the high growth in size and complexity of current operational networks (2022) demands new solutions to process huge amounts of data (including alarms) reliably and swiftly. Further, as the network becomes progressively more virtualized, the hosting of NFV on cloud environments adds a magnitude of possible bottlenecks outside the control of the service owners. In this paper, we propose two deep learning anomaly detection solutions that leverage service exposure and apply it to automate the detection of service degradation and root cause discovery in a cloudified mobile network that is orchestrated by ETSI OSM. A testbed is built to validate these AI models. The testbed collects monitoring data from the OSM monitoring module, which is then exposed to the external AI anomaly detection modules, tuned to identify the anomalies and the network services causing them. The deep learning solutions are tested using various artificially induced bottlenecks. The AI solutions are shown to correctly detect anomalies and identify the network components involved in the bottlenecks, with certain limitations in a particular type of bottlenecks. A discussion of the right monitoring tools to identify concrete bottlenecks is provided.

In recent years, owing to the architectural evolution of 6G towards decentralization, distributed intelligence is being studied extensively for 6G network automation. Distributed intelligence, based on Reinforcement Learning (RL), particularly Q-Learning (QL), has been proposed as a potential direction. The distributed framework consists of independent QL agents, attempting to reach their own individual objectives. The agents need to learn using a sufficient number of training steps before they converge to the optimal performance. After convergence, they can take reliable management actions. However, the scalability of QL could be severely hindered, particularly in the convergence time - when the number of QL agents increases. To overcome the scalability issue of QL, in this paper, we explore the potentials of the Deep Q-Network (DQN) algorithm, a function approximation-based method. Results show that DQN outperforms QL by at least 37% in terms of convergence time. In addition, we highlight that DQN is prone to divergence, which, if solved, could rapidly advance distributed intelligence for 6G.

Distributed automation is expected to play a significant role in the management of 6G networks, as it avoids the drawbacks of a single point of failure and signaling overhead inherent in a centralized paradigm. However, the issue of conflicts is intrinsic to a distributed architecture and when left unaddressed, may severely impair system KPIs. Considering the conflict problem, it is unclear if distributed automation would be scalable to realize the potential of 6G networks. In this paper, we validate the scalability of distributed intelligence, specifically based on Q-Learning, Q-Learning for Cooperation (QLC), consisting of intelligent agents that learn to cooperate on a discrete state space. Results show that the performance of QLC is scalable when compared to the optimal, computed by a centralized solution. Scalability may be limited by the convergence time that increases with the number of agents and the size of the discrete state space. The cooperation overhead is also not critical. These findings indicate that QLC is promising and may be applied to other use cases if the speed of convergence is not a significant detriment in distributing intelligence in 6G.

Cognitive Autonomous Networks (CANs) promise to overcome the shortcomings of current Self-Organizing Network (SON) implementations, i.e., the limited flexibility and adaptability to changing environments, by applying cognition. In CAN, intelligent network automation functions, herein called Cognitive Functions (CFs), apply machine learning techniques to learn context-specific behavioral policies with which to automate network operations. For proper operation, the CAN system needs to learn the environment in which the functions are operating and to abstract the environment and performance observations into states to which the CFs must respond. This paper proposes a design and implementation of an Environmental-state Modeling and Abstraction (EMA) engine that could be tasked to learn the required abstract states in a consistent way across multiple CFs.