IT systems monitoring is a crucial process for managing and orchestrating network resources, allowing network providers to rapidly detect and react to most impediment causing network degradation. However, the high growth in size and complexity of current operational networks (2022) demands new solutions to process huge amounts of data (including alarms) reliably and swiftly. Further, as the network becomes progressively more virtualized, the hosting of NFV on cloud environments adds a magnitude of possible bottlenecks outside the control of the service owners. In this paper, we propose two deep learning anomaly detection solutions that leverage service exposure and apply it to automate the detection of service degradation and root cause discovery in a cloudified mobile network that is orchestrated by ETSI OSM. A testbed is built to validate these AI models. The testbed collects monitoring data from the OSM monitoring module, which is then exposed to the external AI anomaly detection modules, tuned to identify the anomalies and the network services causing them. The deep learning solutions are tested using various artificially induced bottlenecks. The AI solutions are shown to correctly detect anomalies and identify the network components involved in the bottlenecks, with certain limitations in a particular type of bottlenecks. A discussion of the right monitoring tools to identify concrete bottlenecks is provided.

We propose a novel formulation of the “effectiveness problem” in communications, put forth by Shannon and Weaver in their seminal work “The Mathematical Theory of Communication”, by considering multiple agents communicating over a noisy channel in order to achieve better coordination and cooperation in a multi-agent reinforcement learning (MARL) framework. Specifically, we consider a multi-agent partially observable Markov decision process (MA-POMDP), in which the agents, in addition to interacting with the environment, can also communicate with each other over a noisy communication channel. The noisy communication channel is considered explicitly as part of the dynamics of the environment, and the message each agent sends is part of the action that the agent can take. As a result, the agents learn not only to collaborate with each other but also to communicate “effectively” over a noisy channel. This framework generalizes both the traditional communication problem, where the main goal is to convey a message reliably over a noisy channel, and the “learning to communicate” framework that has received recent attention in the MARL literature, where the underlying communication channels are assumed to be error-free. We show via examples that the joint policy learned using the proposed framework is superior to that where the communication is considered separately from the underlying MA-POMDP. This is a very powerful framework, which has many real world applications, from autonomous vehicle planning to drone swarm control, and opens up the rich toolbox of deep reinforcement learning for the design of multi-user communication systems.

A collaborative multi-agent reinforcement learning (RL) problem is considered, where agents communicate over a noisy communication channel towards achieving a common goal. In particular, we consider a remote-controlled version of a single-agent RL problem, in which the system state is observed by a guide agent, while the actions are taken by a scout. The guide can communicate to the scout over a noisy communication link, reminiscent of a remote-controlled version of the single-agent RL problem. This transformation turns the original single-agent Markov decision process (MDP) into a two-agent partially observable MDP (POMDP). In conventional systems, communication and learning tasks are taken care of separately. We show the suboptimality of this approach, and propose a deep Q-learning solution that aims at learning the optimal policy taking into account the channel impairments.

Management and orchestration (MANO) of resources by virtual network functions (VNFs) represents one of the key challenges towards a fully virtualized network architecture as envisaged by 5G standards. Current threshold-based policies inefficiently over-provision network resources and under-utilize available hardware, incurring high cost for network operators, and consequently, the users. In this work, we present a MANO algorithm for VNFs allowing a central unit (CU) to learn to autonomously re-configure resources (processing power and storage), deploy new VNF instances, or offload them to the cloud, depending on the network conditions, available pool of resources, and the VNF requirements, with the goal of minimizing a cost function that takes into account the economical cost as well as latency and the quality-of-service (QoS) experienced by the users. First, we formulate the stochastic resource optimization problem as a parameterized action Markov decision process (PAMDP). Then, we propose a solution based on deep reinforcement learning (DRL). More precisely, we present a novel RL approach, called parameterized action twin (PAT) deterministic policy gradient, which leverages an actor-critic architecture to learn to provision resources to the VNFs in an online manner. Finally, we present numerical performance results, and map them to 5G key performance indicators (KPIs). To the best of our knowledge, this is the first work that considers DRL for MANO of VNFs' physical resources.

Today's mobile data traffic is dominated by content-oriented traffic. Caching popular contents at the network edge can alleviate network congestion and reduce content delivery latency. This paper provides a comprehensive and unified study of caching and delivery techniques in wireless radio access networks (RANs) with caches at all edge nodes (ENs) and user equipments (UEs). Three cache-aided RAN architectures are considered: RANs without fronthaul, with dedicated fronthaul, and with wireless fronthaul. It first reviews in a tutorial nature how caching facilitates interference management in these networks by enabling interference cancelation (IC), zero-forcing (ZF), and interference alignment (IA). Then, two new delivery schemes are presented. One is for RANs with dedicated fronthaul, which considers centralized cache placement at the ENs but both centralized and decentralized placement at the UEs. This scheme combines IA, ZF, and IC together with soft-transfer fronthauling. The other is for RANs with wireless fronthaul, which considers decentralized cache placement at all nodes. It leverages the broadcast nature of wireless fronthaul to fetch not only uncached but also cached contents to boost transmission cooperation among the ENs. The numerical results show that both schemes outperform existing results for a wide range of system parameters, thanks to the various caching gains obtained opportunistically.

A fog radio access network (F-RAN) is studied, in which KTedge nodes (ENs) connected to a cloud server via orthogonal fronthaul links, serve KRusers through a wireless Gaussian interference channel. Both the ENs and the users have finite-capacity cache memories, which are filled before the user demands are revealed. While a centralized placement phase is used for the ENs, which model static base stations, a decentralized placement is leveraged for the mobile users. An achievable transmission scheme is presented, which employs a combination of interference alignment, zero-forcing and interference cancellation techniques in the delivery phase, and the \textit{normalized delivery time} (NDT), which captures the worst-case latency, is analyzed.

A KT× KRcache-aided wireless interference network, in which both the transmitters and the receivers are equipped with cache memories is studied. Each user requests one file from a library of N popular files. The goal is to design the cache contents without the knowledge of the particular user demands, such that all possible demand combinations can be satisfied reliably over the interference channel. The achievable sum degrees-of-freedom (sDoF) and the normalized delivery time (NDT) are studied for centralized and decentralized network architectures, respectively. First, using a combination of interference alignment (IA), zero-forcing (ZF) and interference cancellation (IC) techniques, a novel caching and transmission scheme for centralized networks is introduced, and it is shown to improve the sDoF upon the state-of-the-art. Then, the NDT is studied when the content placement at the receiver caches is carried out in a decentralized manner. Our results indicate that, for this particular network architecture, caches located at the receiver side are more effective than those at the transmitter side in order to reduce the NDT.

Fundamental limits of communication is studied in a 4 × 4 interference network, in which the transmitters are equipped with cache memories. Each of the receivers requests one file from a library of N equal-size files. The caches at the transmitters are filled without the knowledge of the user demands, such that all possible demand combinations can be satisfied reliably over the interference channel. The achievable normalized delivery time (NDT) is studied under centralized cache placement. By combining the interference alignment (IA) and zero-forcing (ZF) techniques, a novel caching and transmission scheme is presented, and is shown to be optimal for all possible cache sizes; fully characterizing the NDT for the 4× 4 interference network with caches at the transmitter side.