Fully autonomous, zero-touch systems emphasizing on energy efficiency, high reliability, and ultra-low latency will be possible with the introduction of 6G networks. But with more devices and services, energy usage is expected to rise, necessitating sustainable solutions. A Decision Engine (DE) based on Hierarchical Reinforcement Learning (HRL) is presented in this research to improve the deployment of Service Function Chains (SFCs) based on Cloud-Native Functions (CNF) in dynamic contexts. The goal of the framework is to lower energy consumption while improving scalability and flexibility in the cloud, far-edge, and edge domains. By simulating actual 6G situations, we demonstrate that the HRL-based DE improves resource allocation, reduces latency by 80%, and considerably reduces energy usage by 60% compared to the flat RL. By assisting in self-optimizing network management, our method presents a viable route to intelligent, sustainable 6G networks.

In the realm of modern telecommunication ecosystems, resource-aware management is critical in ensuring quality of service (QoS) while enabling proactive, zero-touch operations. Furthermore, recent advances in software defined networking (SDN) have significantly contributed to beyond 5G (B5G) network optimization, offering innovative solutions to tackle their growing complexity. Moreover, the inherently non-convex nature of service provisioning challenges presents critical obstacles to delivering a seamless user experience. This paper introduces a novel framework that integrates Dueling Double Deep Q-Networks (Dueling DDQN) within a cloud-native edge infrastructure, enabling zero-touch service scaling. By harnessing the advanced capabilities of deep reinforcement learning (DRL), the proposed method autonomously identifies and addresses scaling demands, thereby improving overall network capabilities. The effectiveness of our approach is validated through experiments conducted on a multi-node Kubernetes testbed, demonstrating its ability to alleviate performance bottlenecks in distributed environments.

Although the next generations of wireless networks are anticipated to be enabled by Artificial Intelligence (AI), the development of practical, scalable, and efficient system models and network orchestration strategies remains a significant open challenge that requires thorough investigation. In this research work, we study a system model including two parts of AI-driven management and orchestration and AI-enabled infrastructures consisting of multiple automated domains. While the network orchestration manages the autonomous domains, the intra-domain resource allocation and network slicing are also automated and AI-enabled. Consequently, we propose an efficient and scalable strategy for automating network edge domains. This approach reduces the complexity and, thus, increases the scalability and energy efficiency by implementing one Double Deep Q-Learningbased controller in each domain that can perform network slicing for multiple service types. One agent provides dynamic network services by utilizing unsupervised discovered assignment skills in an offline phase. We implement the domain controller with multiple algorithms to find the best performance and efficiency. Finally, we compare the reliability of the algorithms, and we select the algorithm that offers the best trade-off between complexity and performance.

The 6thgeneration (6G) of mobile networks are anticipated to provide significant benefits such as high data-rates, low-latency and seamless connectivity to a huge number of users. The realization of these networking potentialities is anchored in the innovative design and development of the networking operations with purpose to deal with complex environments in a efficient manner. In particular, principal networking operations such as the resource management will be mainly conducted with the aid of the Deep Learning (DL) models. The DL models are characterized by their efficiency in dealing with intense tasks but they lack of transparency and trust. For this reason, we propose a framework that incorporates both a methodology for uncertainty quantification (UQ) of the DL models and a trafficengineering model targeting on improving the trustworthiness in the resource allocation procedures. It is shown that the exploitation of trustworthy DL models significantly improves both the system's resource utilization and the system's service provisioning capability to the users.

With the advent of beyond 5G applications, the execution of computationally intense tasks moves further closer to the network edge. Alongside the capabilities of a Multi-Access Edge Computing (MEC), smart decision-making considering sustainability aspects has become achievable. In this paper, a resource management technique utilizing Reinforcement Learning (RL) at the MEC is presented in order to promote power efficient solutions. CPU resources at the MEC are managed and distributed to several network services for their individual disposal. A direct relation between the CPU resources and power consumption at the MEC is proposed further establishing the need for efficient resource handling. A Soft-Actor Critic (SAC) approach is leveraged to learn the patterns for intelligent resource allocation minimizing the power expenditure. Further, two baseline algorithms, the Knapsack method and the proportional resource allocation scheme, are implemented to prove the dominance of the proposed RL-based algorithm. The results confirm that in the SAC-based RL implementation, the power consumption at the MEC server is lower compared to the two baseline algorithms. The promising results pave way for the deployment of RL-based algorithms for efficient performance, thus promoting green technologies at the MEC.

Cell- Free (CF) massive Multiple- Input- Multiple-Output (mMIMO) has recently gained significant research attention as a promising technology for future wireless networks. Since the conventional CF mMIMO has been considered unscalable and impractical, user-centric CF systems were proposed to improve its flexibility. While Access Point (AP) clustering has been studied in many research works as a challenging task in Radio Access Network (RAN), the limitations of the connecting links to the servers in a real scenario have not been taken into account. In this work, we consider the innovative combination of Open RAN (ORAN) and CF-RAN architecture that aims to improve the CF network limitations related to the connecting links. Then, we propose two control loops for ORAN Radio Unit (ORU) clustering and ORAN Distributed Unit (O-DU) assignment procedures that are conducted by Reinforcement Learning (RL) agents. Numerical results show that the proposed approach can successfully provide the user's requirements while the network is balanced.

Network Virtualization (NV) has proved a promising technology that allows multiple heterogeneous Virtual Networks (VNs) to operate simultaneously on the same infrastructure. Dynamic Virtual Network Embedding (NVE) has emerged as an enabler of elasticity and scalability in the VN deployment and resource allocation of the physical infrastructure. However, the key challenge in realizing NV in a sustainable way is how to dynamically embed VNs efficiently into physical network, which is defined as the VNE problem. To address this challenge, this paper proposes an approach that leverages Multi-Agent Reinforcement Learning (MARL) to solve the dynamic VNE of VNs for 5G/6G communication systems. The proposed framework consists of multiple horizontally distributed RL agents that co-operate to devise temporally dynamic VNE placements. The key contributions of this work are introducing a novel dynamic VNE orchestration framework for multi-domain networks based on Distributed RL, providing a scalable VNE framework targeted to Ultra-Reliable Low-Latency Communication (URLLC) services, evaluating and comparing the proposed algorithm with existing solutions in the state of the art. The paper concludes that there is a significant improvement in latency of 144.191% when compared to the baselines.

The evolution of the telecommunication networks towards their 6th generation has emerged new research directions in order to deal with the challenges that are posed by the high-complexity of the new infrastructure. In this new era, the structuring of the network management infrastructure is a complex endeavor that should efficiently control both communication and computational resources. In this paper, we present a set of strategies for managing the computational load in a cell-free-based, converged optical-wireless 6G network. The proposed approaches target to control the computational load of a cell-free network, by either compressing the load by considering load-thresholds, or offloading the load to the SDN controller of the fixed network. Both strategies are mathematically formulated by considering traffic-engineering formulas, while their performance is evaluated by comparing analytical results with corresponding results from a baseline scenario, where no load control strategies are applied. Moreover, the proposed analysis can be applied in order to determine the capacity of the network controllers that is required in order to guarantee pre-determined Quality of Service requirements.

The evolution of Software-Defined Networking (SDN) and Network Function Virtualization (NFV) in the telecommunications industry have intensified the issues of network management at large scales. Dynamic service orchestration and adaptive resource allocation became a necessity for network operators to manage the rapid growth of users and data-intensive applications. The impact of network automation on energy consumption and overall operating costs is often overlooked. Guaranteeing strict performance constraints of Ultra-Reliable Low Latency Communication (URLLC) services while enhancing energy efficiency is one of the major critical problems of future communication networks, given the urgency to reduce carbon emissions and energy consumption. In this work, we study the problem of zero-touch Service Function Chain (SFC) orchestration for multi-domain networks, targeting the latency reduction of URLLC services while improving energy efficiency for beyond-5G networks. Specifically, we propose SCHE2MA, a Service CHain Energy-Efficient Management framework based on distributed Reinforcement Learning (RL), that can intelligently deploy SFCs with shared VNFs per se into a multi-domain network. Finally, we evaluate SCHE2MA through model validation and simulation while demonstrating its ability to jointly reduce average service latency by 103.4% and energy consumption by 17.1% compared to a centralized RL solution.

With the evolution of 5G networks towards fully software, Network Slicing (NS) has emerged as a new paradigm to create a set of customised logical network instances on top of the same physical infrastructure. Telecom operators provide “slices” of their networks to different industries (e.g., automotive, Industrial Internet of Things (IIoT), and eHealth), targeting to increase the resource utilization and Return on Investment (ROI). However, as the service demand increases, slice management becomes particularly complex, especially in the Radio Access Network (RAN) section, since virtualized infrastructures are significantly more complex to manage compared to physical networks. In this paper, a novel 5G New Radio (5G NR) NS management framework is presented, which targets to define the optimal NS parameterization for the entire Fronthaul (FH) 5G network. Our solution is divided into three phases: i) definition of the 5G Downlink Hybrid Automatic Repeat Request (DL-HARQ) parameters (K0, K1), ii) the dynamic (inter-slice) partitioning of the radio resources among the slices, inline with the Service Level Agreement (SLA), and, iii) an intra-slice scheduling algorithm for the user prioritization, given real-time user's traffic Key Performance Indicators (KPIs) analysis. To the best of our knowledge, the proposed solution is the first attempt of a real-time NS management framework for the complete FH section, involving frame granularity service customization.

Smart homes that contain renewable energy sources, storage systems, and controllable loads will be key components of the future smart grid. In this article, we develop a reinforcement-learning (RL)-based scheme for the real-time energy management of a smart home that contains a photovoltaic system, a storage device, and a heating, ventilation, and air conditioning (HVAC) system. The objective of the proposed scheme is to minimize the smart home’s electricity cost and the residents’ thermal discomfort by appropriately scheduling the storage device and the HVAC system on a daily basis. The problem is formulated as a Markov decision process, which is solved using the deep deterministic policy gradient (DDPG) algorithm. The main contribution of our study compared to the existing literature on RL-based energy management is the development of a clustering process that partitions the training data set into more homogeneous training subsets. Different DDPG agents are trained based on the data included in the derived subsets, while in real time, the test days are assigned to the appropriate agent, which is able to achieve more efficient energy schedules when compared to a single DDPG agent that is trained based on a unified training data set.

Future mobile networks will offer high data rates based on high-capacity fronthaul. Current fronthaul design has two main components that communicate via the common public radio interface and fiber links, i.e., remote units (RUs) that implement simple signal processing and centralized baseband units (CBBUs) in high power-consuming data centers that perform complex network functions. Various functional splits between CBBUs and RUs are feasible, inducing trade-offs between centralization gains and bandwidth demands. This design lacks in capacity and flexibility, motivating the use of converged fiber-wireless (Fi-Wi) fronthaul with high-bandwidth fiber and millimeter-wave links, and splits that move functionalities to RUs reducing the delay demands. Further flexibility is offered by analog radio-over-fiber fronthaul that supports dynamic functional splitting via software-defined networking (SDN). Ensuring acceptable delay for all RUs, i.e., minimizing fronthaul grade-of-service (GoS), requires selection of CBBUs, channel bandwidth and functional splits of RUs. The split type affects fronthaul power consumption determining which fronthaul components are active and their processing power. Using a simulated annealing-based dynamic fronthaul resource allocation (DFRA) scheme, we jointly optimize GoS and power consumption in a novel SDN Fi-Wi fronthaul. Our results show that DFRA minimizes GoS and power consumption for all load levels outperforming baseline approaches.

Network Slicing (NS) represents a key technology enabler for advanced connectivity and data processing tailored to customers' specific requirements. While significant progress has already been achieved for Core NS, Radio Access Network (RAN) slicing still presents limitations in terms of sharing infrastructure, Service Level Agreement (SLA) guarantees, isolation, resource scheduling and allocation. In this context, this paper firstly introduces a novel slices configuration framework for the 5G New Radio (5G NR) infrastructure able to dynamically migrates the radio resources among the slices, while preserving the Quality of Service (QoS) of the served users. Our solution is illustrated in detail and tested on top of a real case 5G scenario, using a software-based simulator. Finally, this paper investigates the flexibility, scalability, and real-time properties of the proposed method, as required in the future 5G cloud-based architectures.

5G New Radio's (NR) spectrum expansion towards higher bands, although critical towards achieving the envisioned 5G capacity requirements, creates the need for installing a very large number of Access Points (APs), which asserts tremendous capital burden on the Mobile Network Operators. Current centralization solutions such as the Cloud Radio Access Network (C-RAN) alleviate partially the costs of densification by moving the majority of radio processing functionalities from the Remote Radio Heads (RRHs) to the central Base Band Unit (BBU), but still require very high-speed Point-to-Point links between the BBU and each RRH mainly due to the digitized Common Public Radio Interface (CPRI) that is excessively inefficient for hauling broadband signals. In this article, we present a novel architecture that employs an analog converged Fiber-Wireless scheme in order to create a very spectrally efficient Point-to-Multipoint network capable of interconnecting a large number of APs, while allowing compatibility with mature Ethernet-based low-cost equipment. Preliminary simulation results show very low end-to-end Ethernet packet delay, well below eCPRI's 100 μs mark, even for fiber lengths up to 10 km, indicating the suitability of our solution for employment in 5G NR large-scale fronthaul networks.

The 5G networks are expected to satisfy diverse use cases and business models with significant advancements in terms of capacity, reliability, and latency. The allocation and provisioning of network resources pose a challenge for this novel architecture to guarantee higher flexibility and quality of service. As a potential enabler, network slicing was proposed as an innovative approach for the control of the network resources. Although a static slicing approach can be suitable for the transport and core network, the stochastic behavior of the wireless channel requires fast and secure slicing techniques for resource allocation. In this paper, we propose a dynamic slicing approach for the radio access network, where the network resources are carefully assigned to guarantee the service level agreements and increase the number of served users. To prove the performance of our approach, we implemented a fronthaul testbed to emphasize the strength of our method in terms of throughput and resource utilization, compared to static slicing.

Fifth Generation (5G) Cloud-Radio Access Networks (C-RANs) are about to exploit both optical and Millimeter Wave (mmWave) technology to meet the ever-increasing traffic demands. In this new type of converged Fiber-Wireless (FiWi) systems efficient Medium Transparent-Medium Access Control (MT-MAC) protocols should be designed, able to satisfy the very strict 5G service requirements. To this end, in this paper, we propose an MT-MAC protocol for mmWave Analog Radio-over-Fiber (A-RoF) C-RANs, which employs gated service, according to which users are granted transmission windows equal to the number of bytes contained in their buffer. An analytical model is also proposed for the mean packet delay, which is verified by means of simulation for different fiber length values, network load conditions and optical capacity values. Our results not only prove the accuracy of the proposed model but also the suitability of the proposed MT-MAC protocol to meet the sub-ms delay challenge of latency-critical 5G network requirements.

We present a novel model for the optimal design and power management of a group of buildings with diverse load patterns that are able to exchange energy through a common dc bus. The determination of the optimal sizes of the photovoltaic arrays, energy storage systems and inverters, and the optimal scheduling of power exchanges are achieved through the formulation of a mixed integer linear programming problem. Furthermore, the Nash bargaining method is used in order to fairly distribute the cooperation profits among the participants. The proposed approach achieves the reduction of the microgrid's (MG's) cost and carbon emissions compared to noncooperative approaches, and promotes the enhancement of the MG's energy sufficiency by allowing energy exchanges among buildings with energy surplus and buildings with energy deficit. Our model also takes into account the case where additional buildings join the MG after the initial coalition establishment.

The upcoming generation of mobile networks is expected to serve numerous mobile users with high quality-of-service (QoS) demands, requiring high-capacity fronthaul. As the provision of fiber connections directly to the end users is not cost-efficient, the integrated fiber wireless (FiWi) fronthaul design based on wireless networking and passive optical networks (PONs) has been proposed. The FiWi design involves modern networking technologies that can accommodate the need for data rates in the Gb/s scale and low delay, such as the wavelength division multiplexing (WDM) in the optical domain and the multiple input multiple output (MIMO) communication over millimeter wave (mmWave) spectrum in the wireless domain. The co-existence of two network types requires resource management in a medium transparent manner, i.e., the sharing of the bandwidth in the wireless domain should allow the organization of the data packets in optical frames. As the traffic circulating in the FiWi fronthaul involves packets of different priorities, i.e., different QoS classes, the resource management scheme should support QoS differentiation. To this end, we propose a resource management scheme for FiWi fronthaul and we extensively study its performance in terms of experienced delay and throughput. Our simulation results demonstrate that the proposed scheme significantly reduces the delay of the high priority class.

Facing the increasing energy demands associated with the perspective of fifth generation (5G) wireless networks, the Mobile Network Operators (MNOs) are motivated to gradually convert their traditional Radio Access Network (RAN) infrastructure to more flexible and power efficient centralized architectures, i.e., Cloud-RAN (C-RAN). Apart from their promising benefits in terms of management and network optimization, these new architectures further enable the sharing of spectrum and network elements, such as the Remote Radio Heads (RRHs) and the Baseband Units (BBUs), among multiple operators. In this paper, we introduce a novel scheme based on coalitional game theory to identify the potential room for cooperation among different MNOs that provide service to the same area. The proposed scheme sets the rules for profitable collaboration and identifies the core formation conditions (i.e., pricing) for various scenarios with different market and spectrum shares among three operators. Our results show that i) cooperation among subcoalitions of MNOs is always beneficial, yielding both higher revenues and enhanced Quality of Service (QoS) for the end users, and ii) the cooperation of all operators (grand coalition) is profitable for given user pricing in different scenarios.

Full-Duplex (FD) wireless and Device-to-Device (D2D) communication are two promising technologies that aspire to enhance the spectrum and energy efficiency of wireless networks, thus fulfilling key requirements of the 5thgeneration (5G) of mobile networks. Both technologies, however, generate excessive interference, which, if not managed effectively, threatens to compromise system performance. To this direction, we propose two transmission policies that enhance the communication of two interfering FD-enabled D2D pairs, derived from game theory and optimization theory. The game-theoretic policy allows the pairs to choose their transmission modes independently and the optimal policy to maximize their throughput, achieving significant gains when the pairs interfere strongly with each other.

We demonstrate for the first time an analytical model for computing the end to end packet delay of an Optical/Wireless 60GHz Radio-over-Fiber (RoF) network operating under the Medium-Transparent MAC (MT-MAC) protocol. The model takes into account contention both at the optical and the wireless layer, effectively incorporating the MT-MAC mechanism for seamless and dynamic capacity allocation over both optical and wireless transmission media. Based on this model, we provide an extensive delay performance analysis of the Medium Transparent MAC protocol for various optical capacity availability scenarios, varying load conditions and optical network fiber lengths. The theoretical results are found to be in good agreement with respective simulation-based findings, confirming that the employment of Medium Transparent MAC protocols can allow for efficient incorporation of the MT-MAC scheme into the upcoming era of 5G mm-wave small-cell networks.

Optical Code Division Multiple Access (OCDMA) provides increased security communications with large dedicated bandwidth to end users and simplified network control. We analyse the call-level performance of an OCDMA Passive Optical Network (PON) configuration, which accommodates multiple service-classes with finite traffic source population. The considered user activity is in accordance with the bursty nature of traffic, so that calls may alternate between active (steady transmission of a burst) and passive states (no transmission at all). Parameters related to multiple access interference, additive noise, user activity and number of traffic sources are incorporated to our analysis, which is based on a two-dimensional Markov chain. An approximate recursive formula is derived for efficient calculation of call blocking probability. Furthermore, we determine the burst blocking probability; burst blocking occurs when a burst delays its returning from passive to active state. The accuracy of the model is completely satisfactory and is verified through simulation. Moreover, we reveal the consistency and necessity of the proposed model.

In this paper, we consider a single link that supports both elastic and adaptive traffic of Batch Poisson arriving calls, under the Bandwidth Reservation (BR) policy, whereby we can achieve specific QoS per service-class. Arriving batches have a generally distributed batch size, and can be serviced either as a whole or in part (partial batch blocking discipline), depending on the available link bandwidth. Blocked calls are lost. Accepted calls of a batch can compress or expand their bandwidth; elastic calls expand or compress their service time accordingly, while adaptive calls do not alter their service time. This system does not have a Product Form Solution. For the efficient calculation of time and call congestion probabilities as well as link utilization, we derive approximate but recursive formulas. The accuracy of the model is completely satisfactory and is verified together with the model's consistency, through simulation. Comparison of the new model with existing models reveals its necessity.

We present a new loss model for the call-level analysis of a single link, which accommodates calls of different service-classes with elastic bandwidth requirements. Calls arrive in the link according to a Batch Poisson process, a process that can be used to model traffic, which is more 'peaked' and 'bursty' than the Poisson process. The available link bandwidth is shared to calls according to the Bandwidth Reservation policy, whereby we can guarantee certain Quality-of-Service for each service-class. In the proposed model, we assume a general batch size distribution and the Partial Batch Blocking discipline. According to this discipline, one or more calls of an arriving batch can be accepted, while the rest can be discarded, depending on the available link bandwidth. New and in-service calls tolerate bandwidth compression/expansion. The analysis of the system is based on Markov chains. Since no Product Form Solution exists, for an efficient solution, we propose an approximate reversible Markov chain. Based on it, we derive a recursive formula for the calculation of link occupancy distribution and consequently time and call congestion probabilities (important call-level performance metrics). The proposed model's accuracy and its consistency are verified by simulation and found to be quite satisfactory.

We investigate the performance of the IEEE 802.11e in respect of end-to-end delay, which is estimated by the sum of queuing delay and MAC delay. The MAC delay analysis is performed based on elementary probability theory (conditional probabilities) while avoiding the complex Markov Chain method. A comprehensive study of the MAC delay is presented by providing higher moments of the MAC delay distribution. To this end, we use the Z-transform of the backoff duration. The first moment corresponds to the mean MAC delay, while the second moment corresponds to the standard deviation of the MAC delay; the latter depicts the jitter. We also estimate the probability mass function (pmf) of the MAC delay through the lattice Poisson algorithm. As far as the queuing delay is concerned, we provide the mean queuing delay by considering a queuing system with one queue per Access Category (AC) per mobile station, with a single server (the wireless medium), common to all mobile stations, and a Markov modulated Poisson process as input, that expresses the bursty nature of Internet traffic. The presented analytical model provides results of the mean end-to-end delay for both saturated and non-saturated channel conditions.