Artificial Intelligence (AI) is being incorporated in several optimization, scheduling, orchestration as well as in native communication network functions. This paradigm shift results in increased energy consumption, however, quantifying the end-to-end energy consumption of adding intelligence to communication systems remains an open challenge since conventional energy consumption metrics focus on either communication, computation infrastructure, or model development. To address this, we propose a new metric, the Energy Cost of AI Lifecycle (eCAL) of an AI model in a system. eCAL captures the energy consumption throughout the development, deployment and utilization of an AI-model providing intelligence in a communication network by (i) analyzing the complexity of data collection and manipulation in individual components and (ii) deriving overall and per-bit energy consumption. We show that as a trained AI model is used more frequently for inference, its energy cost per inference decreases, since the fixed training energy is amortized over a growing number of inferences. For a simple case study we show that eCAL for 100 inferences is 2.73 times higher than for 1000 inferences. Additionally, we have developed a modular and extendable opensource simulation tool to enable researchers, practitioners, and engineers to calculate the end-to-end energy cost with various configurations and across various systems, ensuring adaptability to diverse use cases.

Modern edge-cloud systems face challenges in ef-ficiently scaling resources to handle dynamic and unpredictable workloads. Traditional scaling approaches typically rely on static thresholds and predefined rules, which are often inadequate for optimizing resource utilization and maintaining performance in distributed and dynamic environments. This inefficiency hin-ders the adaptability and performance required in edge-cloud infrastructures, which can only be achieved through the newly proposed in-place scaling. To address this problem, we propose the Multi-Agent Reinforcement Learning-based In-place Scaling Engine (MARLISE) that enables seamless, dynamic, reactive control with in-place resource scaling. We develop our solution using two Deep Reinforcement Learning algorithms: Deep Q-Network (DQN), and Proximal Policy Optimization (PPO). We analyze each version of the proposed MARLISE solution using dynamic workloads, demonstrating their ability to ensure low response times of microservices and scalability. Our results show that MARLISE- based approaches outperform heuristic method in managing resource elasticity while maintaining microservice response times and achieving higher resource efficiency.

In this paper, we demonstrate a smart resource auto-scaling solution based on Multi-Agent Deep Reinforcement Learning (MADRL). Our decentralized approach dynamically adapts compute resources to services to ensure a smooth response to fluctuating user demands. The demonstration, consisting of a visualization and control dashboard, a cloud edge hardware configuration and an AI-based wireless fingerprinting service, shows in-place real-time scaling of resources in the virtual environment and its impact on service response time and performance.

The rise of cloud computing has been enabled by advances in virtualization and containerization technology. Over the past decade, the use of cloud computing has grown rapidly and has had a significant impact on digital transformation with many enterprises migrating to public clouds. While convenient and cost efficient, such approaches are prone to certain data privacy, compliance and security risks. The ongoing democratisation of cloud technologies represented by the increasing number of open source projects, has enabled certain enterprises to easily develop their on-premise cloud infrastructure. However, these open source projects are largely enterprise level and still too complex for small and micro enterprises and academic environments. To further decrease the on-premise infrastructure deployment and management barrier, we first provide an analysis of the existing on-premises PaaS workflows and solutions, along with the complexity of their deployment models, and identify the requirements for simple PaaS solutions for small environments with limited resources. We then introduce Kubitect as an enabler of on-premises PaaS democratization and expedite digital transformation. Kubitect is a lightweight single file declarative infrastructure configuration solution for on-premises cluster definition, instantiation and update. Our qualitative and quantitative evaluation shows the advantage of Kubitect for small environments where simplicity is more important than deploy time, assuming the latter is relatively comparable with alternative solutions.

In this paper, we build on advances introduced by the Deep Q-Networks (DQN) approach to extend the multi-objective tabular Reinforcement Learning (RL) algorithm W-learning to large state spaces. W-learning algorithm can naturally solve the competition between multiple single policies in multi-objective environments. However, the tabular version does not scale well to environments with large state spaces. To address this issue, we replace underlying Q-tables with DQN, and propose an addition of W-Networks, as a replacement for tabular weights (W) representations. We evaluate the resulting Deep W-Networks (DWN) approach in two widely-accepted multi-objective RL benchmarks: deep sea treasure and multi-objective mountain car. We show that DWN solves the competition between multiple policies while outperforming the baseline in the form of a DQN solution. Additionally, we demonstrate that the proposed algorithm can find the Pareto front in both tested environments.

Deep Reinforcement Learning (DRL) solutions are becoming pervasive at the edge of the network as they enable autonomous decision-making in a dynamic environment. However, to be able to adapt to the ever-changing environment, the DRL solution implemented on an embedded device has to continue to occasionally take exploratory actions even after initial convergence. In other words, the device has to occasionally take random actions and update the value function, i.e., re-train the Artificial Neural Network (ANN), to ensure its performance remains optimal. Unfortunately, embedded devices often lack processing power and energy required to train the ANN. The energy aspect is particularly challenging when the edge device is powered only by a means of Energy Harvesting (EH). To overcome this problem, we propose a two-part algorithm in which the DRL process is trained at the sink. Then the weights of the fully trained underlying ANN are periodically transferred to the EH-powered embedded device taking actions. Using an EH-powered sensor, real-world measurements dataset, and optimizing for Age of Information (AoI) metric, we demonstrate that such a DRL solution can operate without any degradation in the performance, with only a few ANN updates per day.

In a system with energy-constrained sensors, each transmitted observation comes at a price. The price is the energy the sensor expends to obtain and send a new measurement. The system has to ensure that sensors’ updates are timely, i.e., their updates represent the observed phenomenon accurately, enabling services to make informed decisions based on the information provided. If there are multiple sensors observing the same physical phenomenon, it is likely that their measurements are correlated in time and space. To take advantage of this correlation to reduce the energy use of sensors, in this paper we consider a system in which a gateway sets the intervals at which each sensor broadcasts its readings. We consider the presence of battery-powered sensors as well as sensors that rely on Energy Harvesting (EH) to replenish their energy. We propose a Deep Reinforcement Learning (DRL)-based scheduling mechanism that learns the appropriate update interval for each sensor, by considering the timeliness of the information collected measured through the Age of Information (AoI) metric, the spatial and temporal correlation between readings, and the energy capabilities of each sensor. We show that our proposed scheduler can achieve near-optimal performance in terms of the expected network lifetime.

Millions of battery-powered sensors deployed for monitoring purposes in a multitude of scenarios, e.g., agriculture, smart cities, industry, etc., require energy-efficient solutions to prolong their lifetime. When these sensors observe a phenomenon distributed in space and evolving in time, it is expected that collected observations will be correlated in time and space. This article proposes a deep reinforcement learning (DRL)-based scheduling mechanism capable of taking advantage of correlated information. The designed solution employs deep deterministic policy gradient (DDPG) algorithm. The proposed mechanism can determine the frequency with which sensors should transmit their updates, to ensure accurate collection of observations, while simultaneously considering the energy available. The solution is evaluated with multiple data sets containing environmental observations obtained in multiple real deployments. The real observations are leveraged to model the environment with which the mechanism interacts as realistically as possible. The proposed solution can significantly extend the sensors’ lifetime and is compared to an idealized, all-knowing scheduler to demonstrate that its performance is near optimal. Additionally, the results highlight the unique feature of the proposed design, energy-awareness, by displaying the impact of sensors’ energy levels on the frequency of updates.

Millions of sensors, cameras, meters, and other edge devices are deployed in networks to collect and analyse data. In many cases, such devices are powered only by Energy Harvesting (EH) and have limited energy available to analyse acquired data. When edge infrastructure is available, a device has a choice: to perform analysis locally or offload the task to other resource-rich devices such as cloudlet servers. However, such a choice carries a price in terms of consumed energy and accuracy. On the one hand, transmitting raw data can result in a higher energy cost in comparison to the required energy to process data locally. On the other hand, performing data analytics on servers can improve the task's accuracy. Additionally, due to the correlation between information sent by multiple devices, accuracy might not be affected if some edge devices decide to neither process nor send data and preserve energy instead. For such a scenario, we propose a Deep Reinforcement Learning (DRL) based solution capable of learning and adapting the policy to the time-varying energy arrival due to EH patterns. We leverage two datasets, one to model energy an EH device can collect and the other to model the correlation between cameras. Furthermore, we compare the proposed solution performance to three baseline policies. Our results show that we can increase accuracy by 15% in comparison to conventional approaches while preventing outages.

The massive device deployments in the Internet of Things (IoT) generate immense amounts of data that can be leveraged to improve overall network performance. This paper outlines how data gathered from correlated sensor nodes can be used to improve the timeliness of updates of another sensor node in the network. We consider a system of two correlated information sources, i.e., sensor nodes, which periodically send updates to a gateway, regarding the observed physical phenomenon distributed in space and evolving in time. The optimal use of updates in such a system greatly depends on the correlation between the two sources, and to explore this effect we investigate three different models of the covariance between independently obtained observations of the phenomenon of the interest. We extract values for the parameters in the covariance models from data coming from a real sensor network, to provide the reader with a realistic feel for scaling parameters values and the applicability of our analysis in a real scenario. We demonstrate that using correlated information results in a significant increase in device lifetime and compare our approach to others proposed in the literature.

The success of the Internet of Things (IoT) strongly depends on the development of efficient strategies for gathering and processing large amounts of data while saving energy and increasing device lifetime. This work investigates the use of correlated sources of information to improve the timeliness of data collected from sensing devices in the IoT. We consider information sources that transmit periodic updates to a gateway regarding the status of an observed process and determine the optimal update strategy for these sources. We show that there is an optimal waiting time for the first update to be sent by a secondary, correlated source such that estimation error is the lowest.