Several reinforcement techniques are compared to take control of Unmanned Aerial Vehicles (UAVs) and optimize communication offloading in cellular networks. Navigation actions are calculated to send the drones to the required position and turn them back when not needed. First, a use case is expressed and solved in form of a linear programming problem (ILP). Then, a Q learning algorithm is designed and evaluated to solve the same problem. Finally, a deep neural network based on Long Short Term Memory recurrent networks is used. The results of the three approaches are obtained with a real dataset extracted from the CDRs (Call Detail Records) in Milan city, Italy. It is shown that Q learning needs long convergence times to succeed to approach the ILP optimal results. Also, we demonstrate that deep neural network techniques learn much faster and mimic the ILP with very high scores.

An AI based energy management system (EMS) for microgrids is proposed. It is composed of three modules: a strategy based module, a deep learning (DL) and a reinforcement learning module (RL). This framework determines heuristically the optimal actions for the microgrid system under different time-dependent environmental conditions. In essence, a main innovation is applied to the EMS. Our deep learning algorithm uses recurrent neural networks (RNNs) instead of the habitual State Action Reward (SAR) approach (whether classical or deep). Learning is hence guided by successful actions rather than by blind exploration. A large improvement in learning rates is hence observed when compared to classical Q-learning on real datasets that present a large diversity in energy consumption profiles, acquired in French premises over a long period. It leads to question about the best appropriate reinforcement policies to adopt when solving large state environments.

Transfer learning (TL) is a useful technique that enables the wide spreading of neural networks after re-adaptation of their weights. In ths paper, two methods are introduced for transfer learning of recurrent neural networks: D-LSTM (Long Short Term Memory with deep layers) and CNN-1D (Convolutional Neural Network of One Dimension). The first is used to improve the prediction of time series when datasets are too small to obtain satisfactory results. The second enables personalizing and hence re-adaptation of an already-trained network to a new class of time series. In fact, the CNN-1D classification is applied to those real datasets to classify different behaviors in a large city. We show that our architecture drastically improves prediction when transfer learning is used in the same class of behavior but also on different classes of behaviors.

LPWANs are networks characterized by the scarcity of their radio resources and their limited payload size. To extend the efficiency of the data transmission by decreasing the traffic sent from sensors, this paper proposes a lossy compression method using known ML techniques. We embedded a pre-trained neural network directly on constrained LoRaWAN devices and we tested the trade-off between compression ratio and accuracy of the compression algorithm. This paper studies multiple aspects of the system - energy consumption, error rate due to the lossy compression, compression ratio and the impact of LSTM parameter quantization - to measure the possible strengths and weaknesses of using a dual prediction system in order to reduce transmission costs. Surprisingly, machine learning used in this context does not consume a lot of energy and it even leads to energy saving in the very constrained devices which are the sensors.

Unusual Spatio-Temporal fluctuations in cellular network traffic may lead to drastic network management misbehaviors and at least abnormal drops in quality of experience. It is also expected that the management of future cellular networks will mostly rely on machine learning and automation. In this article, we present a dynamic on-line data mining technique to detect these network anomalies allowing, network operators to pro-actively monitor and control a variety of real-world phenomena with less damage to the overall experience. To overcome the network performance degradation that can occur in real time, the network manager must imperatively and instantly identify abnormalities and hence provide a better continuous quality of service for the subscribers. Based on real cellular communication traces, we propose an automated framework, called STAD, ensuring spatio-temporal detection outliers using a combination of machine learning techniques including One-class SVM (OCSVM), Support Vector Regression (SVR) and recurrent neural networks, Long Short-Term Memory (LSTM). STAD is double checked with two real datasets of CDRs where results show high accuracy compared to the Isolation Forest and Auto-Regressive Integrated Moving Average (ARIMA) models.

This paper presents a framework based on the data analysis concept to automate the management of resources in cellular networks. Three processes are defined: identifying and detecting anomalies, analyzing the causes, and triggering adequate recovery actions. First, the proposed solution executes Deep Learning algorithms to forecast the normal behavior of the network and defines dynamic thresholds. Then, it identifies cells with peak demands and raises alarms if the measured real-time data exceeds the threshold values. Second, we define QoS optimization methods to proceed with suitable design for resource allocation as well as fault detection and avoidance. Hence, we distinguish three cases and define two classes of data: Real-time and non-real-time traffic. This solution is applied to a pre-analyzed semi-synthetic real dataset extracted from the CDRs (Call Detail Records) in Milan city, Italy. This dataset contains the Internet activity records of two months in three areas. The preliminary results elucidate the feasibility and preeminence of our proposed anomaly detection framework.

With the emergence of the Internet of Things (IoT) applications, a huge amount of information is generated to help the optimization of operational cellular networks, smart transportation, and energy management systems. Applying Artificial Intelligence approaches to exploit this data seems to be promising. In this paper, we propose a dual deep neural network architecture. It is used to classify time series and to predict future data. It is essentially based on Long Short Term Memory (LSTM) algorithms for accurate time series prediction and on deep neural network, classifiers to classify input streams. It is shown to work on different domains (cellular, energy management, and transportation systems). Cloud architecture is used for IoT data collection and our algorithm is applied on real-time energy data for accurate energy classification and prediction.

The need to integrate flexible and intelligent mechanisms for energy management becomes a necessity. In this paper, we are considering a microgrid with infrastructures having production capacities and consumption needs. Several data and constraints related to the microgrid consumption have been collected, in addition to data concerning the production of renewable energy from Photovoltaic panels (PV). Data history is used as input to a neural network to predict one day ahead of consumption and production. Then, a prioritized scheduling family of algorithms is presented. First, we introduce a mathematical formulation to our problem. Then, we propose various scenarios that go from an exact solution to heuristic-based use cases, including scheduling of several energy classes with a maximum scheduling time lapse. Results show that prioritized scheduling, including time lapse based on predictions, can give more reliable results than scheduling based on bin packing.

We propose a solution for Electric Vehicles (EVs) energy management in smart cities, where a deep learning approach is used to enhance the energy consumption of electric vehicles by trajectory and delay predictions. Two Recurrent Neural Networks are adapted and trained on 60 days of urban traffic. The trained networks show precise prediction of trajectory and delay, even for long prediction intervals. An algorithm is designed and applied on well known energy models for traction and air conditioning. We show how it can prevent from a battery exhaustion. Experimental results combining both RNN and energy models demonstrate the efficiency of the proposed solution in terms of route trajectory and delay prediction, enhancing the energy management.

Electrical vehicular (EV) energy management is a promising trend. Forecasting vehicular trajectories and delay is crucial for EV energy management. The presented work is devoted to the study and the application of deep learning techniques on specific road trajectories. First, exhaustive deep learning algorithms are considered. Second, road traces are converted to time series. Then, delays and road trajectories are analyzed. In fact, we consider two Recurrent Neural Networks (RNN): LSTM (Long Short Term Memory) and GRU (Gated Recurrent Units). Neural Networks are adapted and trained on 60 days of real urban traffic of Rome in Italy. We calculate the Loss function for both machine learning techniques which is defined by mean square error (MSE) and Root mean square error (RMSE). Experimental results demonstrate that both LSTM and GRU are adequate for the context of EV in terms of route trajectory and delay prediction.