Recently, we have witnessed an operational deployment of 5G in big cities to increase the communication capacity of users. However, with the ever-increasing of devices in dense networks, the current 5G could fail due to its unscalable issues. Scientists in academia and industry are actively analyzing this open issue within the context of Beyond 5G (B5G) and 6G before their realistic deployment. Moreover, the high densities of devices lead to the challenges of over-energy consumption and unfair Quality of Service (QoS) caused by the distances from the base station and existing obstacles separating the communicating devices. The network relay election method can be seen as an appropriate option to overcome these challenges, but this solution should be optimized according to the environment's dynamics. Therefore, in this paper, we propose an energy-efficient election-based method that selects relays so that it maximizes the overall network lifetime while maintaining a certain level of QoS. Considering that our system is deployed in an unknown dynamic environment, we employ a deep reinforcement learning technique that aims to optimize the network relay selection, maximize the residual energy levels of devices, and ensure fair and high data rates. Our system has been tested through a discrete-event simulator, and its performances have been evaluated and compared to a set of baseline and benchmark methods. It has been shown that our proposed system outperforms existing relevant solutions.

Mobile communication networks could make a significant difference in rescuing affected people in post-disaster scenarios. However, the existing communication infrastructures tend to be out of service in such scenarios. To solve this issue, Unmanned Aerial Vehicles (UAVs) could be launched as flying base stations to provide the required coverage to Rescue Members (RMs) and allow them to communicate and transmit crucial information through the established links. Meanwhile, with the unpredictable movements of RMs, three serious issues are affecting the deployment of UAVs: (i) the control of their mobility, (ii) their limited energy capacity, and (iii) their restricted communication ranges. Aiming to address these issues, we propose deploying an intelligent connected group of energy-efficient UAVs assisting RMs and providing them communication coverage in the long run. These requirements are satisfied using a deep reinforcement learning strategy to learn the environment dynamics and make good trajectory decisions. Simulation experiments have demonstrated the potential of our framework compared to baseline methods to provide temporary communication networks for emergency response teams during disaster relief missions.

Most of the recent Internet of Things (IoT) applications are highly dependent on the freshness of collected data from IoT devices. Therefore, the concept of Unmanned Aerial Vehicle (UAV) assisted IoT has received a lot of interest where UAVs are deployed as data collectors and can effectively meet the requirement of IoT applications in terms of reducing the Age of Information (AoI) metric of collected data. However, as it is widely known, UAVs are energy-constrained devices with limited computational and communication capacities, preventing them from timely completing their missions. This energy issue can be addressed by deploying fixed charging stations, but this option would interrupt UAVs from their missions, and they should regularly return to these stations to charge their batteries. In this paper, a UAV data collector is dispatched to serve a set of IoT devices while being permanently followed and supported by an Unmanned Ground Vehicle (UGV) as a mobile charging station. Considering an unknown dynamic IoT environment, the problem of AoI minimization is reformulated as a Markov Decision Process (MDP). Indeed, we employ the multi-agent deep Q-network (MADQN) method called AGAIN to minimize the average AoI of IoT devices, timely recharge the UAV using UGV, and optimize the UGV trajectory on the ground by considering the terrestrial obstacles and the IoT movements. A series of simulations are run to demonstrate the efficiency of the proposed method in reducing the AoI of IoT devices compared to baseline methods.

Unmanned aerial vehicles (UAVs) have attracted much attention lately and are being used in a multitude of applications. But the duration of being in the sky remains to be an issue due to their energy limitation. In particular, this represents a major challenge when UAVs are used as base stations (BSs) to complement the wireless network. Therefore, as UAVs execute their missions in the sky, it becomes beneficial to wirelessly harvest energy from external and adjustable flying energy sources (FESs) to power their onboard batteries and avoid disrupting their trajectories. For this purpose, wireless power transfer (WPT) is seen as a promising charging technology to keep UAVs in flight and allow them to complete their missions. In this work, we leverage a multiagent deep reinforcement learning (MADRL) method to optimize the task of energy transfer between FESs and UAVs. The optimization is performed by carrying out three essential tasks: 1) maximizing the sum-energy received by all UAVs based on FESs using WPT; 2) optimizing the energy loading process of FESs from a ground BS; and 3) computing the most energy-efficient trajectories of the FESs while carrying out their charging duties. Furthermore, to ensure high-level reliability of energy transmission, we use directional energy transfer for charging both FESs and UAVs by using laser beams and energy beam-forming technologies, respectively. In this study, the simulation results show that the proposed MADRL method has efficiently optimized the trajectories and energy consumption of FESs, which translates into a significant energy transfer gain compared to the baseline strategies.

Routing data in Vehicular Ad hoc Networks is still a challenging topic. The unpredictable mobility of nodes renders routing of data packets over optimal paths not always possible. Therefore, there is a need to enhance the routing service. Bus Rapid Transit systems, consisting of buses characterized by a regular mobility pattern, can be a good candidate for building a backbone to tackle the problem of uncontrolled mobility of nodes and to select appropriate routing paths for data delivery. For this purpose, we propose a new routing scheme called Bus-based Routing Technique (BRT) which exploits the periodic and predictable movement of buses to learn the required time (the temporal distance) for each data transmission to Road-Side-Units (RSUs) through a dedicated bus-based backbone. Indeed, BRT comprises two phases: (i) Learning process which should be carried out, basically, one time to allow buses to build routing tables entries and expect the delay for routing data packets over buses, (ii) Data delivery process which exploits the pre-learned temporal distances to route data packets through the bus backbone towards an RSU (backbone mode). BRT uses other types of vehicles to boost the routing of data packets and also provides a maintenance procedure to deal with unexpected situations like a missing nexthop bus, which allows BRT to continue routing data packets. Simulation results show that BRT provides good performance results in terms of delivery ratio and end-to-end delay.

It is a challenging task to develop an efficient routing solution for a reliable data delivery in urban vehicular environments. Indeed, it is difficult to find a shortest end-to-end connected path especially in urban city given the mobility pattern of the vehicles and the various obstructions to a clear transmission such as buildings. To overcome these difficulties, we investigate how unmanned aerial vehicles (UAVs) can assist vehicles on the ground in relaying in urban areas. In this paper, we propose UVAR (UAV-Assisted VANET Routing Protocol), a new routing technique for Vehicular Ad hoc Networks (VANets). This protocol is based on the use of the traffic density and the knowledge of vehicular connectivity in the streets. With this approach UAVs collect information about the traffic density on the ground and the state of vehicles connectivity, and exchange them with vehicles through Hello messages. These information allow UAV to place themselves so as to allow relaying data when connectivity between sole vehicles on the ground is not possible. Through vehicle-to-UAV (V2U) communication, the overall connectivity between vehicles is improved and therefore the routing process is efficiently improved. The performance of the proposed protocol is evaluated and the results to different scenarios are discussed.

Routing in Vehicular Ad hoc Networks (VANETs) is an important factor to ensure a reliable and efficient delivery of data packets. In urban environments, routing protocols must efficiently handle the constantly changing network topology and frequent disconnections due to the high mobility and direction changes of vehicles. The challenge is greater when there are traffic lights fixed along intersections which affect directly the mobility and therefore can greatly impact routing in urban areas. In our previous work [1] we have proposed IRTIV (Intelligent Routing protocol using real time Traffic Information in urban Vehicular environment) that takes into account the real time traffic variation without any use of pre-installed infrastructures or additional messages. However, IRTIV does not take into consideration the traffic lights impact. In this paper, we propose ETAR (Efficient Traffic Light Aware Routing Protocol for Vehicular Networks). This protocol's objective is to find the most stable path for delivering data packets based on traffic lights and traffic density of vehicles using the periodical exchange of Hello messages. We present simulation-based performance results, which show that the proposed protocol increases the packet delivery ratio and reduces the end-to-end delay.