Abstract From a future perspective and with the current advancements in technology, deep reinforcement learning (DRL) is set to play an important role in several areas like transportation, automation, finance, medical and in many more fields with less human interaction. With the popularity of its fast‐learning algorithms there is an exponential increase in the opportunities for handling dynamic environments without any explicit programming. Additionally, DRL sophisticatedly handles real‐world complex problems in different environments. It has grasped great attention in the areas of natural language processing (NLP), speech recognition, computer vision and image classification which has led to a drastic increase in solving complex problems like planning, decision‐making and perception. This survey provides a comprehensive analysis of DRL and different types of neural network, DRL architectures, and their real‐world applications. Recent and upcoming trends in the field of artificial intelligence (AI) and its categories have been emphasized and potential challenges have been discussed.

Vehicular Ad Hoc Networks (VANETs) is enhancing comfort and traffic control and have brought about a paradigm shift in the design of contemporary transportation systems. However, as smart sensing technologies become more widely used with the advent of the Internet of Things (IoT), intruders have found vehicular sensor networks to be a soft target. In this article, an optimized Attention Induced Multi Head Convolutional Neural Network for Intrusion Detection System in VANETs (AIMHCNN-IDS-VANET) is proposed. The data is collected from the CAN_HCRL_OTIDS dataset. This data is fed to a pre-processing segment where Tanh-based normalization (ThN) is used to normalize the data. Then, the pre-processed data serves as input to AIMHCNN which classifies the data into denial of service (DoS) attack, fuzzy attack, impersonation attack, and normal (attack-free). In general, AIMHCNN doesn’t express some adaption of optimization approaches to determine optimal parameters to assure accurate classification of attack detection. Hence, the Capuchin search optimization algorithm is proposed to enhance the weight parameter of the AIMHCNN classifier, which precisely classifies the IDS. The proposed method is implemented and its efficacy is analyzed on several performance parameters. The method is observed to attain higher accuracy, higher precision, and higher specificity when compared with existing methods.

Energy management systems (EMS) in smart grids provide end users with the optimal operational efficiency of power from nonsmart microgrids, including power grids, energy storage systems (ESS), and residential loads. This article proposes a novel distributed online control policy for Ambient Intelligence (AmI)-based Internet of Things (IoT) environments, optimizing a consensus utility function, including electricity cost and the lifespan of ESS. Different from the existing methods, the distributed EMS via IoT can gain cooperative$\boldsymbol {L_{2}}$performance by rejecting external disturbances and providing consensus policies for robust optimal charging and discharging. First, consensus dynamics of AmI-agents are constructed, and the Hamilton–Jacobi-Isaacs (HJI) equations are established, where the Nash equilibrium points are approximated by ADP and zero-sum game theory. Second, with the aid of an actor-critic structure, a robust optimal distributed control algorithm in an online manner for EMS is proposed. Therefore, collecting sample sets and training offline are completely avoided. Third, to deal with the unknown internal dynamics of ESS, the$Q$-learning algorithm is employed instead of system identification techniques that require available sample sets. The algorithm guarantees that the global load is balanced and that the consensus tracking error and the function approximation error are uniformly ultimately bounded. Finally, numerical simulations are provided to verify the effectiveness of the proposed algorithm for a large-scale system of nonsmart microgrids.

Channel selection is a challenging task in cognitive radio vehicular networks. Vehicles have to sense the channels periodically. Due to this, a lot of time is wasted which could have been utilised for transmission of data. Employing road side units (RSUs) in sensing can prove to be useful for this purpose. The RSUs may select the channel and allocate it to the vehicles on demand. However, this sensing should be proactive. RSUs should know in advance the channel to be allocated when requested. For this purpose, a deep reinforcement learning algorithm namely deep reinforcement learning based optimal channel selection is proposed in this study for training the network according to the previously sensed data. Proposed protocol is simulated and results are compared with the existing methods. The packet delivery ratio is increased by 2%, throughput is increased by 1.8%, average delay is decreased by 2% and primary user collision ratio is reduced by 3.2% when compared with similar recent work by varying number of vehicles. On the other hand, when compared with similar recent work by varying channel availability, the packet delivery ratio is increased by 4.5 %, throughput by 4.3%, average delay is decreased by 3% and PU collision ratio by 5.5%.

Road accidents and safe driving are one of the main concerns of transportation systems and the companies that explore different solutions to reduce the accident rate. The most interesting option to achieve this goal is through an on-board training of professional drivers to apply safe driving techniques during their work activity. The purpose of this study is to analyze a monitoring system that is not limited to the real-time vehicle tracking but is also capable of monitoring and providing real-time feedback and in-vehicle training. We analyze the influence of different sociodemographic factors on driving behavior. The analyzed data correspond to an urban public transport company, obtained from a study performed on 246 drivers. The drivers received training based on a blended learning system with an on-board feedback device, accompanied by both theoretical and practical sessions. The driving behavior of each driver is obtained from the data gathered from the vehicles that allow us to characterize their driving patterns. The information related to safe driving is completed with a list of the records of road accidents. The results of the sociodemographic influence on driving behavior provide significant information, giving an elaborated classification of safety driving patterns in order to apply intelligent transportation systems.

Majority of research contributions in wireless access in vehicular environment (WAVE)/IEEE 802.11p standard focus on life critical safety-related applications. These applications require regular status update of vehicle’s position referred to as beaconing. Periodic beaconing in vehicle to vehicle communication leads to severe network congestion in the communication channel. The condition worsens under high vehicular density where it impacts reliability and upper bound latency of safety messages. In this paper, WAVE compliant enhancement to the existing IEEE 802.11p protocol is presented which targets prioritized delivery of safety messages while simultaneously provisioning the dissemination of nonsafety messages. Proposed scheme relies on dynamic generation of beacons to mitigate channel congestion and inefficient bandwidth utilization by reducing transmission frequency of beacons. Through the use of clustering mechanism, different beaconing frequencies and different data transmission rates are assigned to prioritize vehicular mobility. Through extensive simulation results, the performance of the proposed approach is evaluated in terms of a wide range of quality of service (QoS) parameters for two different transmission ranges. Results show that the proposed protocol provides significant enhancement and stability of the clustered topology in vehicular ad hoc network over existing standard and other protocols with similar applications.