Reinforcement learning (RL) has been widely used in the decision-making of autonomous vehicles (AVs) in recent studies. However, existing RL methods generally find the optimal policy by maximizing the expectation of future returns, which lacks distributional treatments of risky situations. Additionally, various uncertainties arising from the environment could also cause unreliable decisions, particularly in some complex urban environments. In this paper, the fully parameterized quantile network (FPQN) is utilized to estimate the full return distribution. Then, the conditional value-at-risk (CVaR) is utilized with the return distribution information to generate uncertainty-aware driving behavior. Additionally, an uncontrolled four-way intersection is developed by the Simulation of Urban Mobility (SUMO) simulation platform, which considers both the surrounding vehicles (SVs) and pedestrians. More specifically, to simulate the real-world traffic environment, the uncertainty arising from the occlusion, and the behavior uncertainty of surrounding traffic participants are also considered. The experiment results suggest that the proposed method outperforms the baseline methods in terms of safety. Furthermore, the results also indicate that the proposed method can make reasonable decisions in some challenging driving cases in the presence of uncertainty.

The large-scale retirement of electric vehicle traction batteries poses a huge challenge to environmental protection and resource recovery since the batteries are usually replaced well before their end of life. Direct disposal or material recycling of retired batteries does not achieve their maximum economic value. Thus, the second-life use of EV batteries has become the most economical and environmentally friendly solution. However, there are still many issues facing second-life batteries (SLBs). To better understand the current research status, this article reviews the research progress of second-life lithium-ion batteries for stationary energy storage applications, including battery aging mechanisms, repurposing, modeling, battery management, and optimal sizing. Energy management strategies are reviewed to maximize the economic benefits for SLBs, and the less-demanding applications of SLBs are presented. The technical challenges and future development trends of battery reusing technologies are also discussed. Finally, the conclusions and relevant recommendations for future studies are summarized.

As one of adaptive optimal controls, the Q-learning based supervisory control for hybrid electric vehicle (HEV) energy management is rarely studied for its adaptability. In real-world driving scenarios, conditions such as vehicle loads, road conditions and traffic conditions may vary. If these changes occur and the vehicle supervisory control does not adapt to it, the resulting fuel economy may not be optimal. To our best knowledge, for the first time, the study investigates the adaptability of Q-learning based supervisory control for HEVs. A comprehensive analysis is presented for the adaptability interpretation with three varying factors: driving cycle, vehicle load condition, and road grade. A parallel HEV architecture is considered and Q-learning is used as the reinforcement learning algorithm to control the torque split between the engine and the electric motor. Model Predictive Control, Equivalent consumption minimization strategy and thermostatic control strategy are implemented for comparison. The Q-learning based supervisory control shows strong adaptability under different conditions, and it leads the fuel economy among four supervisory controls in all three varying conditions.

Deep learning plays an increasingly important role in industrial applications, such as the remaining useful life (RUL) prediction of machines. However, when dealing with multifeature data, most deep learning approaches do not have effective mechanisms to weigh the input features adaptively. In this article, a novel feature-attention-based end-to-end approach is proposed for RUL prediction. First, the proposed feature-attention mechanism is directly applied to the input data, which gives greater attention weights to more important features dynamically in the training process. This helps the model focus more on those critical inputs, and the prediction performance is therefore improved. Next, bidirectional gated recurrent units (BGRU) are used to extract long-term dependencies from the weighted input data, and convolutional neural networks are employed to capture local features from the output sequences of BGRU. Finally, fully connected networks are used to learn the above-mentioned abstract representations to predict the RUL. The proposed approach is validated in a case study of turbofan engines. The experimental results demonstrate that the proposed approach outperforms other latest existing approaches.

A novel encoder-decoder model based on deep neural networks is proposed for the prediction of remaining useful life (RUL) in this work. The proposed model consists of an encoder and a decoder. In the encoder, the Bi-directional Long Short-Term Memory Networks (Bi-LSTM) and Convolutional Neural Networks (CNN) are used to capture the long-term temporal dependencies and important local features from the sequential data, respectively. Besides, single 1*1 convolution filter in the last convolutional layer is used for dimensionality reduction. In the decoder, the fully connected networks are employed to decode the feature information to predict RUL. In addition, the proposed data-driven method can achieve end-to-end prediction, which does not need feature engineering. To evaluate the proposed model, experimental verification is carried out on a commonly used aero-engine C-MAPSS dataset. Compared with other state-of-the-art approaches on the same dataset, the effectiveness and superiority of the proposed framework are demonstrated. For example, the scoring function value of the second subset is reduced by up to 64.99% compared with the best existing result.