Vehicle-to-everything technologies (V2X) have become an ideal paradigm to extend the perception range and see through the occlusion. Exiting efforts focus on single-frame cooperative perception, however, how to capture the temporal cue between frames with V2X to facilitate the prediction task even the planning task is still underexplored. In this paper, we introduce the Co-MTP, a general cooperative trajectory prediction framework with multi-temporal fusion for autonomous driving, which leverages the V2X system to fully capture the interaction among agents in both history and future domains to benefit the planning. In the history domain, V2X can complement the incomplete history trajectory in single-vehicle perception, and we design a heterogeneous graph transformer to learn the fusion of the history feature from multiple agents and capture the history interaction. Moreover, the goal of prediction is to support future planning. Thus, in the future domain, V2X can provide the prediction results of surrounding objects, and we further extend the graph transformer to capture the future interaction among the ego planning and the other vehicles' intentions and obtain the final future scenario state under a certain planning action. We evaluate the Co-MTP framework on the real-world dataset V2X-Seq, and the results show that Co-MTP achieves state-of-the-art performance and that both history and future fusion can greatly benefit prediction. Our code is available on our project website: https://xiaomiaozhang.github.io/Co-MTP/

With the rapid advancement of end-to-end autonomous driving, the integration of prediction and planning has increasingly become a research focus in the field of autonomous driving. However, most existing methods do not adequately consider the robustness of driving trajectories during the trajectory generation, making them less effective in handling complex driving scenarios. To address this issue, this paper introduces Planning with Path-Informed Prediction for Autonomous Driving (PPP), which constructs a prediction-decision module that fuses multi-dimensional information by integrating the ego vehicle's potential multimodal future paths with environmental features. Moreover, we introduce a multi-stage trajectory evaluation mechanism during the trajectory generation process, which significantly enhances the system's performance in dynamic environments, thereby achieving improvements in both accuracy and robustness in complex driving scenarios. Through experiments on the nuPlan dataset, our method demonstrates exceptional competitiveness in closed-loop tests. Notably, in complex scenario tests, PPP outperforms learning-based and hybrid methods. Code will be available under https://github.com/Keria0812/PPP.

With the continuous development of Unmanned Aerial Vehicle (UAV) visual positioning technology, dynamic target detection and feature point optimization have become one of the difficult problems for UAVs to achieve high-precision visual positioning in a dynamic environment. To solve the problem of UAV target detection accuracy in a dynamic environment, this paper proposes a dynamic target detection method for airborne cameras based on background prediction and semantic compensation. Firstly, to solve the problem of high false detection rate of the traditional background difference method on the camera of moving carrier, this paper proposes a background compensation method based on region of interest prediction and uses a technique combining a scale-transformed Unscented Kalman filter (ST-UKF) and Rodrigues Formula with Perspective Transformation (RFPT) to predict the background model. Then, a moving target discrimination method based on semantic confidence is proposed to solve the problem that the traditional semantic map cannot effectively discriminate the current state of the object and leads to an excessive elimination of effective feature points; in addition, a general detection framework for airborne cameras to obtain accurate and reliable target selection boxes are proposed to improve the positioning accuracy of traditional visual positioning methods in dynamic environments, the feasibility, and innovation of the algorithm in this paper are verified through data set simulation and experimental environment.

Various artificial intelligence (AI) algorithms have been developed for autonomous vehicles (AVs) to support environmental perception, decision making and automated driving in real-world scenarios. Existing AI methods, such as deep learning and deep reinforcement learning, have been criticized due to their black box nature. Explainable AI technologies are important for assisting users in understanding vehicle behaviors to ensure that users trust, accept, and rely on AI devices. In this paper, an explainable$Q$-learning method for AV longitudinal control is proposed. First, AI control of AVs is realized by constructing a deep$Q$-network (DQN) with an intelligent driver model, with the control objective maximizing vehicle speed while preventing collisions. Then, a deep explainer for humans is developed via a Shapley additive explanation (SHAP), and a novel positive SHAP method that defines new base values is proposed to explain how individual state features contribute to decisions. Finally, statistical analyses and intuitive explanations are quantified based on SHAP tools to improve clarity. Elaborate numerical simulations are conducted to demonstrate the effectiveness of the proposed algorithm. The code is available at https://github.com/limeng-1234/Pos$\_$Shap.

Autonomous vehicles with a self-evolving ability are expected to cope with unknown scenarios in the real-world environment. Take advantage of trial and error mechanism, reinforcement learning is able to self evolve by learning the optimal policy, and it is particularly well suitable for solving decision-making problems. However, reinforcement learning suffers from safety issues and low learning efficiency, especially in the continuous action space. Therefore, the motivation of this paper is to address the above problem by proposing a hybrid Mechanism-Experience-Learning augmented approach. Specifically, to realize the efficient self-evolution, the driving tendency by analogy with human driving experience is proposed to reduce the search space of the autonomous driving problem, while the constrained optimization problem based on a mechanistic model is designed to ensure safety during the self-evolving process. Experimental results show that the proposed method is capable of generating safe and reasonable actions in various complex scenarios, improving the performance of the autonomous driving system. Compared to conventional reinforcement learning, the safety and efficiency of the proposed algorithm are greatly improved. The training process is collision-free, and the training time is equivalent to less than 10 minutes in the real world.

As artificial intelligence (AI) has already seen numerous successful applications, the upcoming challenge lies in how to realize artificial general intelligence (AGI). Self-learning algorithms can autonomously acquire knowledge and adapt to new, demanding applications, recognized as one of the most effective techniques to overcome this challenge. Although many related studies have been conducted, there is still no comprehensive and systematic review available, nor well-founded recommendations for the application of autonomous intelligent systems, especially autonomous driving. As a result, this article comprehensively analyzes and classifies self-learning algorithms into three categories: broad self-learning, narrow self-learning, and limited self-learning. These categories are used to describe the popular usage, the most promising techniques, and the current status of hybridization with self-supervised learning. Then, the narrow self-learning is divided into three parts based on the self-learning realization path: sample self-learning, model self-learning, and self-learning architecture. For each method, this article discusses in detail its self-learning capacity, challenges, and applications to autonomous driving. Finally, the future research directions of self-learning algorithms are pointed out. It is expected that this study has the potential to eventually contribute to revolutionizing autonomous driving technology.

Autonomous vehicles are expected to achieve self-evolution in the real-world environment to gradually cover more complex and changing scenarios. Reinforcement learning focuses on how agents act in the environment to maximize the cumulative reward, with a great potential to achieve self-evolution ability. However, most of reinforcement learning algorithms suffer from a low sample efficiency, which greatly limits their application in autonomous driving. This paper presents an efficient self-evolution method for any given algorithm based on the combination of Soft Actor Critic (SAC) and Behavioral Cloning(BC). First, the states of the sample trajectory in the replay buffer are separated and input into the given algorithm (algorithm with fundamental performance) to get the output label of actions such that the SAC algorithm can be guided using BC to achieve fast iteration in the direction of optimization with existing basic performance. Then, the value iteration algorithm is combined to achieve the proportion allocation of mixed gradient feedback, in order to trade off exploitation and exploration. In addition, the proposed methodology is evaluated in simulation environment taking automated speed control as an example. Experiment results show that compared with SAC algorithm, the proposed method can realize more than three times of convergence efficiency improvement, while without destroying the exploration enhancement advantage of reinforcement learning algorithm, that is, the performance is improved by 20% compared with the given algorithm (Intelligent Driver Model, IDM). The proposed method can easily extended to improve any given model no matter it is model-based or learning-based algorithm.

The explainability of complex machine-learning models is becoming increasingly significant in safety-critical domains such as autonomous driving. In this context, counterfactual explanation (CE), as an effective explainability method in explainable artificial intelligence, plays an important role. It aims to identify minimal alterations to input that can change the model’s output, thereby revealing key factors influencing model decisions. However, generating counterfactual samples might involve manually selecting input features, potentially leading to suboptimal and biased explanations. This study introduces a feature contribution guided CE generation framework to address this issue. Our method utilizes feature contributions based on Shapley values to guide the model’s focus on the most influential features. This enables end-users to quickly pinpoint the search direction in generating CEs (e.g., prioritizing the most critical features) and producing representative CEs. To comprehensively evaluate our method, we conducted experimental validation on two representative machine learning models: autonomous driving decision-making using Deep Q-Network and lane-changing prediction using deep learning. In addition, we conducted a user-centered study to evaluate the practical applicability of the SVCE in autonomous driving scenarios, which serves as a crucial validation of the presented SVCE. The results show that SVCE can help users understand and diagnose the model.

In recent years, reinforcement learning (RL) has been widely used in decision-making. However, it still faces challenges when it is applied to autonomous driving, especially in complex multi-task scenarios. This paper introduces an unsupervised reinforcement learning(URL), called an improved Contrastive Intrinsic Control (CIC), to address this problem. CIC generates skills as transferable factors between different tasks to enable multi-task expansion. By comparing skills as potential state transfers with real state transfers, the mutual information between the two serves as the curiosity that drives the agents to explore the environment and gather experience in advance. This helps the collection of valuable experiences and the acquisition of effective skills. In the multi-task expansion phase, the unified training skills are used as a prior to enabling rapid convergence in various environments. Unified training is performed without rewards, followed by repeated training on multiple downstream tasks. Experiments are conducted in a highway environment, where three different driving modes are differentiated as separate RL tasks through reward functions. The experimental results demonstrate that the proposed method possesses the ability of multi-task learning. Compared to the Deep Deterministic Policy Gradient (DDPG) baseline, it achieves a 30% to 50% improvement in convergence speed at the single-task level and a 20% to 40% improvement in the final learning performance. Furthermore, even in complex tasks, where other RL methods struggle to learn effectively, it still achieves an obvious learning ability. This approach realizes an effective combination of curiosity mechanism, and RL decision making in the multitasking domain.

Trajectory tracking is a crucial responsibility for autonomous vehicles as they strive to avoid collisions. During combined-slip emergency situations where steering and driving/braking joint control are required, the nonlinearity and coupling of tire forces become increasingly important, rendering a linear tire model-based controller ineffective and leading to degraded path-tracking performance. Such degradation can ultimately jeopardize vehicle stability. To address the aforementioned issue, we establish a hierarchical coordinated controller for four-wheel independent drive (4WID) autonomous vehicles, specifically tailored to handle combined-slip trajectory tracking and yaw stability control, considering variable tire cornering stiffness. At the upper level, a model predictive lateral motion controller is engineered based on a novel combined-slip UniTire-Ctrl model. The predictive model captures the intricate nonlinear and coupling characteristics of tire forces through an analytical expression of effective cornering stiffness. This enables the controller to account for the impact of longitudinal force on lateral motion control and coordinate the front-wheel steering angle and direct yaw moment in an efficient manner. Additionally, a linear quadratic longitudinal motion controller is developed to follow the desired longitudinal speed. The lower-level torque distribution controller is constructed to prioritize vehicle stability by minimizing tire adhesion utilization. Finally, the effectiveness of the controller under combined-slip conditions is validated through the CarSim and Matlab/Simulink co-simulation platforms, which demonstrates that the developed combined-slip motion controller with UniTire-Ctrl model exhibits superior tracking precision and stability under extreme combined-slip conditions.

Recently, thanks to the introduction of human feedback, Chat Generative Pre-trained Transformer (ChatGPT) has achieved remarkable success in the language processing field. Analogically, human drivers are expected to have great potential in improving the performance of autonomous driving under real-world traffic. Therefore, this study proposes a novel framework for evolutionary decision-making and planning by developing a hybrid augmented intelligence (HAI) method to introduce human feedback into the learning process. In the framework, a decision-making scheme based on interactive reinforcement learning (Int-RL) is first developed. Specifically, a human driver evaluates the learning level of the ego vehicle in real-time and intervenes to assist the learning of the vehicle with a conditional sampling mechanism, which encourages the vehicle to pursue human preferences and punishes the bad experience of conflicts with the human. Then, the longitudinal and lateral motion planning tasks are performed utilizing model predictive control (MPC), respectively. The multiple constraints from the vehicle’s physical limitation and driving task requirements are elaborated. Finally, a safety guarantee mechanism is proposed to ensure the safety of the HAI system. Specifically, a safe driving envelope is established, and a safe exploration/exploitation logic based on the trial-and-error on the desired decision is designed. Simulation with a high-fidelity vehicle model is conducted, and results show the proposed framework can realize an efficient, reliable, and safe evolution to pursue higher traffic efficiency of the ego vehicle in both multi-lane and congested ramp scenarios.

Vehicle control is one of the most critical challenges in autonomous vehicles (AVs) and connected and automated vehicles (CAVs), and it is paramount in vehicle safety, passenger comfort, transportation efficiency, and energy saving. This survey attempts to provide a comprehensive and thorough overview of the current state of vehicle control technology, focusing on the evolution from vehicle state estimation and trajectory tracking control in AVs at the microscopic level to collaborative control in CAVs at the macroscopic level. First, this review starts with vehicle key state estimation, specifically vehicle sideslip angle, which is the most pivotal state for vehicle trajectory control, to discuss representative approaches. Then, we present symbolic vehicle trajectory tracking control approaches for AVs. On top of that, we further review the collaborative control frameworks for CAVs and corresponding applications. Finally, this survey concludes with a discussion of future research directions and the challenges. This survey aims to provide a contextualized and in-depth look at the state of the art in vehicle control for AVs and CAVs, identifying critical areas of focus and pointing out the potential areas for further exploration.

In the research of autonomous vehicles, most existing studies treat the decision/planning and control as two separate problems. This idea originates from robotics. But since there are essential differences between robot and autonomous vehicle, the structure in Robotics may not be suitable for autonomous vehicles. Considering decision/planning and control separately may affect the performance of autonomous vehicle under complex driving conditions. To fill in the research gap, this paper proposes a novel scheme which considers the local motion planning and control in a combined manner. Firstly, the local motion planning is transformed into the longitudinal control problem based on the proposed scenario adaptive MPC, by which the motion behavior (driving along the global path, car-following, lane-change) can be automatically decided. Then, the lateral MPC controller is designed to track the global path and conduct the local motion commands. To ensure the performance of the path tracking control and a smooth lane-change process simultaneously, an adaptive weight mechanism is introduced in the lateral controller. Comprehensive case studies including both straight and curve road are conducted based on Carsim-Simulink co-simulation platform. The results show that the proposed algorithm can not only ensure the vehicle safety in complex driving conditions, but also ensure that the vehicle can drive at its desired velocity as much as possible by intelligently judging the most proper motion behaviors.

The concept of intelligent tires has drawn the attention of researchers in the areas of autonomous driving, advanced vehicle control, and artificial intelligence. The focus of this paper is on intelligent tires and the application of machine learning techniques to tire force estimation. We present an intelligent tire system with a tri-axial acceleration sensor, which is installed onto the inner liner of the tire. Neural Network techniques are used for real-time processing of the sensor data. The accelerometer is capable of measuring the acceleration in x,y, and z directions. When the accelerometer enters the tire contact patch, it starts generating signals until it fully leaves it. Simultaneously, by using MTS Flat-Trac test platform, tire actual forces are measured. Signals generated by the accelerometer and MTS Flat-Trac testing system are used for training three different machine learning techniques with the purpose of online prediction of tire forces. It is shown that the developed intelligent tire in conjunction with machine learning is effective in accurate prediction of tire forces under different driving conditions. The results presented in this work will open a new avenue of research in the area of intelligent tires, vehicle systems, and tire force estimation.

Existing hierarchical planning and control architectures can cause the upper-level target trajectory passed to the lower-level tracking controller to be too conservative or impossible to track on slippery roads. To solve this problem, this paper proposes a new simultaneous planning and control scheme that determines the control inputs without explicit path planning and requires only information about the control objectives and safety constraints. First, we establish the vehicle stability boundary and the safety distance constraints to ensure that the vehicle avoids drifting and collisions on slippery roads. Moreover, real-time adaptive model predictive control (MPC) with online model linearization is designed to approximate the nonlinear programming as a quadratic program (QP), which allows the use of fast convex optimization tools. The steering angle and longitudinal acceleration are thus obtained. Finally, we design the controller to convert the longitudinal acceleration into an actuatable drive torque to avoid tire skidding on slippery surfaces. The simulation results show that under the conditions of low adhesion road and$\mu $-split roads, the proposed algorithm makes the sideslip angle of the vehicle within 1 degree. In contrast, the sideslip angle of the hierarchical algorithm reaches 6 degrees, and the vehicle has a noticeable drift. The proposed algorithm dramatically improves stability and driving comfort.

This paper investigates the lane keeping control of autonomous ground vehicles (AGVs) considering the rollover prevention and input saturation. An enhanced state observer-based sliding mode control (SMC) strategy is proposed to achieve the control purpose and maintain the lane keeping errors as well as the roll angle within the prescribed performance boundaries. Three contributions are made in this paper. First, a prescribed performance function (PPF) is proposed in the controller design, aiming to implement the error transformation so as to constrain the controlled variables within the prescribed performance boundaries. Second, a modified sliding surface is developed incorporating two nonlinear functions, whose specialities and benefits are taken advantage of: one is a barrier function to restrict the load transfer ratio (LTR) in a safe boundary to guarantee the roll stability; another is a monotonely decreasing function to adaptively change the damping ratio of the closed-loop system to improve the transient performance, including reducing the transient overshoots and steady-state errors. Third, a modified multivariable adaptive SMC controller is proposed to achieve the integrated lane-keeping and roll control in the presence of the input saturation and bound-unknown disturbances. The stability of the closed-loop system is rigorously proved via the Lyapunov function. Finally, the effectiveness of the proposed control strategy is verified with a high-fidelity and full-car model via the CarSim platform.

A motion planning method for autonomous vehicles confronting emergency situations where collision is inevitable, generating a path to mitigate the crash as much as possible, is proposed in this paper. The Model predictive control (MPC) algorithm is adopted here for motion planning. If avoidance is impossible for the model predictive motion planning system, the potential crash severity, and artificial potential field are filled into the controller objective to achieve general obstacle avoidance and the lowest crash severity. Furthermore, the vehicle dynamic is also considered as an optimal control problem. Based on the analysis mentioned earlier, the model predictive controller can optimize the command following, obstacle avoidance, vehicle dynamics, road regulation, and mitigate the inevitable crash based on the predicted values. The proposed MPC algorithm has been proved by simulation to have the ability to avoid obstacles and mitigate the crash if collision is inevitable.

A path planning approach to generate a path which mitigates the effects of an inevitable crash for autonomous vehicles is presented in this brief. The model predictive control algorithm is adopted here for path planning. The artificial potential field, which describes the obstacles and the potential crash severity, are added to the control objectives to avoid the obstacle, and also to mitigate the inevitable crash. The vehicle dynamic is also considered as an optimal control objective. Based on the analysis above, the model predictive controller can guarantee the command following, obstacle avoidance, vehicle dynamics, and mitigate the inevitable crash. Simulation results verified that the proposed MPC has the abilities of obstacles avoidance and mitigation of the inevitable crash.

This paper presents a shared control driver assistance system based on the driving intention identification and situation assessment to avoid obstacles. A constrained linear-time-varying model predictive controller is designed to follow the obstacle-avoidance path, which is obtained by the artificial potential method in real time. A human driver's driving intention and the desired maneuver are recognized by the inductive multilabel classification with an unlabeled data approach that is trained based on the lateral offset and lateral velocity to the road center line. In addition, the situation assessment of the collision risk is represented by the time to collision and the performance evaluation is designed according to lateral deviation. All of them are employed for the design of the shared control fuzzy controller. The cooperative coefficient, denoting the control authority between the controller and a human driver, is determined by three fuzzy controllers in different conditions, which are the consistent, the advanced inconsistent, and the lagged inconsistent fuzzy controller, respectively. More importantly, there are two scenarios studies provided to verify the proposed system. The results prove that the shared control driver assistance system can successfully help drivers to avoid obstacles and obtains great vehicle stability performance in different scenarios.

Differential drive assistance steering (DDAS) is an emerging assisted steering mechanism in in-wheel-motor driven (IWMD) electric vehicles, yielded by the differential moment of the front tires in the steering system. DDAS can steer the front wheels when there is no steering power from the steering motor, and thus can be used as a redundant steering mechanism. To realize the yaw control when the active front steering entirely breaks down and guarantee the transient control performance therein, this paper proposes an integral sliding mode control (ISMC) approach for IWMD electric vehicles steered by DDAS. Two contributions are made in this paper: 1) An improved disturbance observer based ISMC strategy is designed to cope with the unknown mismatched disturbances, and the composite nonlinear feedback technique is employed to design the nominal part of the controller to restrain overshoots and remove steady-state errors considering the tire force saturations; 2) An adaptive super-twisting control approach is proposed to deal with the disturbances with unknown boundaries using a continuous controller while eliminating the chattering effect. The system stability and robustness are proved via Lyapunov approach. CarSim-Simulink simulation has verified the effectiveness of the proposed control approach in the case of the steering fault.

Atmospheric aerosol has important influences on the global climate either directly by scattering and absorption of the solar radiation or indirectly by affecting cloud droplet concentration or cloud radiative properties[1-2]. A high proportion of aerosol in the Earth's atmosphere consists of non-spherical mineral dust particles[3]. Light scattering by non-spherical particle such as mineral dust is commonly known as a major difficulty in aerosol characterization[1,4]. Compared with the total radiance, polarization is more sensitive to aerosol particle shape. It has a distinct advantage in study non-spherical aerosol particles. In the sky under some atmospheric conditions (e.g., clear sky, cloudy sky, hazy sky), it usually exists a characteristic polarization pattern, which is related to the position of the sun, the distribution of various atmospheric constituents, and the properties of the underlying surface[5]. The polarization pattern can be applied not only in navigation, but also in studying of atmospheric aerosol properties.

As the study for the modernization of Traditional Chinese Medicine (TCM) is moving continuously forward, a growing bond exists between TCM and modern information processing technology. The determination of syndrome and the study for the compatibility of medicines in TCM are main parts of it. In this paper, we clustered syndromes of lung cancer patients according to the clinical based cases by adopting Clustering Algorithm Based On Sparse Feature Vector algorithm (CABOSFV) algorithm and concluded three TCM classifications for lung cancer. Moreover, by the further study of the compatibility of medicines, numerous matches for critical medicines were proposed, and the results are correspond to clinical data.