Reinforcement learning (RL) can be used to create a decision-making agent for autonomous driving. However, previous approaches provide black-box solutions, which do not offer information on how confident the agent is about its decisions. An estimate of both the aleatoric and epistemic uncertainty of the agent’s decisions is fundamental for real-world applications of autonomous driving. Therefore, this paper introduces the Ensemble Quantile Networks (EQN) method, which combines distributional RL with an ensemble approach, to obtain a complete uncertainty estimate. The distribution over returns is estimated by learning its quantile function implicitly, which gives the aleatoric uncertainty, whereas an ensemble of agents is trained on bootstrapped data to provide a Bayesian estimation of the epistemic uncertainty. A criterion for classifying which decisions that have an unacceptable uncertainty is also introduced. The results show that the EQN method can balance risk and time efficiency in different occluded intersection scenarios, by considering the estimated aleatoric uncertainty. Furthermore, it is shown that the trained agent can use the epistemic uncertainty information to identify situations that the agent has not been trained for and thereby avoid making unfounded, potentially dangerous, decisions outside of the training distribution.

Reinforcement learning (RL) can be used to create a tactical decision-making agent for autonomous driving. However, previous approaches only output decisions and do not provide information about the agent's confidence in the recommended actions. This paper investigates how a Bayesian RL technique, based on an ensemble of neural networks with additional randomized prior functions (RPF), can be used to estimate the uncertainty of decisions in autonomous driving. A method for classifying whether or not an action should be considered safe is also introduced. The performance of the ensemble RPF method is evaluated by training an agent on a highway driving scenario. It is shown that the trained agent can estimate the uncertainty of its decisions and indicate an unacceptable level when the agent faces a situation that is far from the training distribution. Furthermore, within the training distribution, the ensemble RPF agent outperforms a standard Deep Q-Network agent. In this study, the estimated uncertainty is used to choose safe actions in unknown situations. However, the uncertainty information could also be used to identify situations that should be added to the training process.

This paper introduces a method for automatically training a general-purpose driver model, applied to the case of a truck-trailer combination. A genetic algorithm is used to optimize a structure of rules and actions, and their parameters, to achieve the desired driving behavior. The training is carried out in a simulated environment, using a two-stage process. The method is then applied to a highway driving case, where it is shown that it generates a model that matches or surpasses the performance of a commonly used reference model. Furthermore, the generality of the model is demonstrated by applying it to an overtaking situation on a rural road with oncoming traffic.

Self-driving vehicles for commercial use cases like logistics or overcast mines increase their owners' economic competitiveness. Volvo maintains, evolves, and distributes a vehicle control product line for different brands like Volvo Trucks, Renault, and Mack in more than 190 markets world-wide. From the different application domains of their customers originates the need for a multi-domain reference architecture concerned with transport mission planning, execution, and tracking on top of the vehicle control product line. This industrial case study is the first of its kind reporting about the systematic process to design such a reference architecture involving all relevant external and internal stakeholders, development documents, low level artifacts, and literature. Quantitative and qualitative metrics were applied to evaluate non-functional requirements on the reference architecture level before a concrete variant was evaluated using a Volvo FMX truck in an exemplary construction site setting.

This paper considers a low speed maneuvering problem for long articulated vehicle combinations. High precision maneuvering is achieved by designing a model-based state feedback optimal control method, commanding the steering of the first unit and a moveable coupling point between the first unit and the trailer. Simulation results are presented for a tight 90 degree turn, involving both forward and backward motions.