Learning-based 3D reconstruction has emerged as a transformative technique in autonomous driving, enabling precise modeling of environments through advanced neural representations. It has inspired pioneering solutions for vital tasks in autonomous driving, such as dense mapping and closed-loop simulation, as well as comprehensive scene feature for driving scene understanding and reasoning. Given the rapid growth in related research, this survey provides a comprehensive review of both technical evolutions and practical applications in autonomous driving. We begin with an introduction to the preliminaries of learning-based 3D reconstruction to provide a solid technical background foundation, then progress to a rigorous, multi-dimensional examination of cutting-edge methodologies, systematically organized according to the distinctive technical requirements and fundamental challenges of autonomous driving. Through analyzing and summarizing development trends and cutting-edge research, we identify existing technical challenges, along with insufficient disclosure of on-board validation and safety verification details in the current literature, and ultimately suggest potential directions to guide future studies.

Game-theoretic decision making has the potential to bring human-like reasoning skills to autonomous vehicles (AVs), fostering trust between humans and AVs. However, to make these approaches sufficiently practical for real-world use, challenges such as game complexity and incomplete information have to be addressed. Game complexity refers to the difficulties in solving a game-theoretic problem, which include solution existence, algorithm convergence, and scalability. We show in our recent work that a possible solution to overcoming these difficulties is to use potential games. However, constructing a potential game often requires specific cost function designs, limiting their broad use. To address this challenge, we propose to employ a game projection technique in this paper, relaxing the cost function design conditions and making the potential game approach applicable to broader scenarios, even including the ones that cannot be modelled as a potential game. Incomplete information refers to the ego vehicle’s lack of knowledge of other traffic agents’ cost functions. In a driving scenario, deviations of the ego vehicle assumed/estimated others’ cost functions from their actual ones are often inevitable. This necessitate the robustness analysis of a game-theoretic solution. This paper defines the robustness margin of a game solution as the maximum magnitude of cost function deviations that can be accommodated without changing the optimality of the game solution. With this definition, closed-form robustness margins are derived. Numerical studies using highway lane-changing scenarios are reported.

A Stackelberg differential game theoretic model predictive controller is proposed for an autonomous highway driving problem. The hierarchical controller’s high-level component is the two-player Stackelberg differential lane change game, where each player uses a model predictive controller (MPC) to control his/her own motion. The differential game is converted into a bi-level optimization problem and is solved with the branch and bound algorithm. Additionally, an inverse MPC algorithm is developed to estimate the weights of the MPC cost function of the target vehicle. The low-level hybrid MPC controls both the autonomous vehicle’s longitudinal motion and its real-time lane determination. Simulations indicate both the inverse MPC’s capability on aggressiveness estimation of target vehicles and DGTMPC’s superior performance in interactive lane change situations.

Performing real-time receding horizon motion planning for autonomous vehicles while providing safety guarantees remains difficult. This is because existing methods to accurately predict ego vehicle behavior under a chosen controller use online numerical integration that requires a fine time discretization and thereby adversely affects real-time performance. To address this limitation, several recent papers have proposed to apply offline reachability analysis to conservatively predict the behavior of the ego vehicle. Reachable sets can be constructed by utilizing a simplified model whose behavior is assumeda priorito conservatively bound the dynamics of a full-order model. However, it can be challenging to meticulously construct this conservative bound. This paper proposes a framework named REFINE to overcome the limitations of these existing approaches. REFINE utilizes a parameterized robust controller that partially linearizes the vehicle dynamics even in the presence of modeling error. Zonotope-based reachability analysis is then performed on the closed-loop, full-order vehicle dynamics to offline compute the corresponding control-parameterized, over-approximate Forward Reachable Sets (FRS). Because reachability analysis is applied to the full-order model, the potential conservativeness introduced by using a simplified model is avoided. The pre-computed, control-parameterized FRS is then used online in an optimization framework to ensure safety. The proposed method is compared to several state-of-the-art methods during a simulation-based evaluation on a full-size vehicle model and is demonstrated on a$\frac{1}{10}$th race car robot in real hardware testing. In contrast to existing methods, REFINE is shown to enable the vehicle to safely navigate itself through complex environments.

Decision-making for autonomous driving is challenging, considering the complex interactions among multiple traffic agents (including autonomous vehicles (AVs), human-driven vehicles, and pedestrians) and the computational load needed to evaluate these interactions. This paper develops two general potential game-based frameworks, namely, finite and continuous potential games, for decision-making in autonomous driving. The two frameworks account for the AVs’ two types of action spaces, i.e., finite and continuous action spaces, respectively. The developed frameworks provide theoretical guarantees for the existence of pure-strategy Nash equilibria and for the convergence of the Nash equilibrium (NE) seeking algorithms. The scalability challenge is also addressed. In addition, we provide cost function shaping approaches such that the agents’ cost functions not only reflect common driving objectives but also yield potential games. The performance of the developed algorithms is demonstrated in diverse traffic scenarios, including intersection-crossing and lane-changing scenarios. Statistical comparative studies, including 1) finite potential game vs. continuous potential game, 2) best response dynamics vs. potential function optimization, and 3) potential game vs. reinforcement learning (RL) vs. control barrier function (CBF), are conducted to compare the robustness against various surrounding vehicles’ strategies and to compare the computational efficiency. It is shown that the developed potential game frameworks have better robustness than RL and than CBF if the surrounding vehicles are not safety-conscious, and are computationally feasible for real-time implementation.

Merging is, in general, a challenging task for both human drivers and autonomous vehicles, especially in dense traffic, because the merging vehicle typically needs to interact with other vehicles to identify or create a gap and safely merge into. In this paper, we consider the problem of autonomous vehicle control for forced merge scenarios. We propose a novel game-theoretic controller, called the Leader-Follower Game Controller (LFGC), in which the interactions between the autonomous ego vehicle and other vehicles with a priori uncertain driving intentions is modeled as a partially observable leader-follower game. The LFGC estimates the other vehicles’ intentions online based on observed trajectories, and then predicts their future trajectories and plans the ego vehicle’s own trajectory using Model Predictive Control (MPC) to simultaneously achieve probabilistically guaranteed safety and merging objectives. To verify the performance of LFGC, we test it in simulations and with the NGSIM data, where the LFGC demonstrates a high success rate of 97.5% in merging.

Meta Reinforcement Learning (Meta-RL) has seen substantial advancements recently. In particular, off-policy methods were developed to improve the data efficiency of Meta-RL techniques. Probabilistic embeddings for actor-critic$\boldsymbol{RL}$(PEARL) is a leading approach for multi-MDP adaptation problems. A major drawback of many existing Meta-RL methods, including PEARL, is that they do not explicitly consider the safety of the prior policy when it is exposed to a new task for the first time. Safety is essential for many real world applications, including field robots and Autonomous Vehicles (AVs), In this paper, we develop the PEARL PLUS (PEARL+) algorithm, which optimizes the policy for both prior (pre-adaptation) safety and posterior (after-adaptation) performance. Building on top of PEARL, our proposed PEARL+algorithm introduces a prior regularization term in the reward function and a new Q-network for recovering the state-action value under prior context assumptions, to improve the robustness to task distribution shift and safety of the trained network exposed to a new task for the first time. The performance of PEARL+is validated by solving three safety-critical problems related to robots and AVs, including two MuJoCo benchmark problems. From the simulation experiments, we show that safety of the prior policy is significantly improved and more robust to task distribution shift compared to PEARL.

In this paper, we present a safe deep reinforcement learning system for automated driving. The proposed framework leverages merits of both rule-based and learning-based approaches for safety assurance. Our safety system consists of two modules namely handcrafted safety and dynamically-learned safety. The handcrafted safety module is a heuristic safety rule based on common driving practice that ensure a minimum relative gap to a traffic vehicle. On the other hand, the dynamically-learned safety module is a data-driven safety rule that learns safety patterns from driving data. Specifically, the dynamically-leaned safety module incorporates a model lookahead beyond the immediate reward of reinforcement learning to predict safety longer into the future. If one of the future states leads to a near-miss or collision, then a negative reward will be assigned to the reward function to avoid collision and accelerate the learning process. We demonstrate the capability of the proposed framework in a simulation environment with varying traffic density. Our results show the superior capabilities of the policy enhanced with dynamically-learned safety module.

Travel time in urban centers is a significant contributor to the quality of living of its citizens. Mobility on Demand (MoD) services such as Uber and Lyft have revolutionized the transportation infrastructure, enabling new solutions for passengers. Shared MoD services have shown that a continuum of solutions can be provided between the traditional private transport for an individual and the public mass transit-based transport, by making use of the underlying cyber-physical substrate that provides advanced, distributed, and networked computational and communicational support. In this article, we propose a novel shared mobility service using a dynamic framework. This framework generates a dynamic route for multi-passenger transport, optimized to reduce time costs for both the shuttle and the passengers and is designed using a new concept of a space window. This concept introduces a degree of freedom that helps reduce the cost of the system involved in designing the optimal route. A specific algorithm based on the Alternating Minimization approach is proposed. Its analytical properties are characterized. Detailed computational experiments are carried out to demonstrate the advantages of the proposed approach and are shown to result in an order of magnitude improvement in the computational efficiency with minimal optimality gap when compared to a standard Mixed Integer Quadratically Constrained Programming-based algorithm.

Deep reinforcement learning (DRL) techniques have been used to solve a discretionary lane change decision-making problem and are showing promising results. However, since the input information for the discretionary lane change problem is continuous and can be in high dimension, it is an open challenge for DRL to optimize the exploration-exploitation trade-off. Conventional model-less exploration methods lack a systematic way to incorporate additional engineering or model-based knowledge of our application into consideration and as a result, the training can be inefficient and may dwell on a policy, e.g. lane change strategy that is impractical. In previous related work, many used the rule-based safety check policy to guide the exploration and collect input information data. However, it is not guaranteed to get the optimal policy and the performance is dependent on the safety check policy selected. In this paper, we developed an explicit statistical aggregated environment model using a conditional variational auto-encoder and a model-based exploration strategy leveraging it. The agent is guided to explore with surprise-based intrinsic reward derived from the environment model. The result is compared with annealing epsilon-greedy exploration and with rule-based safety check exploration. We demonstrate that the performance of the developed model-based exploration method is comparable with the best rule-based safety check exploration and much better than the epsilon-greedy exploration.

The operational space of an autonomous vehicle (AV) can be diverse and vary significantly. Due to this, formulating a rule based decision maker for selecting driving maneuvers may not be ideal. Similarly, it may not be efficient to solve optimal control problem in real-time for a predefined cost function. In order to address these issues and to avoid peculiar behaviors when encountering unforeseen scenario, we propose a reinforcement learning (RL) based method, where the ego car, i.e., an autonomous vehicle, learns to make decisions by directly interacting with the simulated traffic. Here the decision maker is a deep neural network that provides an action choice for a given system state. We demonstrate the performance of the developed algorithm in highway driving scenario where the trained AV encounters varying traffic density.

In automated driving, the road geometry information such as waypoints is usually available from previously stored maps. In this paper, we present a scenario based Model Predictive Control (MPC) trajectory planning algorithm that consists of spatial planning with embedded temporal optimization that leverages waypoints information of the road. Our trajectory planning algorithm is structured such that the spatial and temporal planning is integrated so that both the longitudinal and lateral aspects, reflected in the shape and length of the planned trajectory, are dynamically changing to best negotiate the constraints from road curvature and surrounding vehicles. The concept of a vehicle connected to shadow trolleys traveling along the rails on the road is introduced. Given waypoints, a reference cubic spline can be constructed to define the rail for trolleys and form a curvilinear coordinate. This concept facilitates the description of vehicle motion, trajectories, and surroundings with respect to the trolley, which is especially convenient for a vehicle traveling on high curvature road and switchbacks. A temporal optimization of the trolley instances is first conducted, which in turn allows us to make a proper approximation and reduce an originally complex nonlinear spatial-temporal optimization problem into one that requires only Quadratic Programming (QP). We present simulation results of various challenging scenarios on complex road geometry with multiple surrounding vehicles of varying behavior to demonstrate the effectiveness and efficiency of the proposed algorithm. Simulation results show reasonable, flexible, and safe maneuvers.

One of the important goals of a smart city is to increase the quality of urban mobility. This paper will explore the use of dynamic tariffs for this purpose. Transactive control, the concept of feedback through economic transactions, is a promising concept for accomplishing such dynamic tariffs, and is the focus of this paper. Two specific examples of transactive control are considered in this paper, the first of which is the synthesis of dynamic toll prices with the goal of reducing traffic congestion in highways. We examine how a model-based approach can result in optimal toll pricing schemes. Sociotechnical models that combine behavioral models of drivers and traffic flow models, together with real-time traffic information obtained from on-road sensors are used to determine the transactive control strategies. The overall pricing strategy is evaluated using real traffic data from an existing dynamic toll-pricing framework. The second example of transactive control is in the context of Mobility on Demand (MoD), where new modes of transportation other than private and public are being proposed, providing a smorgasbord of options for passengers. We investigate a dynamic routing concept for multipassenger transport, and propose a transactive control strategy to regulate the achievable performance around desired values. Numerical simulations using actual passenger data are carried out to demonstrate the advantages of the proposed concept.

Path planning for autonomous vehicles in dynamic environments is an important but challenging problem, due to the constraints of vehicle dynamics and existence of surrounding vehicles. Typical trajectories of vehicles involve different modes of maneuvers, including lane keeping, lane change, ramp merging, and intersection crossing. There exist prior arts using the rule-based high-level decision making approaches to decide the mode switching. Instead of using explicit rules, we propose a unified path planning approach using Model Predictive Control (MPC), which automatically decides the mode of maneuvers. To ensure safety, we model surrounding vehicles as polygons and develop a type of constraints in MPC to enforce the collision avoidance between the ego vehicle and surrounding vehicles. To achieve comfortable and natural maneuvers, we include a lane-associated potential field in the objective function of the MPC. We have simulated the proposed method in different test scenarios and the results demonstrate the effectiveness of the proposed approach in automatically generating reasonable maneuvers while guaranteeing the safety of the autonomous vehicle.