Game-theoretic decision making has the potential to bring human-like reasoning skills to autonomous vehicles (AVs), fostering trust between humans and AVs. However, to make these approaches sufficiently practical for real-world use, challenges such as game complexity and incomplete information have to be addressed. Game complexity refers to the difficulties in solving a game-theoretic problem, which include solution existence, algorithm convergence, and scalability. We show in our recent work that a possible solution to overcoming these difficulties is to use potential games. However, constructing a potential game often requires specific cost function designs, limiting their broad use. To address this challenge, we propose to employ a game projection technique in this paper, relaxing the cost function design conditions and making the potential game approach applicable to broader scenarios, even including the ones that cannot be modelled as a potential game. Incomplete information refers to the ego vehicle’s lack of knowledge of other traffic agents’ cost functions. In a driving scenario, deviations of the ego vehicle assumed/estimated others’ cost functions from their actual ones are often inevitable. This necessitate the robustness analysis of a game-theoretic solution. This paper defines the robustness margin of a game solution as the maximum magnitude of cost function deviations that can be accommodated without changing the optimality of the game solution. With this definition, closed-form robustness margins are derived. Numerical studies using highway lane-changing scenarios are reported.

Decision-making for autonomous driving is challenging, considering the complex interactions among multiple traffic agents (including autonomous vehicles (AVs), human-driven vehicles, and pedestrians) and the computational load needed to evaluate these interactions. This paper develops two general potential game-based frameworks, namely, finite and continuous potential games, for decision-making in autonomous driving. The two frameworks account for the AVs’ two types of action spaces, i.e., finite and continuous action spaces, respectively. The developed frameworks provide theoretical guarantees for the existence of pure-strategy Nash equilibria and for the convergence of the Nash equilibrium (NE) seeking algorithms. The scalability challenge is also addressed. In addition, we provide cost function shaping approaches such that the agents’ cost functions not only reflect common driving objectives but also yield potential games. The performance of the developed algorithms is demonstrated in diverse traffic scenarios, including intersection-crossing and lane-changing scenarios. Statistical comparative studies, including 1) finite potential game vs. continuous potential game, 2) best response dynamics vs. potential function optimization, and 3) potential game vs. reinforcement learning (RL) vs. control barrier function (CBF), are conducted to compare the robustness against various surrounding vehicles’ strategies and to compare the computational efficiency. It is shown that the developed potential game frameworks have better robustness than RL and than CBF if the surrounding vehicles are not safety-conscious, and are computationally feasible for real-time implementation.

Merging is, in general, a challenging task for both human drivers and autonomous vehicles, especially in dense traffic, because the merging vehicle typically needs to interact with other vehicles to identify or create a gap and safely merge into. In this paper, we consider the problem of autonomous vehicle control for forced merge scenarios. We propose a novel game-theoretic controller, called the Leader-Follower Game Controller (LFGC), in which the interactions between the autonomous ego vehicle and other vehicles with a priori uncertain driving intentions is modeled as a partially observable leader-follower game. The LFGC estimates the other vehicles’ intentions online based on observed trajectories, and then predicts their future trajectories and plans the ego vehicle’s own trajectory using Model Predictive Control (MPC) to simultaneously achieve probabilistically guaranteed safety and merging objectives. To verify the performance of LFGC, we test it in simulations and with the NGSIM data, where the LFGC demonstrates a high success rate of 97.5% in merging.

Goal inference is of great importance for a variety of applications that involve interaction, coordination, and/or competition with goal-oriented agents. Typical goal inference approaches use as many pointwise measurements of the agent's trajectory as possible to pursue a most accurate a-posteriori estimate of the goal. However, taking frequent measurements may not be preferred in situations where sensing is associated with high cost (e.g., sensing + perception may involve high computational/bandwidth cost and sensing may raise security concerns in privacy-critical/data-sensitive applications). In such situations, a sensible tradeoff between the information gained from measurements and the cost associated with sensing actions is highly desirable. This paper introduces a cost-effective sensing strategy for goal inference tasks based on hybrid Kalman filtering and model predictive control. Our key insights include: 1) a model predictive approach can be used to predict the amount of information gained from new measurements over a horizon and thus to optimize the tradeoff between information gain and sensing action cost, and 2) the high computational efficiency of hybrid Kalman filtering can ensure real-time feasibility of such a model predictive approach. We evaluate the proposed cost-effective sensing approach in a goal-oriented task, where we show that compared to standard goal inference approaches, our approach takes a considerably reduced number of measurements while not impairing the speed, accuracy, and reliability of goal inference by taking measurements smartly.

Motivated by the need for simulation tools for testing, verification and validation of autonomous driving systems that operate in traffic consisting of both autonomous and human-driven vehicles, we propose a game-theoretic framework for modeling the interactive behavior of vehicles at uncontrolled intersections. The proposed vehicle interaction model is based on a novel formulation of dynamic games with multiple concurrent leader-follower pairs, induced from common traffic rules. Based on simulation results for various intersection scenarios, we show that the model exhibits reasonable behavior expected in traffic, including the capability of reproducing scenarios extracted from real-world traffic data and reasonable performance in resolving traffic conflicts. The model is further validated based on the level-of-service traffic quality rating system and demonstrates manageable computational complexity compared to traditional multi-player game-theoretic models.

For a foreseeable future, autonomous vehicles (AVs) will operate in traffic together with human-driven vehicles. Their planning and control systems need extensive testing, including early-stage testing in simulations where the interactions among autonomous/human-driven vehicles are represented. Motivated by the need for such simulation tools, we propose a game-theoretic approach to modeling vehicle interactions, in particular, for urban traffic environments with unsignalized intersections. We develop traffic models with heterogeneous (in terms of their driving styles) and interactive vehicles based on our proposed approach, and use them for virtual testing, evaluation, and calibration of AV control systems. For illustration, we consider two AV control approaches, analyze their characteristics and performance based on the simulation results with our developed traffic models, and optimize the parameters of one of them.

This article provides an overview of fuzzy encoded Markov chains (FEMCs), which are finite-state Markov chains applied to transitions between fuzzy sets that encode signal or variable values. FEMCs can be used for modeling of dynamic systems, predicting/forecasting future signal values, for state estimation, and for the development of fuzzy rules for control. Under suitable assumptions, the state possibility distribution can be propagated using FEMC models in a similar manner as the state probability distribution using conventional Markov chain models. The article first discusses FEMC theory, procedures to identify FEMCs from data, and the use of FEMCs for forecasting and control. Then, we introduce, for the first time, observers for partially observable FEMCs. The observer theory is developed and computational approaches are presented. Finally, we briefly review some FEMC applications in the automotive domain.

In this paper, we present a safe deep reinforcement learning system for automated driving. The proposed framework leverages merits of both rule-based and learning-based approaches for safety assurance. Our safety system consists of two modules namely handcrafted safety and dynamically-learned safety. The handcrafted safety module is a heuristic safety rule based on common driving practice that ensure a minimum relative gap to a traffic vehicle. On the other hand, the dynamically-learned safety module is a data-driven safety rule that learns safety patterns from driving data. Specifically, the dynamically-leaned safety module incorporates a model lookahead beyond the immediate reward of reinforcement learning to predict safety longer into the future. If one of the future states leads to a near-miss or collision, then a negative reward will be assigned to the reward function to avoid collision and accelerate the learning process. We demonstrate the capability of the proposed framework in a simulation environment with varying traffic density. Our results show the superior capabilities of the policy enhanced with dynamically-learned safety module.

This paper presents the stability analysis of an optimal tour for the symmetric traveling salesman problem (TSP) by obtaining stability regions. The stability region of an optimal tour is the set of all cost changes for which that solution remains optimal and can be understood as the margin of optimality for a solution with respect to perturbations in the problem data. It is known that it is not possible to test in polynomial time whether an optimal tour remains optimal after the cost of an arbitrary set of edges changes. Therefore, this paper develops tractable methods to obtain under and over approximations of stability regions based on neighborhoods and relaxations. The application of the results to the two-neighborhood and the minimum 1 tree (M1T) relaxation are discussed in detail. For Euclidean TSPs, stability regions with respect to vertex location perturbations and the notion of safe radii and location criticalities are introduced. Benefits of this paper include insight into robustness properties of tours, minimum spanning trees, M1Ts, and fast methods to evaluate optimality after perturbations occur. Numerical examples are given to demonstrate the methods and achievable approximation quality.

By performing stability analysis on an optimal tour for problems belonging to classes of the traveling salesman problem (TSP), this paper derives margins of optimality for a solution with respect to disturbances in the problem data. Specifically, we consider the asymmetric sequence-dependent TSP, where the sequence dependence is driven by the dynamics of a stack. This is a generalization of the symmetric non sequence-dependent version of the TSP. Furthermore, we also consider the symmetric sequence-dependent variant and the asymmetric non sequence-dependent variant. Amongst others these problems have applications in logistics and unmanned aircraft mission planning. Changing external conditions such as traffic or weather may alter task costs, which can render an initially optimal itinerary suboptimal. Instead of optimizing the itinerary every time task costs change, stability criteria allow for fast evaluation of whether itineraries remain optimal. This paper develops a method to compute stability regions for the best tour in a set of tours for the symmetric TSP and extends the results to the asymmetric problem as well as their sequence-dependent counterparts. As the TSP is NP-hard, heuristic methods are frequently used to solve it. The presented approach is also applicable to analyze stability regions for a tour obtained through application of the k -opt heuristic with respect to the k -neighborhood. A dimensionless criticality metric for edges is proposed, such that a high criticality of an edge indicates that the optimal tour is more susceptible to cost changes in that edge. Multiple examples demonstrate the application of the developed stability computation method as well as the edge criticality measure that facilitates an intuitive assessment of instances of the TSP.

By performing stability analysis on an optimal runway schedule, this paper derives a method to determine whether an optimized landing sequence of aircraft remains optimal after an arbitrary number of aircraft in that sequence are delayed by an arbitrary amount of time. We consider the problem of scheduling aircraft landing on a single runway with the objective of maximizing throughput under changing external conditions such as delays caused by weather. Instead of optimizing the schedule every time delays occur, stability criteria allow for fast evaluation of whether schedules remain optimal. This paper develops a method to compute stability regions for a set of schedules. Sensitivity analysis of the linear programming relaxation and a nonlinear relationship between the delay of individual aircraft and the incurred cost change for all landing sequences yield the stability information. Furthermore, the properties of a first-come-first-serve policy are studied by giving sufficient conditions and a heuristic condition for the optimality of first-come-first-serve sequences. The given results are shown to be also applicable to landing sequences obtained through local neighborhood search, sequences that obey a position shift constraint, and subsequences of landing sequences as used in a rolling horizon approach.

We propose a novel architecture for a team of machine and human classifiers (i.e., a mixed-initiative team). We adopt a model of performance that is workload-dependent for the human and workload-independent for the machine. The team is structured in a nested architecture that exploits a primary trichotomous classifier (returning true, false, or unknown) with workload-independent performance that turns over the data classified as unknown to a secondary dichotomous classifier (returning true or false) with workload-dependent performance. The novel classifier architecture outperforms other classifiers, such as a single dichotomous classifier or a simple nested two-classifier team.

This paper introduces the minimum-time arbitrarily constrained distributed scheduling problem. The optimal distributed nonsequential backtracking algorithm is used by agents to schedule tasks where the tasks are related by constraints and where the assignment of tasks to agents may change during scheduling. The problem is distributed in that no agent has full knowledge either of the communication network or of the scheduling constraints. Each agent knows the assignments of the agents with which it can communicate and the scheduled times of the tasks assigned to those agents. The minimum-time arbitrarily constrained distributed scheduling problem is for the agents to use this local knowledge to find a schedule that satisfies all constraints and minimizes the time needed to complete all tasks. The optimal distributed nonsequential backtracking algorithm is able to find minimum-time schedules that satisfy all constraints. This algorithm is designed to work concurrently with a distributed assignment algorithm that satisfies certain communication requirements detailed in this paper. This paper presents proof of the algorithm's correctness, completeness, and optimality.

Bone Age (BA) determination using radiological images of left hands and wrists is important in pediatric endocrinology to correctly assess growth and pubertal maturation. In this paper, we propose a fully automated Greulich and Pyle Atlas (GP) bone age determination system using feature extraction and machine learning classifiers. The original contributions of this paper are as follows: (i) We use commercially available morphing tools to create a modified GP atlas that has images regularly spaced at three month intervals, (ii) We propose a novel Singular Value Decomposition (SVD) based feature extractor to create a feature vector. We use the Scale Invariant Feature Transform (SIFT) to extract features from the images then apply SVD to compose the feature vectors. Then, we train a Neural Network classifier using the generated feature vectors. Our preliminary results show that, even with a small number of training data sets, we obtain promising results. Future direction is discussed.

A large focus of the Unmanned Aerial Vehicle (UAV) community has been shifted to addressing the requirements necessary for managing systems of UAVs. The ability to automate the process of tracking and responding to the health of UAVs contributes to the reliable and persistent operation of multiple UAV systems. In particular, the automation of managing UAVs and their resources removes a critical, frequent, and time consuming task from an operator's workload. We have developed a battery swapping mechanism capable of `refueling' UAVs autonomously. This paper presents an automated battery swapping system for multiple small-scale UAVs. The system includes a battery swapping mechanism and online algorithms to address resource management, vehicle health monitoring, and precision landing onto the battery swapping mechanism's landing platform.