In this work we consider the control of dynamical systems subject to state and input constraints of arbitrary shape. In particular, we are interested in cases where some of the constraints are not described by a smooth function but are instead specified by a mix of functions and logical statements, a lookup table, or an oracle that returns whether a queried point is safe. We propose to use a Support Vector Machine (SVM) classifier to approximate the constraint boundaries with a smooth function and define a smooth nonconvex optimal control problem (OCP). We derive tightening bounds on the classifier to ensure safety of the system and investigate a class of kernels that lead to the OCP being a Difference-of-Convex (DC) programming problem. Moreover, we construct a natural difference of convex function decomposition for the Gaussian Radial Basis Function. The proposed approach is validated through numerical simulations including the control of a planar robotic manipulator with obstacle avoidance constraints.

In this paper, a control barrier function (CBF) is constructed for the lane-keeping problem which is applicable to both human-driven and automated vehicles. Based on the resulting CBF, a safety filter is developed that prevents the vehicle from crossing the lane boundaries, while only modifying the nominal steering input when necessary. The effectiveness of the proposed control approach is demonstrated in a series of numerical simulations and real vehicle experiments with a human driver. The experimental results show that the safety filter can successfully keep the vehicle inside the lane boundaries by seamlessly modifying the steering input of the human driver in a minimally invasive manner.

Game-theoretic decision making has the potential to bring human-like reasoning skills to autonomous vehicles (AVs), fostering trust between humans and AVs. However, to make these approaches sufficiently practical for real-world use, challenges such as game complexity and incomplete information have to be addressed. Game complexity refers to the difficulties in solving a game-theoretic problem, which include solution existence, algorithm convergence, and scalability. We show in our recent work that a possible solution to overcoming these difficulties is to use potential games. However, constructing a potential game often requires specific cost function designs, limiting their broad use. To address this challenge, we propose to employ a game projection technique in this paper, relaxing the cost function design conditions and making the potential game approach applicable to broader scenarios, even including the ones that cannot be modelled as a potential game. Incomplete information refers to the ego vehicle’s lack of knowledge of other traffic agents’ cost functions. In a driving scenario, deviations of the ego vehicle assumed/estimated others’ cost functions from their actual ones are often inevitable. This necessitate the robustness analysis of a game-theoretic solution. This paper defines the robustness margin of a game solution as the maximum magnitude of cost function deviations that can be accommodated without changing the optimality of the game solution. With this definition, closed-form robustness margins are derived. Numerical studies using highway lane-changing scenarios are reported.

This paper presents two machine learning-based constraint management approaches based on Reference Governors (RGs). The first approach, termed NN-DTC, uses regression neural networks to approximate the distance to constraints. The second, termed NN-NL-RG, uses regression neural networks to approximate the input-output map of a nonlinear RG. Both approaches are shown to enforce constraints for a nonlinear second order system. NN-NL-RG requires a smaller dataset size as compared to NN-DTC for well-trained neural networks. For systems with multiple constraints, NN-NL-RG is also more computationally efficient than NN - DTC. Finally, promising results are reported by having both approaches implemented on a more complex spacecraft proximity maneuvering and docking application, through simulations.

We present an integrated spatio-temporal framework for multi-range traction power and speed prediction for connected vehicles (CVs). It combines data-driven and model-based strategies to enable CVs energy efficiency optimization. The proposed framework focuses on urban arterial corridors with signalized intersections, and leverages the historical and real-time data collected from CVs and infrastructure to predict location-specific traction loads (e.g. acceleration at intersections), and augment them with time-specific speed profiles (e.g., stop duration at intersections). A Bayesian network is developed to provide a long-term load prediction informed by probabilistic analysis of historical traffic data at intersections and between intersections. Moreover, a shockwave profile model is adopted for modeling the queuing process at intersections by leveraging vehicle-to-infrastructure (V2I) communications, providing a short-range prediction of the vehicle speed with an enhanced accuracy. The benefits of the proposed load prediction framework are demonstrated for energy management of connected hybrid electric vehicles (C-HEVs). By incorporating the predicted loads into a multi-horizon model predictive controller (MPC), integrated power and thermal management of light-duty C-HEVs is enabled over real-world driving cycles, demonstrating a near globally-optimal fuel consumption over the entire trip with a < 1% deviation from dynamic programming (DP) results.

Decision-making for autonomous driving is challenging, considering the complex interactions among multiple traffic agents (including autonomous vehicles (AVs), human-driven vehicles, and pedestrians) and the computational load needed to evaluate these interactions. This paper develops two general potential game-based frameworks, namely, finite and continuous potential games, for decision-making in autonomous driving. The two frameworks account for the AVs’ two types of action spaces, i.e., finite and continuous action spaces, respectively. The developed frameworks provide theoretical guarantees for the existence of pure-strategy Nash equilibria and for the convergence of the Nash equilibrium (NE) seeking algorithms. The scalability challenge is also addressed. In addition, we provide cost function shaping approaches such that the agents’ cost functions not only reflect common driving objectives but also yield potential games. The performance of the developed algorithms is demonstrated in diverse traffic scenarios, including intersection-crossing and lane-changing scenarios. Statistical comparative studies, including 1) finite potential game vs. continuous potential game, 2) best response dynamics vs. potential function optimization, and 3) potential game vs. reinforcement learning (RL) vs. control barrier function (CBF), are conducted to compare the robustness against various surrounding vehicles’ strategies and to compare the computational efficiency. It is shown that the developed potential game frameworks have better robustness than RL and than CBF if the surrounding vehicles are not safety-conscious, and are computationally feasible for real-time implementation.

Merging is, in general, a challenging task for both human drivers and autonomous vehicles, especially in dense traffic, because the merging vehicle typically needs to interact with other vehicles to identify or create a gap and safely merge into. In this paper, we consider the problem of autonomous vehicle control for forced merge scenarios. We propose a novel game-theoretic controller, called the Leader-Follower Game Controller (LFGC), in which the interactions between the autonomous ego vehicle and other vehicles with a priori uncertain driving intentions is modeled as a partially observable leader-follower game. The LFGC estimates the other vehicles’ intentions online based on observed trajectories, and then predicts their future trajectories and plans the ego vehicle’s own trajectory using Model Predictive Control (MPC) to simultaneously achieve probabilistically guaranteed safety and merging objectives. To verify the performance of LFGC, we test it in simulations and with the NGSIM data, where the LFGC demonstrates a high success rate of 97.5% in merging.

Goal inference is of great importance for a variety of applications that involve interaction, coordination, and/or competition with goal-oriented agents. Typical goal inference approaches use as many pointwise measurements of the agent's trajectory as possible to pursue a most accurate a-posteriori estimate of the goal. However, taking frequent measurements may not be preferred in situations where sensing is associated with high cost (e.g., sensing + perception may involve high computational/bandwidth cost and sensing may raise security concerns in privacy-critical/data-sensitive applications). In such situations, a sensible tradeoff between the information gained from measurements and the cost associated with sensing actions is highly desirable. This paper introduces a cost-effective sensing strategy for goal inference tasks based on hybrid Kalman filtering and model predictive control. Our key insights include: 1) a model predictive approach can be used to predict the amount of information gained from new measurements over a horizon and thus to optimize the tradeoff between information gain and sensing action cost, and 2) the high computational efficiency of hybrid Kalman filtering can ensure real-time feasibility of such a model predictive approach. We evaluate the proposed cost-effective sensing approach in a goal-oriented task, where we show that compared to standard goal inference approaches, our approach takes a considerably reduced number of measurements while not impairing the speed, accuracy, and reliability of goal inference by taking measurements smartly.

Motivated by the need for simulation tools for testing, verification and validation of autonomous driving systems that operate in traffic consisting of both autonomous and human-driven vehicles, we propose a game-theoretic framework for modeling the interactive behavior of vehicles at uncontrolled intersections. The proposed vehicle interaction model is based on a novel formulation of dynamic games with multiple concurrent leader-follower pairs, induced from common traffic rules. Based on simulation results for various intersection scenarios, we show that the model exhibits reasonable behavior expected in traffic, including the capability of reproducing scenarios extracted from real-world traffic data and reasonable performance in resolving traffic conflicts. The model is further validated based on the level-of-service traffic quality rating system and demonstrates manageable computational complexity compared to traditional multi-player game-theoretic models.

For a foreseeable future, autonomous vehicles (AVs) will operate in traffic together with human-driven vehicles. Their planning and control systems need extensive testing, including early-stage testing in simulations where the interactions among autonomous/human-driven vehicles are represented. Motivated by the need for such simulation tools, we propose a game-theoretic approach to modeling vehicle interactions, in particular, for urban traffic environments with unsignalized intersections. We develop traffic models with heterogeneous (in terms of their driving styles) and interactive vehicles based on our proposed approach, and use them for virtual testing, evaluation, and calibration of AV control systems. For illustration, we consider two AV control approaches, analyze their characteristics and performance based on the simulation results with our developed traffic models, and optimize the parameters of one of them.

This article provides an overview of fuzzy encoded Markov chains (FEMCs), which are finite-state Markov chains applied to transitions between fuzzy sets that encode signal or variable values. FEMCs can be used for modeling of dynamic systems, predicting/forecasting future signal values, for state estimation, and for the development of fuzzy rules for control. Under suitable assumptions, the state possibility distribution can be propagated using FEMC models in a similar manner as the state probability distribution using conventional Markov chain models. The article first discusses FEMC theory, procedures to identify FEMCs from data, and the use of FEMCs for forecasting and control. Then, we introduce, for the first time, observers for partially observable FEMCs. The observer theory is developed and computational approaches are presented. Finally, we briefly review some FEMC applications in the automotive domain.

In this paper, we present a safe deep reinforcement learning system for automated driving. The proposed framework leverages merits of both rule-based and learning-based approaches for safety assurance. Our safety system consists of two modules namely handcrafted safety and dynamically-learned safety. The handcrafted safety module is a heuristic safety rule based on common driving practice that ensure a minimum relative gap to a traffic vehicle. On the other hand, the dynamically-learned safety module is a data-driven safety rule that learns safety patterns from driving data. Specifically, the dynamically-leaned safety module incorporates a model lookahead beyond the immediate reward of reinforcement learning to predict safety longer into the future. If one of the future states leads to a near-miss or collision, then a negative reward will be assigned to the reward function to avoid collision and accelerate the learning process. We demonstrate the capability of the proposed framework in a simulation environment with varying traffic density. Our results show the superior capabilities of the policy enhanced with dynamically-learned safety module.

This paper describes a Markov decision process approach to a robust spacecraft mission control policy that maximizes the expected value of science reward assuming a circular orbit. The control policy that governs mission steps can be computed off-board or onboard depending upon the availability of communication bandwidth and on-board computational resources. This paper considers a sample science mission, where the spacecraft collects data from celestial objects viewable only within a certain orbit true anomaly window. Science data collection requires the spacecraft to slew its instrument(s) toward each target, and continue pointing in the direction of the target while the spacecraft traverses its orbit. Robustness and stochastic optimization of scientific reward, is achieved at the cost of computational complexity. Approximate dynamic programming (ADP) is exploited to reduce the computational time and effort to manageable levels and to treat larger problem sizes. The proposed ADP algorithm partitions the state-space based on true anomaly regions, enabling grouping of adjacent science targets. Results of a simulation case study demonstrate that our proposed ADP approach performs quite well for reasonable ranges of key problem parameters.

This paper investigates the problem of driver distraction detection using driving performance indicators from onboard kinematic measurements. First, naturalistic driving data from the integrated vehicle-based safety system program are processed, and cabin camera data are manually inspected to determine the driver's state (i.e., distracted or attentive). Second, existing driving performance metrics, such as steering entropy, steering wheel reversal rate, and lane offset variance, are reviewed against the processed naturalistic driving data. Furthermore, a nonlinear autoregressive exogenous (NARX) driving model is developed to predict vehicle speed based on the range (distance headway), range rate, and speed history. For each driver, the NARX model is then trained on the attentive driving data. We show that the prediction error is correlated with driver distraction. Finally, two features, steering entropy and mean absolute speed prediction error from the NARX model are selected, and a support vector machine is trained to detect driving distraction. Prediction performances are reported.

The objective of drift counteraction optimal control (DCOC) problem is to compute an optimal control law that maximizes the expected time of violating specified system constraints. In this paper, we reformulate the DCOC problem as a reinforcement learning (RL) one, removing the requirements of disturbance measurements and prior knowledge of the disturbance evolution. The optimal control policy for the DCOC is then trained with RL algorithms. As an example, we treat the problem of adaptive cruise control, where the objective is to maintain desired distance headway and time headway from the lead vehicle, while the acceleration and speed of the host vehicle are constrained based on safety, comfort, and fuel economy considerations. An informed approximate Q-learning algorithm is developed with efficient training, fast convergence, and good performance. The control performance is compared with a heuristic driver model in simulation and superior performance is demonstrated.

This paper investigates a comfort-based route planner that considers both travel time and ride comfort. We first present a framework of simultaneous road profile estimation and anomaly detection with commonly available vehicle sensors. A jump-diffusion process-based state estimator is developed and used along with a multi-input observer for road profile estimation. The estimation framework is evaluated in an experimental test vehicle and promising performance is demonstrated. Second, three objective comfort metrics are developed based on factors such as travel time, road roughness, road anomaly, and intersection. A comfort-based route planning problem is then formulated with these metrics and an extended Dijkstra's algorithm is exploited to solve the problem. A cloud-based implementation of our comfort-based route planning approach is proposed to facilitate information access and fast computation. Finally, a real-world case study, comfort-based route planning from Ford Research and Innovation Center, Michigan to Ford Rouge Factory Tour, Michigan, is presented to illustrate the efficacy of the proposed route planning framework.

This paper considers a new clustering algorithm for processing time-evolving road anomaly reports. Two cluster categories, main and outlier, are defined to deal with outliers as well as to capture the evolving nature of road anomalies. The Mahalanobis distance is exploited to quantify the similarity between a new report and the existing clusters. The clusters are maintained online and the Woodbury matrix inverse lemma is used for their recursive updates. The proposed clustering algorithm can localize isolated anomalies and compress information for densely distributed anomalies. A simulation is presented to demonstrate the efficacy of the proposed algorithm.

This paper presents a safety-based route planner that exploits vehicle-to-cloud-to-vehicle (V2C2V) connectivity. Time and road risk index (RRI) are considered as metrics to be balanced based on user preference. To evaluate road segment risk, a road and accident database from the highway safety information system is mined with a hybrid neural network model to predict RRI. Real-time factors such as time of day, day of the week, and weather are included as correction factors to the static RRI prediction. With real-time RRI and expected travel time, route planning is formulated as a multiobjective network flow problem and further reduced to a mixed-integer programming problem. A V2C2V implementation of our safety-based route planning approach is proposed to facilitate access to real-time information and computing resources. A real-world case study, route planning through the city of Columbus, Ohio, is presented. Several scenarios illustrate how the "best" route can be adjusted to favor time versus safety metrics.

In this paper, we propose a method by which the boundaries of retinal layers in optical coherence tomography (OCT) images can be identified from a simple initial user input. The proposed method is a neural network approach in which the neural networks are trained to identify points within each layer, from which, the boundaries between the retinal layers are estimated. This method focuses on training neural networks to identify layers themselves, instead of boundaries, because the available date is richer and more cohesive as compared to boundary identification. Results are presented, demonstrating the effectiveness of this method.

This paper proposes a novel multi-objective route planning approach within the framework of a Vehicle-to-Cloud-to-Vehicle (V2C2V) architecture. Time and road risk index (RRI) are both considered as metrics. To evaluate road segment risk, an accident database from the Highway Safety Information System (HSIS) is processed to build a comprehensive road risk assessment model. Route planning is formulated as a multi-objective network flow problem and further reduced to a Mixed Integer Programming (MIP) problem. A real-world case study, route planning through the city of Columbus, Ohio, is presented. The Vehicle-to-Cloud-to-Vehicle (V2C2V) based implementation of our safety-based route planning approach is proposed to facilitate access to real-time information and computing resources.

This paper presents a modeling framework based on finite-state space Markov chains (MCs) and fuzzy subsets to represent signals that vary in a continuous range. Our special attention to this extension of finite-state space MC modeling is motivated by numerous opportunities in applying MC models to represent physical variables in automotive and aerospace systems and, subsequently, using these models for fault detection, estimation, prediction, stochastic dynamic programming, and stochastic model predictive control. Our generalized MC modeling framework synergistically combines the notion of transition probabilities with information granulation based on fuzzy partitioning. As compared with the case of more familiar interval partitioning, the transition probabilities in our model are defined for transitions between fuzzy subsets rather than intervals/rectangular cells. Our framework is first introduced for scalar-valued signals and then extended to vector-valued signals. A real-time capable recursive algorithm for learning transition probabilities from measured signal data is derived. Formulas that characterize the possibility distribution of the next signal value and predict the next signal value are given. It is shown that the introduced modeling framework based on MC models defined over fuzzy partitioning inherits all properties and represents a natural extension of MC models defined over interval partitioning, while providing interpolation ability and improved prediction accuracy. In addition, we derive an alternative formulation of the Chapman–Kolmogorov equation that applies to models in possibilistic/fuzzy environment. Examples are given to illustrate the key notions and results based on modeling of the vehicle speed and road grade signals.

This paper addresses modeling and predicting vehicle fuel economy based on simple vehicle characteristics. The models are identified using a historical vehicle fuel economy data set. First, the use of least squares regression analysis is pursued, and a mathematical model is created that is capable of predicting highway fuel economy based on six vehicle characteristics: engine displacement volume, vehicle maximum power, vehicle maximum torque, vehicle weight, vehicle wheelbase, and vehicle cross sectional area. Then neural network models are developed and shown to achieve higher accuracy as compared to the regression models, with 70 percent of the data in the validation data set predicted within 2 mpg. Furthermore, we demonstrate that by employing a hybrid architecture, where vehicles are first clustered and then separate models are developed for vehicle clusters, the model accuracy can be improved further.

This paper deals with a new class of Markov Chain type models that can be effectively used for real time modeling and on-line learning of nonlinear systems with uncertainties. We expand the concept of the generalized Markov Chain - a probabilistic model that synergistically combines the idea of transition probabilities with the information granulation paradigm. We consider generalized Markov chains based on two different types of information granules - intervals and fuzzy subsets - and the methods for their learning from data. We also analyze the relationship between the Markov chains and the fuzzy models and derive an alternative formulation of the Chapman-Kolmogorov equation that applies to stochastic models in fuzzy environment. As this approach is motivated by and intended for in-vehicle applications, results are illustrated on examples of granular models of vehicle speed and road grade.

This paper presents a two-step ensemble approach for vehicle fault diagnostics, an ensemble selection algorithm, BFES, and an analog Bayesian ensemble decision function, A-Bayesian-Entropy. We show through experiments that a neural network ensemble designed and trained by the proposed methodology, and selected by BFES with A-Bayesian-Entropy as the ensemble decision function can generalize well to vehicle models that are different from the vehicles used to generate training data.

In the early twentieth century, the Model T Ford defined the desirable, affordable automobile, enabled by new technologies in mechanics, materials, and manufacturing. Control, computing, communications, and the underlying software are the technologies that will shape the personal mobility experience of the twenty-first century. While the Model T was self-contained, the external reach of wireless communication technologies will define the boundaries of the twenty-first century automobile, which will be only one component in a large intelligent transportation infrastructure. This paper reviews advances in control for safety, fuel economy and reduction of tailpipe emissions, and new directions in computing, communication and software, including the interaction of the automobile with consumer electronic devices and the intelligent transportation infrastructure

This paper provides an overview of recent developments on design of hybrid controllers for continuous-time control systems that can be described by linear or nonlinear differential state equations. Hybrid controllers provide a generalization of classical feedback controllers for linear and nonlinear systems. The benefit of hybrid controllers, that they can be used to achieve closed-loop performance objectives that cannot be achieved using classical linear or nonlinear controllers, is emphasized. This paper introduces hybrid controllers in the form of a switching control architecture and provides a summary of recently developed control approaches that utilize this control architecture. We provide a conceptual framework for these results, identify limitations of the results, and discuss the current status of hybrid control design approaches.