Incorporating user preferences into multi-objective Bayesian optimization (MOBO) allows for personalization of the op- timization procedure. Preferences are often abstracted in the form of an unknown utility function, estimated through pair- wise comparisons of potential outcomes. However, utility-driven MOBO methods can yield solutions that are dominated by nearby solutions, as non-dominance is not enforced. Additionally, classical MOBO commonly relies on estimating the entire Pareto front to identify the Pareto-optimal solutions, which can be expensive and ignore user preferences. Here, we present a new method, termed preference-utility-balanced MOBO (PUB-MOBO), that allows users to disambiguate between near-Pareto candidate solutions. PUB-MOBO combines utility-based MOBO with local multi-gradient descent to refine user-preferred solutions to be near-Pareto-optimal. To this end, we propose a novel preference-dominated utility function that concurrently preserves user-preferences and dominance amongst candidate solutions. A key advantage of PUB-MOBO is that the local search is restricted to a (small) region of the Pareto front directed by user preferences, alleviating the need to estimate the entire Pareto-front. PUB-MOBO is tested on three synthetic benchmark problems: DTLZ1, DTLZ2 and DH1, as well as on three real-world problems: Vehicle Safety, Conceptual Marine Design, and Car Side Impact. PUB-MOBO consistently outperforms state-of-the-art competitors in terms of proximity to the Pareto-front and utility regret across all the problems.

We consider the problem of safe multi-agent motion planning for drones in uncertain, cluttered workspaces. For this problem, we present a tractable motion planner that builds upon the strengths of reinforcement learning and constrained-control-based trajectory planning. First, we use single-agent reinforcement learning to learn motion plans from data that reach the target but may not be collision-free. Next, we use a convex optimization, chance constraints, and set-based methods for constrained control to ensure safety, despite the uncertainty in the workspace, agent motion, and sensing. The proposed approach can handle state and control constraints on the agents, and enforce collision avoidance among themselves and with static obstacles in the workspace with high probability. The proposed approach yields a safe, real-time implementable, multi-agent motion planner that is simpler to train than methods based solely on learning. Numerical simulations and experiments show the efficacy of the approach.

Advances in modeling and computation have resulted in high-fidelity digital twins capable of simulating the dynamics of a wide range of industrial systems. These simulation models often require calibration, or the estimation of an optimal set of parameters in some goodness-of-fit sense, to reflect a system’s observed behavior. While searching over the parameter space is an inevitable part of the calibration process, simulation models are rarely designed to be valid for arbitrarily large parameter spaces. The application of existing calibration methods, therefore, often results in repeated model evaluations using parameters that can cause the simulations to be impractically slow or even result in catastrophic failure. In general, the shape of subregions in the parameter space that could result in simulation failure is unknown. In this article, we propose a novel failure-robust Bayesian optimization (FR-BO) algorithm that learns these failure regions (FRs) from online simulations and informs a Bayesian optimization algorithm to avoid FRs while optimizing model parameters. This results in acceleration of the optimizer’s convergence and prevents wastage of time trying to simulate parameters with high failure probabilities. The effectiveness of the proposed FR-BO algorithm is demonstrated via a well-known benchmark example where we compare against state-of-the-art gradient matching techniques, and a practical example related to parameter estimation for digital twins of buildings.

This article presents an extremum-seeking control (ESC) algorithm with an adaptive step size that adjusts the aggressiveness of the controller based on the quality of estimates obtained using a gradient estimator, which is intrinsic to many ESC algorithms. The adaptive step size ensures that the integral-action produced by the ESC control law does not destabilize the closed-loop system. To quantify the quality of the gradient estimate, we present a batch least-squares (BLS) estimator with a novel weighting term and guarantee that the gradient estimation error is bounded. The adaptive step size then maximizes the decrease of the combined plant and controller Lyapunov function for the worst-case estimation error. We also ensure that our ESC controller is input-to-state stable with respect to a class of dither signals. Finally, we demonstrate our ESC controller through benchmark examples and a practical application: leak detection with drones.

We study the problem of safe multi-agent motion planning in cluttered environments. Existing multi-agent reinforcement learning-based motion planners only provide approximate safety enforcement. We propose a safe reinforcement learning algorithm that leverages single-agent reinforcement learning for target regulation and a subsequent convex optimization-based filtering that ensures the collective safety of the system. Our approach yields a safe, real-time implementable multi-agent motion planner that is simpler to train and enforces safety as hard constraints. Our approach can handle state and control constraints on the agents, and enforce collision avoidance among themselves and with static obstacles in the environment. Numerical simulations and hardware experiments show the efficacy of the approach.

This article presents a data-driven method for computing reachable sets where active learning (AL) is used to reduce the computational burden. Set-based methods used to estimate reachable sets typically do not scale well with the state-space dimension, or rely heavily on the existence of a model. If such a model is not available, it is simple to generate state trajectory data by numerically simulating black-box oracles of systems (whose dynamics are unknown) from sampled initial conditions. Using these data samples, the estimation of reachable sets can be posed as a classification problem, wherein AL can intelligently select samples that are most informative and least similar to previously labeled samples. By exploiting submodularity, the actively learned samples can be selected efficiently, with bounded suboptimality. Our proposed framework is illustrated by estimating the domains of attractions of model predictive controllers (MPCs) and reinforcement learners. We also consider a scenario where there are two oracles that differ with respect to evaluation costs and labeling accuracy. We propose a framework to reduce the dependency of the expensive oracle in labeling samples using disagreement-based AL (DBAL). The potential of the DBAL algorithm is demonstrated on a solver selection problem for real-time MPC.

Advances in modeling and computation have resulted in high-fidelity digital models capable of simulating the dynamics of a wide range of industrial systems. These models often require calibration, or the estimation of an optimal set of parameters, to reflect a system’s observed behavior. While searching over the parameter space is an inevitable part of the calibration process, models are seldom designed to be valid for arbitrarily large parameter spaces. Application of existing black-box calibration methods, therefore, often require repeatedly evaluating a model over a wide range of parameters. For some parameter combinations, the simulations could be unreasonably slow or fail altogether. In general, the shape of subregions in the parameter space that could result in simulation failure is unknown and near-impossible to ascertain analytically. In this paper, we propose a novel failure robust Bayesian optimization (FR-BO) algorithm that learns these failure regions from simulation data and informs a Bayesian optimization algorithm to avoid failure regions while searching for optimal parameters. This results in acceleration of the optimizer’s convergence and prevents wastage of time trying to simulate parameters with high failure probabilities.

Efficiently computing solutions to mixed-integer optimization-based control problems, such as in model predictive control (MPC) of hybrid systems, is extremely challenging due to the exponential worst-case complexity. The practical time-complexity of computing good control actions can be reduced by using a combination of two solvers: a strong solver that generates optimal or near-optimal closed-loop solutions with a large number of iterations, and a weak solver that converges quickly to suboptimal closed-loop solutions. In this paper, we propose the use of deep neural networks to learn sub-regions of the admissible state-space where replacing the strong solver with the weak solver maintains constraint satisfaction properties and does not result in a significant deterioration of performance. We illustrate the practical time-complexity reduction of the proposed solver selection mechanism on a station-keeping problem for a satellite.

We develop a method for obtaining safe initial policies for reinforcement learning via approximate dynamic programming (ADP) techniques for uncertain systems evolving with discrete-time dynamics. We employ the kernelized Lipschitz estimation to learn multiplier matrices that are used in semidefinite programming frameworks for computing admissible initial control policies with provably high probability. Such admissible controllers enable safe initialization and constraint enforcement while providing exponential stability of the equilibrium of the closed-loop system.

Data-driven methods for learning near-optimal control policies through approximate dynamic programming (ADP) have garnered widespread attention. In this paper, we investigate how data-driven control methods can be leveraged to imbue near-optimal performance in a core component in modern factory systems: the electric motor drive. We apply policy iteration-based ADP to an induction motor model in order to construct a state feedback control policy for a given cost functional. Approximate error convergence properties of policy iteration methods imply that the learned control policy is near-optimal. We demonstrate that carefully selecting a cost functional and initial control policy yields a near-optimal control policy that outperforms both a baseline nonlinear control policy based on backstepping, as well as the initial control policy.

High-throughput sequencing techniques have generated massive quantities of genotype data. Haplotype phasing has proven to be a useful and effective method for analyzing these data. However, the quality of phasing is undermined due to missing information. Imputation provides an effective means of improving the underlying genotype information. For model organisms, imputation can rely on an available reference genotype panel and a physical or genetic map. For non-model organisms, which often do not have a genotype panel, it is important to design an imputation technique that does not rely on reference data. Here, we present Accurate Data-Driven Imputation Technique (ADDIT), which is composed of two data-driven algorithms capable of handling data generated from model and non-model organisms. The non-model variant of ADDIT (referred to as ADDIT-NM) employs statistical inference methods to impute missing genotypes, whereas the model variant (referred to as ADDIT-M) leverages a supervised learning-based approach for imputation. We demonstrate that both variants of ADDIT are more accurate, faster, and require less memory than leading state-of-the-art imputation tools using model (human) and non-model (maize, apple, and grape) genotype data. Software Availability: The source code of ADDIT and test data sets are available at https://github.com/NDBL/ADDIT.

The problem of actuator fault estimation and fault-tolerant control for a class of uncertain nonlinear systems using Takagi-Sugeno fuzzy models is investigated. A design procedure for nonfragile proportional-integral (PI) observer is proposed to estimate the states of the nonlinear system and reconstruct the abrupt (modeled as step-like faults) and incipient fault signals. Subsequently, a nonfragile fault-tolerant controller is constructed, which is informed by the PI observer. Sufficient conditions of the existence of the PI observer and the fault-tolerant controller are provided in the form of linear matrix inequalities. The proposed fault-tolerant control architecture is tested on two numerical examples.