We present Exact Gauss-Newton (EGN), a stochastic second-order optimization algorithm that combines the generalized Gauss-Newton (GN) Hessian approximation with low-rank linear algebra to compute the descent direction. Leveraging the Duncan-Guttman matrix identity, the parameter update is obtained by factorizing a matrix which has the size of the mini-batch. This is particularly advantageous for large-scale machine learning problems where the dimension of the neural network parameter vector is several orders of magnitude larger than the batch size. Additionally, we show how improvements such as line search, adaptive regularization, and momentum can be seamlessly added to EGN to further accelerate the algorithm. Moreover, under mild assumptions, we prove that our algorithm converges in expectation to a stationary point of the objective. Finally, our numerical experiments demonstrate that EGN consistently exceeds, or at most matches the generalization performance of well-tuned SGD, Adam, GAF, SQN, and SGN optimizers across various supervised and reinforcement learning tasks.

Autonomous driving in urban environments requires safe control policies that account for the non-determinism of moving obstacles, such as the position other vehicles will take while crossing an uncontrolled intersection. We address this problem by proposing a stochastic model predictive control (MPC) approach with robust collision avoidance constraints to guarantee safety. By adopting a stochastic formulation, the quality of closed-loop tracking is increased by avoiding giving excessive importance to future obstacle configurations that are unlikely to occur. We compute the probabilities associated with different obstacle trajectories by learning a classifier on a realistic dataset generated by the microscopic traffic simulator SUMO and show the benefits of the proposed stochastic MPC formulation on a simulated realistic intersection.

This article considers the two-stage approach to solving a partially observable Markov decision process (POMDP): the identification stage and the (optimal) control stage. We present an inexact sequential quadratic programming framework for recurrent neural network learning (iSQPRL) for solving the identification stage of the POMDP, in which the true system is approximated by a recurrent neural network (RNN) with dynamically consistent overshooting (DCRNN). We formulate the learning problem as a constrained optimization problem and study the quadratic programming (QP) subproblem with a convergence analysis under a restarted Krylov-subspace iterative scheme that implicitly exploits the structure of the associated Karush-Kuhn-Tucker (KKT) subsystem. In the control stage, where a feedforward neural network (FNN) controller is designed on top of the RNN model, we adapt a generalized Gauss-Newton (GGN) algorithm that exploits useful approximations to the curvature terms of the training data and selects its mini-batch step size using a known property of some regularization function. Simulation results are provided to demonstrate the effectiveness of our approach.

This paper proposes an active learning (AL) algorithm to solve regression problems based on inverse-distance weighting functions for selecting the feature vectors to query. The algorithm has the following features: (i) supports both pool-based and population-based sampling; (ii) is not tailored to a particular class of predictors; (iii) can handle known and unknown constraints on the queryable feature vectors; and (iv) can run either sequentially, or in batch mode, depending on how often the predictor is retrained. The potentials of the method are shown in numerical tests on illustrative synthetic problems and real-world datasets. An implementation of the algorithm, that we call IDEAL (Inverse-Distance based Exploration for Active Learning), is available at http://cse.lab.imtlucca.it/bemporad/ideal.

We consider the problem of estimating the numerical accuracy of programs with operations in fixed-point arithmetic and variables of arbitrary, mixed precision, and possibly non-deterministic value. By applying a set of parameterised rewrite rules, we transform the relevant fragments of the program under consideration into sequences of operations in integer arithmetic over vectors of bits, thereby reducing the problem as to whether the error enclosures in the initial program can ever exceed a given order of magnitude to simple reachability queries on the transformed program. We describe a possible verification flow and a prototype analyser that implements our technique. We present an experimental evaluation on a particularly complex industrial case study, including a preliminary comparison between bit-level and word-level decision procedures.

Production plants are being re-designed to implement human-centered solutions. Especially considering high added-value operations, robots are required to optimize their behavior to achieve a task quality at least comparable to the one obtained by the skilled operators. A manual programming and tuning of the manipulator is not an efficient solution, requiring to adopt towards automated strategies. Adding external sensors (e.g., cameras) increases the robotic cell complexity and it doesn’t solve the issue since it is usually difficult to build explicit reward functions measuring the robot performance, while it is easier for the user to define a qualitative comparison between two experiments. According to these needs, in this paper, the recently-developed preferences-based optimization approach GLISp is employed and adapted to tune the novel developed path-based velocity planner. The implemented solution defines an intuitive human-centered procedure, capable of transferring (through pairwise preferences between experiments) the task knowledge from the operator to the manipulator. A Franka EMIKA panda robot has been employed as a test platform to perform a robotic sealing task (i.e., material deposition task), validating the proposed methodology. The proposed approach has been compared with a programming by demonstration approach, and with the manual tuning of the path-based velocity planner. Achieved results demonstrate the improved deposition quality obtained with the proposed optimized path-based velocity planner methodology in a limited number of experimental trials (20).

Abstract This paper proposes a method for solving optimization problems in which the decision-maker cannot evaluate the objective function, but rather can only express a preference such as “this is better than that” between two candidate decision vectors. The algorithm described in this paper aims at reaching the global optimizer by iteratively proposing the decision maker a new comparison to make, based on actively learning a surrogate of the latent (unknown and perhaps unquantifiable) objective function from past sampled decision vectors and pairwise preferences. A radial-basis function surrogate is fit via linear or quadratic programming, satisfying if possible the preferences expressed by the decision maker on existing samples. The surrogate is used to propose a new sample of the decision vector for comparison with the current best candidate based on two possible criteria: minimize a combination of the surrogate and an inverse weighting distance function to balance between exploitation of the surrogate and exploration of the decision space, or maximize a function related to the probability that the new candidate will be preferred. Compared to active preference learning based on Bayesian optimization, we show that our approach is competitive in that, within the same number of comparisons, it usually approaches the global optimum more closely and is computationally lighter. Applications of the proposed algorithm to solve a set of benchmark global optimization problems, for multi-objective optimization, and for optimal tuning of a cost-sensitive neural network classifier for object recognition from images are described in the paper. MATLAB and a Python implementations of the algorithms described in the paper are available at http://cse.lab.imtlucca.it/~bemporad/glis .

Optimal control theory and machine learning techniques are combined to formulate and solve in closed form an optimal control formulation of online learning from supervised examples with regularization of the updates. The connections with the classical linear quadratic gaussian (LQG) optimal control problem, of which the proposed learning paradigm is a nontrivial variation as it involves random matrices, are investigated. The obtained optimal solutions are compared with the Kalman filter estimate of the parameter vector to be learned. It is shown that the proposed algorithm is less sensitive to outliers with respect to the Kalman estimate (thanks to the presence of the regularization term), thus providing smoother estimates with respect to time. The basic formulation of the proposed online learning framework refers to a discrete-time setting with a finite learning horizon and a linear model. Various extensions are investigated, including the infinite learning horizon and, via the so-called kernel trick, the case of nonlinear models.

The design of stabilizing controllers for hybrid systems is particularly challenging due to the heterogeneity present within the system itself. In this paper we propose a constructive procedure to design stabilizing dynamic controllers for a fairly general class of hybrid systems. The proposed technique is based on the concept of a hybrid control Lyapunov function (hybrid CLF) that was previously introduced by the authors. In this paper we generalize the concept of hybrid control Lyapunov function, and we show that the existence of a hybrid CLF guarantees the existence of a standard control Lyapunov function (CLF) for the hybrid system. We provide a constructive procedure to design a hybrid CLF and the corresponding dynamic control law, which is stabilizing because of the established connection to a standard CLF that becomes a Lyapunov function for the closed-loop system. The obtained control law can be conveniently implemented by constrained predictive control in the form of a receding horizon control strategy. A numerical example highlighting the features of the proposed approach is presented.

Piecewise affine (PWA) control laws offer an attractive solution to real-time control of linear, nonlinear and hybrid systems. In this paper we provide a compact exposition of the existing state-of-the-art methods for the synthesis of PWA control laws using optimization-based methods.

We present a local approach for planning the motion of a car-like robot navigating among obstacles, suitable for sensor-based implementation. The nonholonomic nature of the robot kinematics is explicitly taken into account. The strategy is to modify the output of a generic local holonomic planner, so as to provide commands that realize the desired motion in a least-squares sense. A feedback action tends to align the vehicle with the local force field. In order to avoid the motion stops away from the desired goal, various force fields are considered and compared by simulation.