Events cameras are ideal sensors for enabling robots to detect and track objects in highly dynamic environments due to their low latency output, high temporal resolution, and high dynamic range. In this paper, we present the Asynchronous Event Multi-Object Tracking (AEMOT) algorithm for detecting and tracking multiple objects by processing individual raw events asynchronously. AEMOT detects salient event blob features by identifying regions of consistent optical flow using a novel Field of Active Flow Directions built from the Surface of Active Events. Detected features are tracked as candidate objects using the recently proposed Asynchronous Event Blob (AEB) tracker in order to construct small intensity patches of each candidate object. A novel learnt validation stage promotes or discards candidate objects based on classification of their intensity patches, with promoted objects having their position, velocity, size, and orientation estimated at their event rate. We evaluate AEMOT on a new Bee Swarm Dataset, where it tracks dozens of small bees with precision and recall performance exceeding that of alternative event-based detection and tracking algorithms by over 37%. Source code and the labelled event Bee Swarm Dataset will be open sourced.11https://github.com/angus-apps/AEMOT

Event-based cameras are popular for tracking fast-moving objects due to their high temporal resolution, low latency, and high dynamic range. In this article, we propose a novel algorithm for tracking event blobs using raw eventsasynchronouslyin real time. We introduce the concept of anevent blobas a spatio-temporal likelihood of event occurrence where the conditional spatial likelihood is blob-like. Many real-world objects, such as car headlights or any quickly moving foreground objects, generate event blob data. The proposed algorithm uses a nearest neighbor classifier with a dynamic threshold criteria for data association coupled with an extended Kalman filter to track the event blob state. Our algorithm achieves highly accurate blob tracking, velocity estimation, and shape estimation even under challenging lighting conditions and high-speed motions ($>$11 000 pixels/s). The microsecond time resolution achieved means that the filter output can be used to derive secondary information, such as time-to-contact or range estimation, that will enable applications to real-world problems, such as collision avoidance, in autonomous driving.

This paper proposes a new method for differentiating through optimal trajectories arising from non-convex, constrained discrete-time optimal control (COC) problems using the implicit function theorem (IFT). Previous works solve a differential Karush-Kuhn-Tucker (KKT) system for the trajectory derivative, and achieve this efficiently by solving an auxiliary Linear Quadratic Regulator (LQR) problem. In contrast, we directly evaluate the matrix equations which arise from applying variable elimination on the Lagrange multiplier terms in the (differential) KKT system. By appropriately accounting for the structure of the terms within the resulting equations, we show that the trajectory derivatives scale linearly with the number of timesteps. Furthermore, our approach allows for easy parallelization, significantly improved scalability with model size, direct computation of vector-Jacobian products and improved numerical stability compared to prior works. As an additional contribution, we unify prior works, addressing claims that computing trajectory derivatives using IFT scales quadratically with the number of timesteps. We evaluate our method on a both synthetic benchmark and four challenging, learning from demonstration benchmarks including a 6-DoF maneuvering quadrotor and 6-DoF rocket powered landing.

In this article, we propose two novel experimental design techniques for designing maximally informative experiments to estimate the parameters of nonlinear dynamical vehicle models. The two techniques include a batch design and a sequential design technique that seek to maximize the expected Shannon information gain of the parameter distribution using either an online or offline approach (respectively). We apply and compare the techniques in both simulation and real-world experiments with a wheeled vehicle. In our simulation experiments, both of our proposed designs provide superior Shannon information gains relative to an unoptimized benchmark technique. In our real-world experiments, our sequential design technique achieves superior expected Shannon information gains relative to our batch design technique and the benchmark technique.

In this paper, we consider the problem of quickly detecting an unknown change in the conditional densities of a dependent stochastic process. In contrast to the existing quickest change detection approaches for dependent stochastic processes, we propose minimax robust versions of the popular Lorden, Pollak, and Bayesian criteria for when there is uncertainty about the parameter of the post-change conditional densities. Under an information-theoretic Pythagorean inequality condition on the uncertainty set of possible post-change parameters, we identify asymptotic minimax robust solutions to our Lorden, Pollak, and Bayesian problems. Finally, through simulation examples, we illustrate that asymptotically minimax robust rules can provide detection performance comparable to the popular (but more computationally expensive) generalized likelihood ratio rule.