Control of off-road vehicles is challenging due to the complex dynamic interactions with the terrain. Accurate modeling of these interactions is important to optimize driving performance, but the relevant physical phenomena, such as slip, are too complex to model from first principles. Therefore, we present an offline meta-learning algorithm to construct a rapidly-tunable model of residual dynamics and disturbances. Our model processes terrain images into features using a visual foundation model (VFM), then maps these features and the vehicle state to an estimate of the current actuation matrix using a deep neural network (DNN). We then combine this model with composite adaptive control to modify the last layer of the DNN in real time, accounting for the remaining terrain interactions not captured during offline training. We provide mathematical guarantees of stability and robustness for our controller, and demonstrate the effectiveness of our method through simulations and hardware experiments with a tracked vehicle and a car-like robot. We evaluate our method outdoors on different slopes with varying slippage and actuator degradation disturbances, and compare against an adaptive controller that does not use the VFM terrain features. We show significant improvement over the baseline in both hardware experimentation and simulation.

Control of off-road vehicles is challenging due to the complex dynamic interactions with the terrain. Accurate modeling of these interactions is important to optimize driving performance, but the relevant physical phenomena, such as slip, are too complex to model from first principles. Therefore, we present an offline meta-learning algorithm to construct a rapidly-tunable model of residual dynamics and disturbances. Our model processes terrain images into features using a visual foundation model (VFM), then maps these features and the vehicle state to an estimate of the current actuation matrix using a deep neural network (DNN). We then combine this model with composite adaptive control to modify the last layer of the DNN in real time, accounting for the remaining terrain interactions not captured during offline training. We provide mathematical guarantees of stability and robustness for our controller, and demonstrate the effectiveness of our method through simulations and hardware experiments with a tracked vehicle and a car-like robot. We evaluate our method outdoors on different slopes with varying slippage and actuator degradation disturbances, and compare against an adaptive controller that does not use the VFM terrain features. We show significant improvement over the baseline in both hardware experimentation and simulation.

We study online policy optimization in nonlinear time-varying systems where the true dynamical models are unknown to the controller. This problem is challenging because, unlike in linear systems, the controller cannot obtain globally accurate estimations of the ground-truth dynamics using local exploration. We propose a meta-framework that combines a general online policy optimization algorithm (\texttt{ALG}) with a general online estimator of the dynamical system’s model parameters (\texttt{EST}). We show that if the hypothetical joint dynamics induced by \texttt{ALG} with \emph{known} parameters satisfies several desired properties, the joint dynamics under \emph{inexact} parameters from \texttt{EST} will be robust to errors. Importantly, the final regret only depends on \texttt{EST}’s predictions on the visited trajectory, which relaxes a bottleneck on identifying the true parameters globally. To demonstrate our framework, we develop a computationally efficient variant of Gradient-based Adaptive Policy Selection, called Memoryless GAPS (M-GAPS), and use it to instantiate \texttt{ALG}. Combining \mbox{M-GAPS} with online gradient descent to instantiate \texttt{EST} yields (to our knowledge) the first local regret bound for online policy optimization in nonlinear time-varying systems with unknown dynamics.

We study how to design learning-based adaptive controllers that enable fast and accurate online adaptation in changing environments. In these settings, learning is typically done during an initial (offline) design phase, where the vehicle is exposed to different environmental conditions and disturbances (e.g., a drone exposed to different winds) to collect training data. Our work is motivated by the observation that real-world disturbances fall into two categories: 1) those that can be directly monitored or controlled during training, which we call "manageable"; and 2) those that cannot be directly measured or controlled (e.g., nominal model mismatch, air plate effects, and unpredictable wind), which we call "latent". Imprecise modeling of these effects can result in degraded control performance, particularly when latent disturbances continuously vary. This paper presents the Hierarchical Meta-learning-based Adaptive Controller (HMAC) to learn and adapt to such multi-source disturbances. Within HMAC, we develop two techniques: 1) Hierarchical Iterative Learning, which jointly trains representations to caption the various sources of disturbances, and 2) Smoothed Streaming Meta-Learning, which learns to capture the evolving structure of latent disturbances over time (in addition to standard meta-learning on the manageable disturbances). Experimental results demonstrate that HMAC exhibits more precise and rapid adaptation to multi-source disturbances than other adaptive controllers.1

We present Model Predictive Trees (MPT), a receding horizon tree search algorithm that improves its performance by reusing information efficiently. Whereas existing solvers reuse only the highest-quality trajectory from the previous iteration as a "hotstart", our method reuses the entire optimal subtree, enabling the search to be simultaneously guided away from the low-quality areas and towards the high-quality areas. We characterize the restrictions on tree reuse by analyzing the induced tracking error under time-varying dynamics, revealing a tradeoff between the search depth and the timescale of the changing dynamics. In numerical studies, our algorithm outperforms state-of-the-art sampling-based cross-entropy methods with hotstarting. We demonstrate our planner on an autonomous vehicle testbed performing a nonprehensile manipulation task: pushing a target object through an obstacle field. Code associated with this work will be made available at https://github.com/jplathrop/mpt.

We present a new method to automatically generate semantic segmentation annotations for thermal imagery captured from an aerial vehicle by utilizing satellite-derived data products alongside onboard global positioning and attitude estimates. This new capability overcomes the challenge of developing thermal semantic perception algorithms for field robots due to the lack of annotated thermal field datasets and the time and costs of manual annotation, enabling precise and rapid annotation of thermal data from field collection efforts at a massively-parallelizable scale. By incorporating a thermal-conditioned refinement step with visual foundation models, our approach can produce highly-precise semantic segmentation labels using low-resolution satellite land cover data for little-tono cost. It achieves 98.5% of the performance from using costly high-resolution options and demonstrates between 70-160% improvement over popular zero-shot semantic segmentation methods based on large vision-language models currently used for generating annotations for RGB imagery. Code will be available at: https://github.com/connorlee77/aerial-auto-segment.

Multimodal deep sensor fusion has the potential to enable autonomous vehicles to visually understand their surrounding environments in all weather conditions. However, existing deep sensor fusion methods usually employ convoluted architectures with intermingled multimodal features, requiring large coregistered multimodal datasets for training. In this work, we present an efficient and modular RGB-X fusion network that can leverage and fuse pre-trained single-modal models via scene-specific fusion modules, thereby enabling joint input-adaptive network architectures to be created using small, coregistered multimodal datasets. Our experiments demonstrate the superiority of our method compared to existing works on RGB-thermal and RGB-gated datasets, performing fusion using only a small amount of additional parameters. Our code is available at https://github.com/dsriaditya999/RGBXFusion.

This work presents a new method for unsupervised thermal image classification and semantic segmentation by transferring knowledge from the RGB domain using a multi-domain attention network. Our method does not require any thermal annotations or co-registered RGB-thermal pairs, enabling robots to perform visual tasks at night and in adverse weather conditions without incurring additional costs of data labeling and registration. Current unsupervised domain adaptation methods look to align global images or features across domains. However, when the domain shift is significantly larger for cross-modal data, not all features can be transferred. We solve this problem by using a shared backbone network that promotes generalization, and domain-specific attention that reduces negative transfer by attending to domain-invariant and easily-transferable features. Our approach outperforms the state-of-the-art RGB-to-thermal adaptation method in classification benchmarks, and is successfully applied to thermal river scene segmentation using only synthetic RGB images. Our code is made publicly available at https://github.com/ganlumomo/thermal-uda-attention.

The ongoing Distributed Aperture Radar Tomographic Sensors (DARTS) project at NASA Jet Propulsion Laboratory aims to mature and demonstrate multi-static SAR measurements for fine-scale 3D imaging of surface topography, vegetation, and surface deformation and change. The project explores the use of drones as SAR platforms and integrates software-defined radar on RF system-on-chip for compact and flexible radar instruments. This paper highlights the progress in DARTS hardware development, experiments, and data processing. The recent experiments have successfully demonstrated monostatic interferometry as well as acquisition and processing of bi-static SAR imagery. By leveraging the advantages of multi-static SAR and drone-based platforms, the project aims to build a testbed for future missions design and enhanced SAR imaging capabilities for scientific applications.

We present a new method to adapt an RGB-trained water segmentation network to target-domain aerial thermal imagery using online self-supervision by leveraging texture and motion cues as supervisory signals. This new thermal capability enables current autonomous aerial robots operating in near-shore environments to perform tasks such as visual navigation, bathymetry, and flow tracking at night. Our method overcomes the problem of scarce and difficult-to-obtain near-shore thermal data that prevents the application of conventional supervised and unsupervised methods. In this work, we curate the first aerial thermal near-shore dataset, show that our approach outperforms fully-supervised segmentation models trained on limited target-domain thermal data, and demonstrate real-time capabilities onboard an Nvidia Jetson embedded computing platform. Code and datasets used in this work will be available at: https://github.com/connorlee77/uav-thermal-water-segmentation.

The problem of safe and fair conflict resolution among inertial, distributed agents—particularly in highly interactive settings—is of paramount importance to the autonomous vehicles industry. The difficulty of solving this problem can be attributed to the fact that agents have to reason over other agents' complex behaviors. We propose the idea of using a behavioral contract to capture a set of explicitly defined assumptions about how all agents in the environment make decisions. In this article, we present a behavioral contract for a specific class of agents that can guarantee the safety and liveness (i.e., progress) of all agents operating in accordance with it. The behavioral contract has two main components—an ordered behavioral rulebook that the agent uses to select its intended action and some additional constraints that define when an agent has precedence (or not) to take its intended action. If all of the agents act according to this contract, we can guarantee safety under all traffic conditions and liveness for all agents under “sparse” traffic conditions. The formalism of the contract also enables assignment of blame. We provide proofs of correctness of the behavioral contract and validate our results in simulation.

In this article, we present new algorithms for distributed factor graph optimization (DFGO) problems that arise in the probabilistic inference of large-scale networked robotic systems for both batch and real-time problems. First, for the batch DFGO problem, we derive a type of the alternating direction method of multipliers (ADMM) algorithm called the local consensus ADMM (LC-ADMM). The LC-ADMM is fully localized; therefore, the computational effort, communication bandwidth, and memory for each agent scale like$o(1)$with respect to thenetwork size. We establish two new theoretical results for the LC-ADMM: 1) exponential convergence when the objective is strongly convex and has a Lipschitz continuous subdifferential and 2)$o(1/k)$convergence when the objective is convex and has a unique solution. We also show that the LC-ADMM allows the use of nonquadratic loss functions, such as$\ell _{1}$-norm and Huber loss. Second, we also develop the incremental DFGO (iDFGO) algorithm for real-time problems by combining the ideas from the LC-ADMM and the Bayes tree. To derive a time-scalable algorithm, we exploit the temporal sparsity of the real-time factor graph and the convergence of the augmented factors of the LC-ADMM. The iDFGO algorithm incrementally recomputes estimates when new factors are added to the graph and is scalable with respect to both network size and time. We validate the LC-ADMM and iDFGO in simulations with examples from multiagent simultaneous localization and mapping and power grids.

In this article, we present generalized polynomial chaos-based sequential convex programming (gPC-SCP) to compute a suboptimal solution for a continuous-time chance-constrained stochastic nonlinear optimal control (SNOC) problem. The approach enables motion planning for robotic systems under uncertainty. The gPC-SCP method involves two steps. The first step is to derive a surrogate problem of deterministic nonlinear optimal control (DNOC) with convex constraints by using gPC expansion and the distributionally robust convex subset of the chance constraints. The second step is to solve the DNOC problem using sequential convex programming for trajectory generation and control. We prove that in the unconstrained case, the optimal value of the DNOC converges to that of SNOC asymptotically and that any feasible solution of the constrained DNOC is a feasible solution of the chance-constrained SNOC. We also present the predictor–corrector extension (gPC-SCP$^\text{PC}$) for real-time motion trajectory generation in the presence of stochastic uncertainty. In the gPC-SCP$^\text{PC}$method, we first predict the uncertainty using the gPC method and then optimize the motion plan to accommodate the uncertainty. We empirically demonstrate the efficacy of the gPC-SCP and the gPC-SCP$^\text{PC}$methods for the following two test cases: first, collision checking under uncertainty in actuation and physical parameters and second, collision checking with stochastic obstacle model for 3DOF and 6DOF robotic systems. We validate the effectiveness of the gPC-SCP method on the 3DOF robotic spacecraft testbed.

Recent innovations in small satellite, ultra-wideband direct RF sampling, and synchronization technologies have made multistatic and MIMO coherent SAR constellations a feasible concept for future missions. The Distributed Aperture Radar Tomographic Sensors (DARTS) mission concept at NASA JPL aims to measure Earth's surface topography and vege-tation using TomoSAR techniques. This paper describes the development of an embedded ultra-wideband next generation software defined radar (SDRadar) testbed capable of multi-band operation implemented with the Xilinx RF System on Chip (RFSoC) architecture, which features 8x 6.4 GSPS DACs and 8x 4 GSPS ADCs. The RFSoC SDRadar repre-sents a state of the art testbed for rapid prototyping of radio, radar, and synchronization technologies. We provide preliminary testing results for airborne monostatic radar imaging from a small uninhabited aerial system (sUAS), successfully demonstrating multi-band operation using first and second Nyquist zone direct RF sampling.

We present H-TD2: Hybrid Temporal Difference Learning for Taxi Dispatch, a model-free, adaptive decision-making algorithm to coordinate a large fleet of automated taxis in a dynamic urban environment to minimize expected customer waiting times. Our scalable algorithm exploits the natural transportation network company topology by switching between two behaviors: distributed temporal-difference learning computed locally at each taxi and infrequent centralized Bellman updates computed at the dispatch center. We derive a regret bound and design the trigger condition between the two behaviors to explicitly control the trade-off between computational complexity and the individual taxi policy’s bounded sub-optimality; this advances the state of the art by enabling distributed operation with bounded-suboptimality. Additionally, unlike recent reinforcement learning dispatch methods, this policy estimation is adaptive and robust to out-of-training domain events. This result is enabled by a two-step modelling approach: the policy is learned on an agent-agnostic, cell-based Markov Decision Process and individual taxis are coordinated using the learned policy in a distributed game-theoretic task assignment. We validate our algorithm against a receding horizon control baseline in a Gridworld environment with a simulated customer dataset, where the proposed solution decreases average customer waiting time by 50% over a wide range of parameters. We also validate in a Chicago city environment with real customer requests from the Chicago taxi public dataset where the proposed solution decreases average customer waiting time by 26% over irregular customer distributions during a 2016 Major League Baseball World Series game.

We presentNeural-Swarm2, a learning-based method for motion planning and control that allows heterogeneous multirotors in a swarm to safely fly in close proximity. Such operation for drones is challenging due to complex aerodynamic interaction forces, such as downwash generated by nearby drones and ground effect. Conventional planning and control methods neglect capturing these interaction forces, resulting in sparse swarm configuration during flight. Our approach combines a physics-based nominal dynamics model with learned deep neural networks with strong Lipschitz properties. We make use of two techniques to accurately predict the aerodynamic interactions between heterogeneous multirotors: 1) Spectral normalization for stability and generalization guarantees of unseen data and 2) heterogeneous deep sets for supporting any number of heterogeneous neighbors in a permutation-invariant manner without reducing expressiveness. The learned residual dynamics benefit both the proposed interaction-aware multirobot motion planning and the nonlinear tracking control design because the learned interaction forces reduce the modelling errors. Experimental results demonstrate thatNeural-Swarm2is able to generalize to larger swarms beyond training cases and significantly outperforms a baseline nonlinear tracking controller with up to three times reduction in worst-case tracking errors.

The advent of smaller SAR satellites and cheaper access to space is bringing the notion of a multistatic SAR constellation into the realm of feasibility. Researchers at JPL are studying a Distributed Aperture Radar Tomographic Sensors (DARTS) mission concept intended to measure Earth's surface topography and vegetation using TomoSAR techniques. This paper describes progress on the airborne testbed for the DARTS study. The testbed is the union of a software-defined radio that implements a radar and synchronization link together with a small uninhabited aerial system (sUAS) that serves as a platform with precise control of the observation geometry. Initial experiments have demonstrated successful multi-sensor synchronization as well as acquisition and processing of monostatic SAR imagery.

Distributed Aperture Radar Tomographic Sensors (DARTS) is a mission concept being studied at the NASA Jet Propulsion Laboratory in collaboration with the California Institute of Technology to enable global and repeated imaging of surface topography and three-dimensional vegetation structure using single-pass tomographic SAR technique. The observing system consists of a distributed formation of multiple small synthetic aperture radar platforms deployed in space with variable distances to achieve look angle diversity and sensitivity to the vertical distribution of vegetation components. Our goal is to identify the optimal system configuration starting from documented community needs and mature the critical technologies that lead to a viable implementation of DARTS. Here, we provide an overview of DARTS and describe our approach for designing and demonstrating single-pass SAR tomographic systems as part of an on-going funded NASA Instrument Incubator Program effort.

We present an online multi-task learning approach for adaptive nonlinear control, which we call Online Meta-Adaptive Control (OMAC). The goal is to control a nonlinear system subject to adversarial disturbance and unknown \emph{environment-dependent} nonlinear dynamics, under the assumption that the environment-dependent dynamics can be well captured with some shared representation. Our approach is motivated by robot control, where a robotic system encounters a sequence of new environmental conditions that it must quickly adapt to. A key emphasis is to integrate online representation learning with established methods from control theory, in order to arrive at a unified framework that yields both control-theoretic and learning-theoretic guarantees. We provide instantiations of our approach under varying conditions, leading to the first non-asymptotic end-to-end convergence guarantee for multi-task nonlinear control. OMAC can also be integrated with deep representation learning. Experiments show that OMAC significantly outperforms conventional adaptive control approaches which do not learn the shared representation, in inverted pendulum and 6-DoF drone control tasks under varying wind conditions.

In this paper, we present Neural-Swarm, a nonlinear decentralized stable controller for close-proximity flight of multirotor swarms. Close-proximity control is challenging due to the complex aerodynamic interaction effects between multirotors, such as downwash from higher vehicles to lower ones. Conventional methods often fail to properly capture these interaction effects, resulting in controllers that must maintain large safety distances between vehicles, and thus are not capable of close-proximity flight. Our approach combines a nominal dynamics model with a regularized permutation-invariant Deep Neural Network (DNN) that accurately learns the high-order multi-vehicle interactions. We design a stable nonlinear tracking controller using the learned model. Experimental results demonstrate that the proposed controller significantly outperforms a baseline nonlinear tracking controller with up to four times smaller worst-case height tracking errors. We also empirically demonstrate the ability of our learned model to generalize to larger swarm sizes.

Fixed-wing vertical take-off and landing (VTOL) aircraft pose a unique control challenge that stems from complex aerodynamic interactions between wings and rotors. Thus, accurate estimation of external forces is indispensable for achieving high performance flight. In this paper, we present a composite adaptive nonlinear tracking controller for a fixed- wing VTOL. The method employs online adaptation of linear force models, and generates accurate estimation for wing and rotor forces in real-time based on information from a three-dimensional airflow sensor. The controller is implemented on a custom-built fixed-wing VTOL, which shows improved velocity tracking and force prediction during the transition stage from hover to forward flight, compared to baseline flight controllers.

We present a novel approach to reduce the processing time required to derive the estimation uncertainty map in deep learning-based optical flow determination methods. Without uncertainty aware reasoning, the optical flow model, especially when it is used for mission critical fields such as robotics and aerospace, can cause catastrophic failures. Although several approaches such as the ones based on Bayesian neural networks have been proposed to handle this issue, they are computationally expensive. Thus, to speed up the processing time, our approach applies a generative model, which is trained by input images and an uncertainty map derived through a Bayesian approach. By using synthetically generated images of spacecraft, we demonstrate that the trained generative model can produce the uncertainty map 100~700 times faster than the conventional uncertainty estimation method used for training the generative model itself. We also show that the quality of uncertainty map derived by the generative model is close to that of the original uncertainty map. By applying the proposed approach, the deep learning model operated in real-time can avoid disastrous failures by considering the uncertainty as well as achieving better performance removing uncertain portions of the prediction result.

This paper presents competitive algorithms for a novel class of online optimization problems with memory. We consider a setting where the learner seeks to minimize the sum of a hitting cost and a switching cost that depends on the previous $p$ decisions. This setting generalizes Smoothed Online Convex Optimization. The proposed approach, Optimistic Regularized Online Balanced Descent, achieves a constant, dimension-free competitive ratio. Further, we show a connection between online optimization with memory and online control with adversarial disturbances. This connection, in turn, leads to a new constant-competitive policy for a rich class of online control problems.

We study the impact of predictions in online Linear Quadratic Regulator control with both stochastic and adversarial disturbances in the dynamics. In both settings, we characterize the optimal policy and derive tight bounds on the minimum cost and dynamic regret. Perhaps surprisingly, our analysis shows that the conventional greedy MPC approach is a near-optimal policy in both stochastic and adversarial settings. Specifically, for length-$T$ problems, MPC requires only $O(\log T)$ predictions to reach $O(1)$ dynamic regret, which matches (up to lower-order terms) our lower bound on the required prediction horizon for constant regret.

Precise near-ground trajectory control is difficult for multi-rotor drones, due to the complex aerodynamic effects caused by interactions between multi-rotor airflow and the environment. Conventional control methods often fail to properly account for these complex effects and fall short in accomplishing smooth landing. In this paper, we present a novel deep-learning-based robust nonlinear controller (Neural-Lander) that improves control performance of a quadrotor during landing. Our approach combines a nominal dynamics model with a Deep Neural Network (DNN) that learns high-order interactions. We apply spectral normalization (SN) to constrain the Lipschitz constant of the DNN. Leveraging this Lipschitz property, we design a nonlinear feedback linearization controller using the learned model and prove system stability with disturbance rejection. To the best of our knowledge, this is the first DNN-based nonlinear feedback controller with stability guarantees that can utilize arbitrarily large neural nets. Experimental results demonstrate that the proposed controller significantly outperforms a Baseline Nonlinear Tracking Controller in both landing and cross-table trajectory tracking cases. We also empirically show that the DNN generalizes well to unseen data outside the training domain.

The papers in this special section present recent advances in aerial swarm robotics, and aims to put together a cohesive set of research goals and visions toward realizing fully autonomous aerial swarm systems. One objective is to emphasize the three-way tradeoff among computational efficiency for large-scale swarms, stability, and robustness under uncertainty, and the optimal system performance. Aerial robotics has been one of the most active areas of research within the robotics community, and recently there have been many reports of promising results in aerial swarm systems. This is partly due to the commoditization of multicopter platforms, and communication, sensing, and processing hardware that has substantially lowered the barriers to entry to the field of aerial swarm robotics. Aerial swarms differ from swarms of ground-based vehicles in two major respects: Aerial robots or unmanned aerial vehicles (UAVs) operate in a three-dimensional space, and the dynamics of individual vehicles add an extra layer of complexity to the problems of path planning and trajectory design. Furthermore, the success of aerial swarms is predicated on the distributed and synergistic capabilities of individual and cooperative control, estimation, and decision making of aerial robots with limited resources, such as modest onboard computation and sensing capabilities and size, weight, and power constraints.

The use of aerial swarms to solve real-world problems has been increasing steadily, accompanied by falling prices and improving performance of communication, sensing, and processing hardware. The commoditization of hardware has reduced unit costs, thereby lowering the barriers to entry to the field of aerial swarm robotics. A key enabling technology for swarms is the family of algorithms that allow the individual members of the swarm to communicate and allocate tasks amongst themselves, plan their trajectories, and coordinate their flight in such a way that the overall objectives of the swarm are achieved efficiently. These algorithms, often organized in a hierarchical fashion, endow the swarm with autonomy at every level, and the role of a human operator can be reduced, in principle, to interactions at a higher level without direct intervention. This technology depends on the clever and innovative application of theoretical tools from control and estimation. This paper reviews the state of the art of these theoretical tools, specifically focusing on how they have been developed for, and applied to, aerial swarms. Aerial swarms differ from swarms of ground-based vehicles in two respects: they operate in a three-dimensional space and the dynamics of individual vehicles adds an extra layer of complexity. We review dynamic modeling and conditions for stability and controllability that are essential in order to achieve cooperative flight and distributed sensing. The main sections of this paper focus on major results covering trajectory generation, task allocation, adversarial control, distributed sensing, monitoring, and mapping. Wherever possible, we indicate how the physics and subsystem technologies of aerial robots are brought to bear on these individual areas.

In this paper, we derive an algorithm for enabling a single robotic unmanned aerial vehicle to herd a flock of birds away from a designated volume of space, such as the air space around an airport. The herding algorithm, referred to as the m-waypoint algorithm, is designed using a dynamic model of bird flocking based on Reynolds' rules. We derive bounds on its performance using a combination of reduced-order modeling of the flock's motion, heuristics, and rigorous analysis. A unique contribution of the paper is the experimental demonstration of several facets of the herding algorithm on flocks of live birds reacting to a robotic pursuer. The experiments allow us to estimate several parameters of the flocking model, and especially the interaction between the pursuer and the flock. The herding algorithm is also demonstrated using numerical simulations.

Bats are unique in that they can achieve unrivaled agile maneuvers due to their functionally versatile wing conformations. Among these maneuvers, roosting (landing) has captured attentions because bats perform this acrobatic maneuver with a great composure. This work attempts to reconstruct bat landing maneuvers with a Micro Aerial Vehicle (MAV) called Allice. Allice is capable of adjusting the position of its Center of Gravity (CG) with respect to the Center of Pressure (CP) using a nonlinear closed-loop feedback. This nonlinear control law, which is based on the method of input-output feedback linearization, enables attitude regulations through variations in CG-CP distance. To design the model-based nonlinear controller, the Newton-Euler dynamic model of the robot is considered, in which the aerodynamic coefficients of lift and drag are obtained experimentally. The performance of the proposed control architecture is validated by conducting several experiments.

We present a distributed control algorithm simultaneously solving both the stochastic target assignment and optimal motion control for large-scale swarms to achieve complex formation shapes. Our probabilistic swarm guidance using inhomogeneous Markov chains (PSG-IMC) algorithm adopts a Eulerian density-control framework, under which the physical space is partitioned into multiple bins and the swarm's density distribution over each bin is controlled in a probabilistic fashion to efficiently handle loss or the addition of agents. We assume that the number of agents is much larger than the number of bins and that each agent knows in which bin it is located, the desired formation shape, and the objective function and motion constraints. PSG-IMC determines the bin-to-bin transition probabilities of each agent using a time IMC. These time-varying Markov matrices are computed by each agent in real time using the feedback from the current swarm distribution, which is estimated in a distributed manner. The PSG-IMC algorithm minimizes the expected cost of transitions per time instant that are required to achieve and maintain the desired formation shape, even if agents are added to or removed from the swarm. PSG-IMC scales well with a large number of agents and complex formation shapes and can also be adapted for area exploration applications. We demonstrate the effectiveness of this proposed swarm guidance algorithm by using numerical simulations and hardware experiments with multiple quadrotors.

It is challenging to analyze the aerial locomotion of bats because of the complicated and intricate relationship between their morphology and flight capabilities. Developing a biologically inspired bat robot would yield insight into how bats control their body attitude and position through the complex interaction of nonlinear forces (e.g., aerodynamic) and their intricate musculoskeletal mechanism. The current work introduces a biologically inspired soft robot called Bat Bot (B2). The overall system is a flapping machine with 5 Degrees of Actuation (DoA). This work reports on some of the preliminary untethered flights of B2. B2 has a nontrivial morphology and it has been designed after examining several biological bats. Key DoAs, which contribute significantly to bat flight, are picked and incorporated in B2's flight mechanism design. These DoAs are: 1) forelimb flapping motion, 2) forelimb mediolateral motion (folding and unfolding) and 3) hindlimb dorsoventral motion (upward and downward movement).

We present a novel method for guiding a large-scale swarm of autonomous agents into a desired formation shape in a distributed and scalable manner. Our Probabilistic Swarm Guidance using Inhomogeneous Markov Chains (PSG-IMC) algorithm adopts an Eulerian framework, where the physical space is partitioned into bins and the swarm's density distribution over each bin is controlled. Each agent determines its bin transition probabilities using a time-inhomogeneous Markov chain. These time-varying Markov matrices are constructed by each agent in real-time using the feedback from the current swarm distribution, which is estimated in a distributed manner. The PSG-IMC algorithm minimizes the expected cost of the transitions per time instant, required to achieve and maintain the desired formation shape, even when agents are added to or removed from the swarm. The algorithm scales well with a large number of agents and complex formation shapes. We demonstrate the effectiveness of this proposed swarm guidance algorithm by using results of numerical simulations and hardware experiments with multiple quadrotors.

We present a simultaneous localization and mapping (SLAM) algorithm that uses Bézier curves as static landmark primitives rather than sparse feature points. Our approach allows us to estimate the full 6-DOF pose of a robot while providing a structured map which can be used to assist a robot in motion planning and control. We demonstrate how to reconstruct the 3-D location of curve landmarks from a stereo pair without searching for point-based stereo correspondences and how to compare the 3-D shape of curve landmarks between chronologically sequential stereo frames to solve the data association problem. We present a method to combine curve landmarks for mapping purposes, resulting in a map with a continuous set of curves that contain fewer landmark states than conventional sparse point-based SLAM algorithms. Note, to combine curves, we assume the curved landmarks are fixed to a larger curved object naturally occurring in the scene. While our algorithm is less accurate than point-based SLAM algorithms, we are able to create maps with considerably less landmark states and our algorithm can operate in settings lacking texture.

In this paper, we present an omnidirectional-vision-based localization and mapping system which can detect whether a robotic mower is contained in a permitted area. We exploit a robot-centric mapping framework that exploits a differential equation of motion of the landmarks, which are referenced with respect to the robot body frame. The estimator in our system generates a 3D point-based map with landmarks. Concurrently, the estimator defines a boundary of the mowing area with the estimated trajectory of the mower. The estimated boundary and the landmark map are provided for the estimation of the mowing location and for the containment detection. We validate the effectiveness of our system through numerical simulations and present the results of the outdoor experiment that we conducted with our robotic mower.

This paper presents a systematic flight controller design based on the mathematics of parametrized manifolds and calculus of variations for the Bat Bot (B2), which possesses many articulated wings. Wing kinematics and morphological properties are crucial in the powered flight of flying vertebrates. The articulated skeleton of these mammals, which contains many degrees of actuation and underactuation, has made it difficult to understand the connection between the bat's flight dynamics and its intricate array of physiological and morphological specializations. B2 is a biomimetic micro aerial vehicle (MAV) that possesses similar morphological properties to a bat in order to duplicate bats powered ballistic motion. In an effort to design the advanced flight control algorithm for B2, this paper reports two major contributions. First, a systematic mathematical framework is introduced that evaluates the holonomically-constrained Lagrangian model of a flapping robot with specified active and passive degrees of freedom (DoF) in order to locate physically feasible and biologically meaningful periodic solutions using optimization. These are parametrized constraint manifolds; the flapping wing dynamics are governed by these manifolds. Second, calculus of variations and the well-recognized method of inverse dynamics are applied in order to synthesize the flight control algorithm for the flapping wings.

We study the problem of minimizing the total distance incurred in assigning a group of mobile robots to an equal number of static targets. Assuming that the robots have limited, range-based communication and target-sensing capabilities, we present a necessary and sufficient condition for ensuring distance optimality when robots and targets are uniformly randomly distributed. We then provide an explicit, non-asymptotic formula for computing the number of robots needed for guaranteeing optimality in terms of the robots' sensing and communication capabilities with arbitrarily high probabilities. The bound given in the formula is also asymptotically tight. Due to the large number of robots needed for high-probability optimality guarantee, we continue to investigate strategies for cases in which the number of robots cannot be freely chosen. We show that a properly designed strategy can be asymptotically optimal or suboptimal with constant approximation ratios.

In this paper, we integrate, implement, and validate formation flying algorithms for a large number of agents using probabilistic guidance of distributed systems with inhomogeneous Markov chains and model predictive control with sequential convex programming. Using an inhomogeneous Markov chain, each agent determines its target position during each iteration in a statistically independent manner while the distributed system converges to the desired formation. Moreover, the distributed system is robust to external disturbances or damages to the formation. Once the target positions are assigned, an optimal control problem is formulated to ensure that the agents reach the target positions while avoiding collisions. This problem is solved using sequential convex programming to determine optimal, collision-free trajectories and model predictive control is implemented to update these trajectories as new state information becomes available. Finally, we validate the probabilistic guidance of distributed systems and model predictive control algorithms using the formation flying testbed.

This paper presents a novel vision-based localization and mapping algorithm using image moments of region features. The environment is represented using regions, such as planes and/or 3D objects instead of only a dense set of feature points. The regions can be uniquely defined using a small number of parameters; e.g., a plane can be completely characterized by normal vector and distance to a local coordinate frame attached to the plane. The variation of image moments of the regions in successive images can be related to the parameters of the regions. Instead of tracking a large number of feature points, variations of image moments of regions can be computed by tracking the segmented regions or a few feature points on the objects in successive images. A map represented by regions can be characterized using a minimal set of parameters. The problem is formulated as a nonlinear filtering problem. A new discrete-time nonlinear filter based on the state-dependent coefficient (SDC) form of nonlinear functions is presented. It is shown via Monte-Carlo simulations that the new nonlinear filter is more accurate and consistent than EKF by evaluating the root-mean squared error (RMSE) and normalized estimation error squared (NEES).

This paper addresses the problem of motion planning for fast, agile flight through a dense obstacle field. A key contribution is the design of two families of motion primitives for aerial robots flying in dense obstacle fields, along with rules to stitch them together. The primitives are obtained by solving for the flight dynamics of the aerial robot, and explicitly account for limited agility using time delays. The first family of primitives consists of turning maneuvers to link any two points in space. The locations of the terminal points are used to obtain closed-form expressions for the control inputs required to fly between them, while accounting for the finite time required to switch between consecutive sets of control inputs. The second family consists of aggressive turn-around maneuvers wherein the time delay between the angle of attack and roll angle commands is used to optimize the maneuver for the spatial constraints. A 3-D motion planning algorithm based on these primitives is presented for aircraft flying through a dense forest.

This paper presents a vision-based localization and mapping algorithm for an autonomous mower. We divide the task for robotic mowing into two separate phases, a teaching phase and a mowing phase. During the teaching phase, the mower estimates the 3D positions of landmarks and defines a boundary in the lawn with an estimate of its own trajectory. During the mowing phase, the location of the mower is estimated using the landmark and boundary map acquired from the teaching phase. Of particular interest for our work is ensuring that the estimator for landmark mapping will not fail due to the nonlinearity of the system during the teaching phase. A nonlinear observer is designed with pseudo-measurements of each landmark's depth to prevent the map estimator from diverging. Simultaneously, the boundary is estimated with an EKF. Measurements taken from an omnidirectional camera, an IMU, and a ground speed sensor are used for the estimation. Numerical simulations and offline teaching phase experiments with our autonomous mower demonstrate the potential of our algorithm.

We describe the design of an aerial robot inspired by birds and the underlying theoretical developments leading to novel control and closed-loop guidance algorithms for a perching maneuver. A unique feature of this robot is that it uses wing articulation to control the flight path angle as well as the heading angle. It lacks a vertical tail for improved agility, which results in unstable lateral-directional dynamics. New closed-loop motion planning algorithms with guaranteed stability are obtained by rewriting the flight dynamic equations in the spatial domain rather than as functions of time, after which dynamic inversion is employed. It is shown that nonlinear dynamic inversion naturally leads to proportional-integral-derivative controllers, thereby providing an exact method for tuning the gains. The capabilities of the proposed bioinspired robot design and its novel closed-loop perching controller have been successfully demonstrated with perched landings on a human hand.

This paper presents a boundary control formulation for distributed parameter systems described by partial differential equations (PDEs) and whose output is given by a spatial integral of weighted functions of the state. This formulation is directly applicable to the control of small robotic aircraft with articulated flexible wings, where the output of interest is the net aerodynamic force or moment. The deformation of flexible wings can be controlled by actuators that are located at the root or the tip of the wing. The problem of designing a tracking controller for wing twist is addressed using a combination of PDE backstepping for feedback stabilization and feed-forward trajectory planning. We also design an adaptive tracking controller for wing tip actuators. For wing bending, we present a novel control scheme that is based on a two-stage perturbation observer. A trajectory planning-based feed-forward tracker is designed using only one component of the observer whose dynamics are homogeneous and amenable to trajectory planning. The two components, put together, estimate the external forces and unmodeled system dynamics. The effectiveness of the proposed controllers for twist and bending is demonstrated by simulations. This paper also reports experimental validation of the perturbation-observer-based controller for beam bending.

A new design for a dielectric elastomer actuator with geometrically confining reinforcements is presented. The resulting structures enable complex 3-dimentional motion without the need of the membrane prestretch. An in situ imaging system is used to capture the complex deformation pattern to evaluate the surface curvatures. The deformation mode is analyzed analytically using the bi-laminate theory to explore the actuator performance and further develop analytical model amenable for control strategies. A finite element material model is also developed to couple the applied electric field to the resulting deformation. The model is used to analyze more complex deformation patterns. The proposed confining reinforcements would enable the development of flexible wings for agile aerial robotics and compliant continuum robotics, utilizing the proposed deformation mechanisms to provide controllable many degrees of freedom.

Existing approaches to visual Simultaneous Localization and Mapping (SLAM) typically utilize points as visual feature primitives to represent landmarks in the environment. Since these techniques mostly use image points from a standard feature point detector, they do not explicitly map objects or regions of interest. Our work is motivated by the need for different SLAM techniques in path and riverine settings, where feature points can be scarce or may not adequately represent the environment. Accordingly, the proposed approach uses cubic Bézier curves as stereo vision primitives and offers a novel SLAM formulation to update the curve parameters and vehicle pose. This method eliminates the need for point-based stereo matching, with an optimization procedure to directly extract the curve information in the world frame from noisy edge measurements. Further, the proposed algorithm enables navigation with fewer feature states than most point-based techniques, and is able to produce a map which only provides detail in key areas. Results in simulation and with vision data validate that the proposed method can be effective in estimating the 6DOF pose of the stereo camera, and can produce structured, uncluttered maps.

This paper presents a novel indoor navigation and ranging strategy by using a monocular camera. The proposed algorithms are integrated with simultaneous localization and mapping (SLAM) with a focus on indoor aerial vehicle applications. We experimentally validate the proposed algorithms by using a fully self-contained micro aerial vehicle (MAV) with on-board image processing and SLAM capabilities. The range measurement strategy is inspired by the key adaptive mechanisms for depth perception and pattern recognition found in humans and intelligent animals. The navigation strategy assumes an unknown, GPS-denied environment, which is representable via corner-like feature points and straight architectural lines. Experimental results show that the system is only limited by the capabilities of the camera and the availability of good corners.

Concurrent synchronization is a regime where diverse groups of fully synchronized dynamic systems stably coexist. We study global exponential synchronization and concurrent synchronization in the context of Lagrangian systems control. In a network constructed by adding diffusive couplings to robot manipulators or mobile robots, a decentralized tracking control law globally exponentially synchronizes an arbitrary number of robots, and represents a generalization of the average consensus problem. Exact nonlinear stability guarantees and synchronization conditions are derived by contraction analysis. The proposed decentralized strategy is further extended to adaptive synchronization and partial-state coupling.