We demonstrate the capabilities of an attentionbased end-to-end approach for high-speed vision-based quadrotor obstacle avoidance in dense, cluttered environments, with comparison to various state-of-the-art learning architectures. Quadrotor unmanned aerial vehicles (UAVs) have tremendous maneuverability when flown fast; however, as flight speed increases, traditional model-based approaches to navigation via independent perception, mapping, planning, and control modules breaks down due to increased sensor noise, compounding errors, and increased processing latency. Thus, learning-based, end-to-end vision-to-control networks have shown to have great potential for online control of these fast robots through cluttered environments. We train and compare convolutional, U-Net, and recurrent architectures against vision transformer (ViT) models for depth image-to-control in high-fidelity simulation, observing that ViT models are more effective than others as quadrotor speeds increase and in generalization to unseen environments, while the addition of recurrence further improves performance while reducing quadrotor energy cost across all tested flight speeds. We assess performance at speeds of up to 7m/s in simulation and hardware. To the best of our knowledge, this is the first work to utilize vision transformers for end-to-end vision-based quadrotor control.

The multi-objective coverage control problem requires a robot swarm to collaboratively provide sensor coverage to multiple heterogeneous importance density fields (IDFs) simultaneously. We pose this as an optimization problem with constraints and study two different formulations: (1) Fair coverage, where we minimize the maximum coverage cost for any field, promoting equitable resource distribution among all fields; and (2) Constrained coverage, where each field must be covered below a certain cost threshold, ensuring that critical areas receive adequate coverage according to predefined importance levels. We study the decentralized setting where robots have limited communication and local sensing capabilities, making the system more realistic, scalable, and robust. Given the complexity, we propose a novel decentralized constrained learning approach that combines primal-dual optimization with a Learnable Perception-Action-Communication (LPAC) neural network architecture. We show that the Lagrangian of the dual problem can be reformulated as a linear combination of the IDFs, enabling the LPAC policy to serve as a primal solver. We empirically demonstrate that the proposed method (i) significantly outperforms state-of-the-art decentralized controllers by 30% on average in terms of coverage cost, (ii) transfers well to larger environments with more robots, and (iii) is scalable in the number of IDFs and robots in the swarm.

Traditionally, unmanned aerial vehicles (UAVs) rely on CMOS-based cameras to collect images about the world below. One of the most successful applications of UAVs is to generate orthomosaics or orthomaps, in which a series of images are integrated to develop a larger map. However, using CMOS-based cameras with global or rolling shutters means that orthomaps are vulnerable to challenging light conditions, motion blur, and high-speed motion of independently moving objects (IMOs) under the camera. Event cameras are less sensitive to these issues, as their pixels trigger asynchronously on brightness changes. This work introduces the first orthomosaic approach using event cameras. We focus on addressing high-dynamic range and low-light problems in orthomosaics. In contrast to existing methods relying only on CMOS cameras, our approach enables map generation even in challenging light conditions, including direct sunlight and after sunset. The source code for EvMAPPER, the high-altitude hardware, and the dataset collected in this paper are available open source11https://evmapper.fcladera.com.

Urban winds are a serious hazard for low-altitude autonomous aerial operations in urban airspaces. Previous methods for motion planning in urban winds require global knowledge of the obstacles and flow field and do not lend themselves to real-time application. In this paper, a planning and control framework is proposed for safe and energy-efficient navigation through urban flow fields that strictly relies on onboard sensing. The algorithm incorporates predictions of local wind flow fields into a receding horizon optimal controller, balancing energy consumption with obstacle avoidance on the fly to reach a goal destination. Simulation studies on a procedurally generated urban map with diverse wind conditions demonstrate that the energy-aware motion planner reduces energy consumption by as much as 30% and results in 32% fewer crashes on average compared to the wind-agnostic baseline. Comparisons to a global wind-aware planner indicate only minor trade-offs associated with planning on a local horizon.

Active perception approaches select future viewpoints by using some estimate of the information gain. An inaccurate estimate can be detrimental in critical situations, e.g., locating a person in distress. However the true information gained can only be calculated post hoc, i.e., after the observation is realized. We present an approach to estimate the discrepancy between the estimated information gain (which is the expectation over putative future observations while neglecting correlations among them) and the true information gain. The key idea is to analyze the mathematical relationship between active perception and the estimation error of the information gain in a gametheoretic setting. Using this, we develop an online estimation approach that achieves sub-linear regret (in the number of timesteps) for the estimation of the true information gain and reduces the sub-optimality of active perception systems. We demonstrate our approach11Code is available at https://github.com/grasp-lyd/active-perception-game. Proofs are available at https://arxiv.org/abs/2404.00769.for active perception using a comprehensive set of experiments on: (a) different types of environments, including a quadrotor in a photorealistic simulation, real-world robotic data, and real-world experiments with ground robots exploring indoor and outdoor scenes; (b) different types of robotic perception data; and (c) different map representations. On average, our approach reduces information gain estimation errors by 42%, increases the information gain by 7%, PSNR by 5%, and semantic accuracy (measured as the number of objects that are localized correctly) by 6%. In real-world experiments with a Jackal ground robot, our approach demonstrated complex trajectories to explore occluded regions.

Trajectory generation in dynamic environments presents a significant challenge for quadrotors, particularly due to the non-convexity in the spatial-temporal domain. Many existing methods either assume simplified static environments or struggle to produce optimal solutions in real-time. In this work, we propose an efficient safe interval motion planning framework for navigation in dynamic environments. A safe interval refers to a time window during which a specific configuration is safe. Our approach addresses trajectory generation through a two-stage process: a front-end graph search step followed by a back-end gradient-based optimization. We ensure completeness and optimality by constructing a dynamic connected visibility graph and incorporating low-order dynamic bounds within safe intervals and temporal corridors. To avoid local minima, we propose a Uniform Temporal Visibility Deformation (UTVD) for the complete evaluation of spatial-temporal topological equivalence. We represent trajectories with B-Spline curves and apply gradient-based optimization to navigate around static and moving obstacles within spatial-temporal corridors. Through simulation and real-world experiments, we show that our method can achieve a success rate of over$\mathbf{9 5 \%}$in environments with different density levels, exceeding the performance of other approaches, demonstrating its potential for practical deployment in highly dynamic environments.

As robots become increasingly capable, users will want to describe high-level missions and have robots infer the relevant details. Because pre-built maps are difficult to obtain in many realistic settings, accomplishing such missions will require the robot to map and plan online. While many semantic planning methods operate online, they are typically designed for well specified missions such as object search or exploration. Recently, Large Language Models (LLMs) have demonstrated powerful contextual reasoning abilities over a range of robotic tasks described in natural language. However, existing LLM-enabled planners typically do not consider online planning or complex missions; rather, relevant subtasks and semantics are provided by a pre-built map or a user. We address these limitations via SPINE, an online planner for missions with incomplete mission specifications provided in natural language. The planner uses an LLM to reason about subtasks implied by the mission specification and then realizes these subtasks in a receding horizon framework. Tasks are automatically validated for safety and refined online with new map observations. We evaluate SPINE in simulation and real-world settings with missions that require multiple steps of semantic reasoning and exploration in cluttered outdoor environments of over 20,000m2. Compared to baselines that use existing LLM-enabled planning approaches, our method is over twice as efficient in terms of time and distance, requires less user interactions, and does not require a full map. Additional resources are provided at https://zacravichandran.github.io/SPINE.

The recent introduction of large language models (LLMs) has revolutionized the field of robotics by enabling contextual reasoning and intuitive human-robot interaction in domains as varied as manipulation, locomotion, and self-driving vehicles. When viewed as a stand-alone technology, LLMs are known to be vulnerable to jailbreaking attacks, wherein mali-cious prompters elicit harmful text by bypassing LLM safety guardrails. To assess the risks of deploying LLMs in robotics, in this paper, we introduce ROBOPAIR, the first algorithm designed to jailbreak LLM-controlled robots. Unlike existing, textual attacks on LLM chatbots, Robopairelicits harmful physical actions from LLM-controlled robots, a phenomenon we experimentally demonstrate in three scenarios: (i) a white-box setting, wherein the attacker has full access to the NVID IA Dolphins self-driving LLM, (ii) a gray-box setting, wherein the attacker has partial access to a Clearpath Robotics Jackal UGV robot equipped with a GPT-40 planner, and (iii) a black-box setting, wherein the attacker has only query access to the GPT-3.5-integrated Unitree Robotics Go2robot dog. In each scenario and across three new datasets of harmful robotic actions, we demonstrate that ROBOPAIR, as well as several static baselines, finds jailbreaks quickly and effectively, often achieving 100 % attack success rates. Our results reveal, for the first time, that the risks of jailbroken LLMs extend far beyond text generation, given the distinct possibility that jailbroken robots could cause physical damage in the real world. Indeed, our results on the U nitree G02represent the first successful jailbreak of a deployed commercial robotic system. Addressing this emerging vulnerability is critical for ensuring the safe deployment of LLMs in robotics. Additional media is available at: https://robopair.org.

We consider the problem of safe real-time navigation of a robot in a dynamic environment with moving obstacles of arbitrary smooth geometries and input saturation constraints. We assume that the robot detects and models nearby obstacle boundaries with a short-range sensor and that this detection is error-free. This problem presents three main challenges: i) input constraints, ii) safety, and iii) real-time computation. To tackle all three challenges, we present a layered control architecture (LCA) consisting of an offline path library generation layer, and an online path selection and safety layer. To overcome the limitations of reactive methods, our offline path library consists of feasible controllers, feedback gains, and reference trajectories. To handle computational burden and safety, we solve online path selection and generate safe inputs that run at 100 Hz. Through simulations on Gazebo and Fetch hardware in an indoor environment, we evaluate our approach against baselines that are layered, end - to-end, or reactive. Our experiments demonstrate that among all algorithms, only our proposed LCA is able to complete tasks such as reaching a goal, safely. When comparing metrics such as safety, input error, and success rate, we show that our approach generates safe and feasible inputs throughout the robot execution.

Tracking controllers enable robotic systems to accurately follow planned reference trajectories. In particular, reinforcement learning (RL) has shown promise in the synthesis of controllers for systems with complex dynamics and modest online compute budgets. However, the poor sample efficiency of RL and the challenges of reward design make training slow and sometimes unstable, especially for high-dimensional systems. In this work, we leverage the inherent Lie group symmetries of robotic systems with a floating base to mitigate these challenges when learning tracking controllers. We model a general tracking problem as a Markov decision process (MDP) that captures the evolution of both the physical and reference states. Next, we prove that symmetry in the underlying dynamics and running costs leads to an MDP homomorphism, a mapping that allows a policy trained on a lower-dimensional “quotient” MDP to be lifted to an optimal tracking controller for the original system. We compare this symmetry-informed approach to an unstructured baseline, using Proximal Policy Optimization (PPO) to learn tracking controllers for three systems: the Particle (a forced point mass), the Astrobee (a fully-actuated space robot), and the Quadrotor (an underactuated system). Results show that a symmetry-aware approach both accelerates training and reduces tracking error at convergence.

Multi-robot collaboration for target tracking in adversarial environments poses significant challenges, including system failures, dynamic priority shifts, and other unpredictable factors. These challenges become even more pronounced when the environment is unknown. In this paper, we propose a resilient coordination framework for multi-robot, multi-target tracking in environments with unknown sensing and communication danger zones. We consider scenarios where failures caused by these danger zones are probabilistic and temporary, allowing robots to escape from danger zones to minimize the risk of future failures. We formulate this problem as a nonlinear optimization with soft chance constraints, enabling real-time adjustments to robot behaviors based on varying types of dangers and failures. This approach dynamically balances target tracking performance and resilience, adapting to evolving sensing and communication conditions in real-time. To validate the effectiveness of the proposed method, we assess its performance across various tracking scenarios, benchmark it against methods without resilient adaptation and collaboration, and conduct several real-world experiments.

Coverage control is the problem of navigating a robot swarm to collaboratively monitor features or a phenomenon of interest not knowna priori. The problem is challenging in decentralized settings with robots that have limited communication and sensing capabilities. We propose a learnable Perception-Action-Communication (LPAC) architecture for the problem, wherein a convolutional neural network (CNN) processes localized perception; a graph neural network (GNN) facilitates robot communications; finally, a shallow multi-layer perceptron (MLP) computes robot actions. The GNN enables collaboration in the robot swarm by computingwhatinformation to communicate with nearby robots andhowto incorporate received information. Evaluations show that the LPAC models—trained using imitation learning—outperform standard decentralized and centralized coverage control algorithms. The learned policy generalizes to environments different from the training dataset, transfers to larger environments with more robots, and is robust to noisy position estimates. The results indicate the suitability of LPAC architectures for decentralized navigation in robot swarms to achieve collaborative behavior.

This paper develops a real-time decentralized metric-semantic SLAM algorithm that enables a heterogeneous robot team to collaboratively construct object-based metric-semantic maps. The proposed framework integrates a data-driven front-end for instance segmentation from either RGBD cameras or LiDARs and a custom back-end for optimizing robot trajectories and object landmarks in the map. To allow multiple robots to merge their information, we design semantics-driven place recognition algorithms that leverage the informativeness and viewpoint invariance of the object-level metric-semantic map for inter-robot loop closure detection. A communication module is designed to track each robot's observations and those of other robots whenever communication links are available. The framework supports real-time, decentralized operation onboard the robots and has been integrated with three types of aerial and ground platforms. We validate its effectiveness through experiments in both indoor and outdoor environments, as well as benchmarks on public datasets and comparisons with existing methods. The framework is open-sourced and suitable for both single-agent and multi-robot real-time metric-semantic SLAM applications.

Data collection for forestry, timber, and agriculture relies on manual techniques which are labor-intensive and time-consuming. We seek to demonstrate that robotics offers improvements over these techniques and can accelerate agricultural research, beginning with semantic segmentation and diameter estimation of trees in forests and orchards. We present TreeScope v1.0, the first robotics dataset for precision agriculture and forestry addressing the counting and mapping of trees in forestry and orchards. TreeScope provides LiDAR data from agricultural environments collected with robotics platforms, such as UAV and mobile robot platforms carried by vehicles and human operators. In the first release of this dataset, we provide ground-truth data with over 1,800 manually annotated semantic labels for tree stems and field-measured tree diameters. We share benchmark scripts for these tasks that researchers may use to evaluate the accuracy of their algorithms. Finally, we run our open-source diameter estimation and off-the-shelf semantic segmentation algorithms and share our baseline results.The dataset can be found at https://treescope.org, and the data pre-processing and benchmark code is available at https://github.com/KumarRobotics/treescope.

In this paper we present a new approach to Mobile Infrastructure on Demand (MID) where a dedicated team of robots creates and sustains a wireless network that satisfies the communication requirements of a different team of task-oriented robots seeking to coordinate their actions in the absence of existing communication infrastructure. Different from previous works, our approach forgoes heuristics for network performance such as algebraic-connectivity or network flow optimizations and instead positions communication support robots to directly maximize the probability of packet delivery by the underlying opportunistic routing protocol. Our system is task agnostic and practical to implement and operate on robots equipped with off-the-shelf WiFi radios. We demonstrate this through a set of experiments showing our MID system maintaining the delivery of critical mission data in a situational awareness setting and enabling foraging robots to effectively coordinate their actions during multi-robot exploration.

Multi-robot collaboration in large-scale environments with limited-sized teams and without external infrastructure is challenging, since the software framework required to support complex tasks must be robust to unreliable and intermittent communication links. In this work, we present MOCHA (Multi-robot Opportunistic Communication for Heterogeneous Collaboration), a framework for resilient multi-robot collaboration that enables large-scale exploration in the absence of continuous communications. MOCHA is based on a gossip communication protocol that allows robots to interact opportunistically whenever communication links are available, propagating information on a peer-to-peer basis. We demonstrate the performance of MOCHA through real-world experiments with commercial-off-the-shelf (COTS) communication hardware. We further explore the system’s scalability in simulation, evaluating the performance of our approach as the number of robots increases and communication ranges vary. Finally, we demonstrate how MOCHA can be tightly integrated with the planning stack of autonomous robots. We show a communication-aware planning algorithm for a high-altitude aerial robot executing a collaborative task while maximizing the amount of information shared with ground robots.The source code for MOCHA and the high-altitude UAV planning system is available open source1.

Motion planning techniques for quadrotors have advanced significantly over the past decade. Most successful planners have two stages: a front-end that determines a path that incorporates geometric (or kinematic or input) constraints and specifies the homotopy class of the trajectory, and a back-end that optimizes this path to respect dynamics and input constraints. While there are many different choices for each stage, the eventual performance depends critically not only on these choices, but also on the environment. Given a new environment, it is difficult to decide a priori how one should design a motion planner. In this work, we develop (i) a procedure to construct parametrized environments, (ii) metrics that characterize the difficulty of motion planning in these environments, and (iii) an open-source software stack that can be used to combine a wide variety of two-stage planners seamlessly. We perform experiments in simulations and a real platform. We find, somewhat conveniently, that geometric front-ends are sufficient for environments with varying complexities if combined with dynamics-aware backends. The metrics we designed faithfully capture the planning difficulty in a given environment. All code is available at https://github.com/KumarRobotics/kr_mp_design.

In this paper, we address the challenge of exploring unknown indoor environments using autonomous aerial robots with Size Weight and Power (SWaP) constraints. The SWaP constraints induce limits on mission time requiring efficiency in exploration. We present a novel exploration framework that uses Deep Learning (DL) to predict the most likely indoor map given the previous observations, and Deep Reinforcement Learning (DRL) for exploration, designed to run on modern SWaP constraints neural processors. The DL-based map predictor provides a prediction of the occupancy of the unseen environment while the DRL-based planner determines the best navigation goals that can be safely reached to provide the most information. The two modules are tightly coupled and run onboard allowing the vehicle to safely map an unknown environment. Extensive experimental and simulation results show that our approach surpasses state-of-the-art methods by 50-60% in efficiency, which we measure by the fraction of the explored space as a function of the trajectory length.

Trajectory generation for quadrotors with limited field-of-view sensors has numerous applications such as aerial exploration, coverage, inspection, videography, and target tracking. Most previous works simplify the task of optimizing yaw trajectories by either aligning the heading of the robot with its velocity, or potentially restricting the feasible space of candidate trajectories by using a limited yaw domain to circumvent angular singularities. In this paper, we propose a novel global yaw parameterization method for trajectory optimization that allows a 360-degree yaw variation as demanded by the underlying algorithm. This approach effectively bypasses inherent singularities by including supplementary quadratic constraints and transforming the final decision variables into the desired state representation. This method significantly reduces the needed control effort, and improves optimization feasibility. Furthermore, we apply the method to several examples of different applications that require jointly optimizing over both the yaw and position trajectories. Ultimately, we present a comprehensive numerical analysis and evaluation of our proposed method in both simulation and real-world experiments.

Planning time-optimal trajectories for quadrotors in cluttered environments is a challenging, non-convex problem. This paper addresses minimizing the traversal time of a given collision-free geometric path without violating actuation bounds of the vehicle. Previous approaches have either relied on convex relaxations that do not guarantee dynamic feasibility or have generated overly conservative time parametrizations. We propose TOPPQuad, a time-optimal path parameterization algorithm for quadrotors which explicitly incorporates quadrotor rigid body dynamics and constraints, such as bounds on inputs (including motor thrusts) and state of the vehicle (including the pose, linear and angular velocity and acceleration). We demonstrate the ability of the planner to generate faster trajectories that respect hardware constraints of the robot compared to planners with relaxed notions of dynamic feasibility in both simulation and hardware. We also demonstrate how TOPPQuad can be used to plan trajectories for quadrotors that utilize bidirectional motors. Overall, the proposed approach paves a way towards maximizing the efficacy of autonomous micro aerial vehicles while ensuring their safety.

We present EvDNeRF, a pipeline for generating event data and training an event-based dynamic NeRF, for the purpose of faithfully reconstructing eventstreams on scenes with rigid and non-rigid deformations that may be too fast to capture with a standard camera. Event cameras register asynchronous per-pixel brightness changes at MHz rates with high dynamic range, making them ideal for observing fast motion with almost no motion blur. Neural radiance fields (NeRFs) offer visual-quality geometric-based learnable rendering, but prior work with events has only considered reconstruction of static scenes. Our EvDNeRF can predict eventstreams of dynamic scenes from a static or moving viewpoint between any desired timestamps, thereby allowing it to be used as an event-based simulator for a given scene. We show that by training on varied batch sizes of events, we can improve test-time predictions of events at fine time resolutions, outperforming baselines that pair standard dynamic NeRFs with event generators. We release our simulated and real datasets, as well as code for multi-view event-based data generation and the training and evaluation of EvDNeRF models1.

Despite the prevalence of wireless connectivity in urban areas around the globe, there remain numerous and diverse situations where connectivity is insufficient or unavailable. To address this, we introducemobile wireless infrastructure on demand, a system of unmanned aerial vehicles (UAVs) that can be rapidly deployed to establish an ad hoc wireless network. This network has the capability of reconfiguring itself dynamically to satisfy and maintain the required quality of communication. The system optimizes the positions of the UAVs and the routing of data flows throughout the network to achieve this Quality of Service (QoS). By these means, task agents using the network simply request a desired QoS, and the system adapts accordingly while allowing them to move freely. We have validated this system both in simulation and in real-world experiments. The results demonstrate that our system effectively offers mobile wireless infrastructure on demand, extending the operational range of task agents and supporting complex mobility patterns, all while ensuring connectivity and being resilient to agent failures.

We propose a new formulation for the multi-robot task planning and allocation problem that incorporates (a) precedence relationships between tasks; (b) coordination for tasks allowing multiple robots to achieve increased efficiency; and (c) cooperation through the formation of robot coalitions for tasks that cannot be performed by individual robots alone. In our formulation, the tasks and the relationships between the tasks are specified by a task graph. We define a set of reward functions over the task graph's nodes and edges. These functions model the effect of robot coalition size on task performance while incorporating the influence of one task's performance on a dependent task. Solving this problem optimally is NP-hard. However, using the task graph formulation allows us to leverage min-cost network flow approaches to obtain approximate solutions efficiently. Additionally, we explore a mixed integer programming approach, which gives optimal solutions for small instances of the problem but is computationally expensive. We also develop a greedy heuristic algorithm as a baseline. Our modeling and solution approaches result in task plans that leverage task precedence relationships and robot coordination and cooperation to achieve high mission performance, even in large missions with many agents.

Traditional approaches for active mapping focus on building geometric maps. For most real-world applications, however, actionable information is related to semantically meaningful objects in the environment. We propose an approach to the active metric-semantic mapping problem that enables multiple heterogeneous robots to collaboratively build a map of the environment. The robots actively explore to minimize the uncertainties in both semantic (object classification) and geometric (object modeling) information. We represent the environment using informative but sparse object models, each consisting of a basic shape and a semantic class label, and characterize uncertainties empirically using a large amount of real-world data. Given a prior map, we use this model to select actions for each robot to minimize uncertainties. The performance of our algorithm is demonstrated through multi-robot experiments in diverse real-world environments. The proposed framework is applicable to a wide range of real-world problems, such as precision agriculture, infrastructure inspection, and asset mapping in factories.

We investigate and develop algorithms for social fairness in coverage control problems. Existing coverage control methods are efficient, optimizing the average expected distance from any event to the nearest robot. However, in societal applications like disaster response or transportation, these conventional objectives lead to disparate coverage costs with respect to different groups within a population. We formulate social fairness for coverage control as the minimization of the maximum coverage cost among a set of groups within a population. Our approach uses Voronoi iteration to solve this novel problem by approximating the non-differentiable objective with the log-sum-exp and defining a gradient based controller that prioritizes fairness while also optimizing average performance when disparities between groups are low. We show convergence properties of this proposed control law and demonstrate the approach in simulations of randomly generated population densities as well as environments generated from U.S. census data on population rates and demographics. Our approach provides greater fairness than existing methods while maintaining similar computational time and convergence properties.

The use of bidirectional propellers provides quadrotors with greater maneuverability which is advantageous in constrained environments. This paper addresses the development of a trajectory planning algorithm for quadrotors with bidirectional motors. Previous work has shown that the property of differential flatness can be leveraged for efficient trajectory planning. However, planners that leverage flatness for quadrotors fail at points where the acceleration of the center of mass is equal to gravity, i.e., when the vehicle experiences free fall. The central contribution of this paper is a flatness-based trajectory planning method that allows quadrotors to use bidirectional propellers and pass through the so-called free-fall singularity. We model our system as a differentially flat hybrid system with the aid of coordinate charts derived from the Hopf fibration and develop an algorithm that computes forward and reverse thrusts for each propeller, resulting in smooth trajectories everywhere in SE(3). We demonstrate the planner's versatility by planning knife-edge maneuvers and trajectories passing through the free-fall singularity, while transitioning from forward to reverse thrust.

We address the problem of efficient 3-D exploration in indoor environments for micro aerial vehicles with limited sensing capabilities and payload/power constraints. We develop an indoor exploration framework that uses learning to predict the occupancy of unseen areas, extracts semantic features, samples viewpoints to predict information gains for different exploration goals, and plans informative trajectories to enable safe and smart exploration. Extensive experimentation in simulated and real-world environments shows the proposed approach outperforms the state-of-the-art exploration framework by 24% in terms of the total path length in a structured indoor environment and with a higher success rate during exploration.

Mechanical systems naturally evolve on principal bundles describing their inherent symmetries. The ensuing factorization of the configuration manifold into a symmetry group and an internal shape space has provided deep insights into the locomotion of many robotic and biological systems. On the other hand, the property of differential flatness has enabled efficient, effective planning and control algorithms for various robotic systems. Yet, a practical means of finding a flat output for an arbitrary robotic system remains an open question. In this work, we demonstrate surprising new connections between these two domains, for the first time employing symmetry directly to construct a flat output. We provide sufficient conditions for the existence of a trivialization of the bundle in which the group variables themselves are a flat output. We call this a geometric flat output, since it is equivariant (i.e. it preserves the symmetry) and often global or almost global, properties not typically enjoyed by other flat outputs. In such a trivialization, the motion planning problem is easily solved, since a given trajectory for the group variables will fully determine the trajectory for the shape variables that exactly achieves this motion. We provide a partial catalog of robotic systems with geometric flat outputs and worked examples for the planar rocket, planar aerial manipulator, and quadrotor.

This paper addresses the problem of active collaborative localization in heterogeneous robot teams with unknown data association. It involves positioning a small number of identical unmanned ground vehicles (UGVs) at desired positions so that an unmanned aerial vehicle (UAV) can, through unlabelled measurements of UGVs, uniquely determine its global pose. We model the problem as a sequential two player game, in which the first player positions the UGVs and the second identifies the two distinct hypothetical poses of the UAV at which the sets of measurements to the UGVs differ by as little as possible. We solve the underlying problem from the vantage point of the first player for a subclass of measurement models using a mixture of local optimization and exhaustive search procedures. Real-world experiments with a team of UAV and UGVs show that our method can achieve centimeter-level global localization accuracy. We also show that our method consistently outperforms random positioning of UGVs by a large margin, with as much as a 90% reduction in position and angular estimation error. Our method can tolerate a significant amount of random as well as non-stochastic measurement noise. This indicates its potential for reliable state estimation on board size, weight, and power (SWaP) constrained UAVs. This work enables robust localization in perceptually-challenged GPS-denied environments, thus paving the road for large-scale multi-robot navigation and mapping.

We consider joint trajectory generation and tracking control for under-actuated robotic systems. A common solution is to use a layered control architecture, where the top layer uses a simplified model of system dynamics for trajectory generation, and the low layer ensures approximate tracking of this trajectory via feedback control. While such layered control architectures are standard and work well in practice, selecting the simplified model used for trajectory generation typically relies on engineering intuition and experience. In this paper, we propose an alternative data-driven approach to dynamicsaware trajectory generation. We show that a suitable augmented Lagrangian reformulation of a global nonlinear optimal control problem results in a layered decomposition of the overall problem into trajectory planning and feedback control layers. Crucially, the resulting trajectory optimization is dynamicsaware, in that, it is modified with a tracking penalty regularizer encoding the dynamic feasibility of the generated trajectory. We show that this tracking penalty regularizer can be learned from system rollouts for independently-designed low layer feedback control policies, and instantiate our framework in the context of a unicycle and a quadrotor control problem in simulation. Further, we show that our approach handles the sim-to-real gap through experiments on the quadrotor hardware platform without any additional training. For both the synthetic unicycle example and the quadrotor system, our framework shows significant improvements in both computation time and dynamic feasibility in simulation and hardware experiments.

Convolutional neural network (CNN)-based object detection has achieved very high accuracy; e.g., single-shot multi-box detectors (SSDs) can efficiently detect and localize various objects in an input image. However, they require a high amount of computation and memory storage, which makes it difficult to perform efficient inference on resource-constrained hardware devices such as drones or unmanned aerial vehicles (UAVs). Drone/UAV detection is an important task for applications including surveillance, defense, and multi-drone self-localization and formation control. In this article, we designed and co-optimized an algorithm and hardware for energy-efficient drone detection on resource-constrained FPGA devices. We trained an SSD object detection algorithm with a custom drone dataset. For inference, we employed low-precision quantization and adapted the width of the SSD CNN model. To improve throughput, we use dual-data rate operations for DSPs to effectively double the throughput with limited DSP counts. For different SSD algorithm models, we analyze accuracy or mean average precision (mAP) and evaluate the corresponding FPGA hardware utilization, DRAM communication, and throughput optimization. We evaluated the FPGA hardware for a custom drone dataset, Pascal VOC, and COCO2017. Our proposed design achieves a high mAP of 88.42% on the multi-drone dataset, with a high energy efficiency of 79 GOPS/W and throughput of 158 GOPS using the Xilinx Zynq ZU3EG FPGA device on the Open Vision Computer version 3 (OVC3) platform. Our design achieves 1.1 to 8.7× higher energy efficiency than prior works that used the same Pascal VOC dataset, using the same FPGA device, but at a low-power consumption of 2.54 W. For the COCO dataset, our MobileNet-V1 implementation achieved an mAP of 16.8, and 4.9 FPS/W for energy-efficiency, which is ∼ 1.9× higher than prior FPGA works or other commercial hardware platforms.

The problem of multi-robot target tracking asks for actively planning the joint motion of robots to track targets. In this article, we focus on such target tracking problems in adversarial environments, where attacks or failures may deactivate robots' sensors and communications. In contrast to the previous works that consider no attacks or sensing attacks only, we formalize the first robust multi-robot tracking framework that accounts for any fixed numbers of worst-case sensingandcommunication attacks. To secure against such attacks, we design the first robust planning algorithm, namedRobust Active Target Tracking(RATT), which approximates the communication attacks toequivalentsensing attacks and then optimizes against the approximated and original sensing attacks. We show thatRATTprovides provable suboptimality bounds on the tracking quality for any non-decreasing objective function. Our analysis utilizes the notations of curvature for set functions introduced in combinatorial optimization. In addition,RATTruns in polynomial time and terminates with the same running time as state-of-the-art algorithms for (non-robust) target tracking. Finally, we evaluateRATTwith both the qualitative and quantitative simulations across various scenarios. In the evaluations,RATTexhibits a tracking quality that is near-optimal and superior to varying non-robust heuristics. We also demonstrateRATT’s superiority and robustness against varying attack models (e.g., worst-case and bounded rational attacks) and with over- and under-estimated numbers of attacks.

Modeling stochastic traffic dynamics is critical to developing self-driving cars. Because it is difficult to develop first principle models of cars driven by humans, there is great potential for using data driven approaches in developing traffic dynamical models. While there is extensive literature on this subject, previous works mainly address the prediction accuracy of data-driven models. Moreover, it is often difficult to apply these models to common planning frameworks since they fail to meet the assumptions therein. In this work, we propose a new stochastic traffic model, Recurrent Traffic Graph Neural Network (RTGNN), by enforcing additional structures on the model so that the proposed model can be seamlessly integrated with existing motion planning algorithms. RTGNN is a Markovian model and is able to infer future traffic states conditioned on the motion of the ego vehicle. Specifically, RTGNN uses a definition of the traffic state that includes the state of all players in a local region and is therefore able to make joint predictions for all agents of interest. Meanwhile, we explicitly model the hidden states of agents, “intentions,” as part of the traffic state to reflect the inherent partial observability of traffic dynamics. The above mentioned properties are critical for integrating RTGNN with motion planning algorithms coupling prediction and decision making. Despite the additional structures, we show that RTGNN is able to achieve state-of-the-art accuracy through comparisons with other similar works.

This paper develops a decentralized approach to mobile sensor coverage by a multi-robot system. We consider a scenario where a team of robots with limited sensing range must position itself to effectively detect events of interest in a region characterized by areas of varying importance. Towards this end, we develop a decentralized control policy for the robots-realized via a Graph Neural Network-which uses inter-robot communication to leverage non-local information for control decisions. By explicitly sharing information between multi-hop neighbors, the decentralized controller achieves a higher quality of coverage when compared to classical approaches that do not communicate and leverage only local information available to each robot. Simulated experiments demonstrate the efficacy of multi-hop communication for multi-robot coverage and evaluate the scalability and transferability of the learning-based controllers.

Fast, autonomous flight in unstructured, cluttered environments such as forests is challenging because it requires the robot to compute new plans in realtime on a computationally-constrained platform. In this paper, we enable this capability with a search-based planning framework that adapts sampling density in realtime to find dynamically-feasible plans while remaining computationally tractable. A paramount challenge in search-based planning is that dense obstacles both necessitate large graphs (to guarantee completeness) and reduce the efficiency of graph search (as heuristics become less accurate). To address this, we develop a planning framework with two parts: one that maximizes planner completeness for a given graph size, and a second that dynamically maximizes graph size subject to computational constraints. This framework is enabled by motion planning graphs that are defined by a single parameter-dispersion-which quantifies the maximum trajectory cost to reach an arbitrary state from the graph. We show through real and simulated experiments how the dispersion can be adapted to different environments in realtime, allowing operation in environments with varying density. The simulated experiment demonstrates improved performance over a baseline search-based planning algorithm. We also demonstrate flight speeds of up to 2.5m/s in real-world cluttered pine forests.

We propose a system that uses semantic object detections to localize a microgravity free-flyer. Many applications require absolute localization in a known reference frame, such as the execution of waypoint trajectories defined by human operators. Classical geometric methods build a map of point features, which may not be able to be associated after lighting or environmental changes. By contrast, semantics remain invariant to changes up to the robustness of the detection algorithm and motion of the semantic objects. In this work, we describe our approaches for both offline semantic map generation as well as online localization against a semantic map, intended to run in real-time on the robot. We additionally demonstrate how our semantic localizer outperforms image-feature matching in some cases, and show the robustness of the algorithm to environmental changes. Crucially, we show in our experiments that when semantics are used to supplement point features, localization is always improved. To our knowledge, these experiments demonstrate the first use of learned semantics for localization on a free-flying robot in microgravity.

In this paper, we present an online adaptive planning strategy for a team of robots with heterogeneous sensors to sample from a latent spatial field using a learned model for decision making. Current robotic sampling methods seek to gather information about an observable spatial field. However, many applications, such as environmental monitoring and precision agriculture, involve phenomena that are not directly observable or are costly to measure, called latent phenomena. In our approach, we seek to reason about the latent phenomenon in real-time by effectively sampling the observable spatial fields using a team of robots with heterogeneous sensors, where each robot has a distinct sensor to measure a different observable field. The information gain is estimated using a learned model that maps from the observable spatial fields to the latent phenomenon. This model captures aleatoric uncertainty in the relationship to allow for information theoretic measures. Additionally, we explicitly consider the correlations among the observable spatial fields, capturing the relationship between sensor types whose observations are not independent. We show it is possible to learn these correlations, and investigate the impact of the learned correlation models on the performance of our sampling approach. Through our qualitative and quantitative results, we illustrate that empirically learned correlations improve the overall sampling efficiency of the team. We simulate our approach using a data set of sensor measurements collected on Lac Hertel, in Quebec, which we make publicly available.

The 17 papers in this special section focus on resilience in networked robotic systems. This collection of articles aims to provide a deeper understanding of resilience as it pertains to multirobot systems, and to disseminate the current advances in designing and operating networked robotic systems. We understand resilience to be a characteristic that enables amultirobot system to withstand or overcome unexpected adverse conditions or shocks, and unknown, unmodeled disturbances. It refers to the contingent nature of the robots’ behaviors that is aimed at preserving their functionality or minimizing the time periods during which their functionality is compromised. The papers explore new algorithmic and mathematical foundations toward resilience.

Convolutional neural network (CNN) based object detection has achieved very high accuracy, e.g. single-shot multi-box detectors (SSD) can efficiently detect and localize various objects in an input image. However, they require a high amount of computation and memory storage, which makes it difficult to perform efficient inference on resource-constrained hardware devices such as drones or unmanned aerial vehicles (UAVs). Drone/UAV detection is an important task for applications including surveillance, defense, and multi-drone self-localization and formation control. In this paper, we designed and co-optimized algorithm and hardware for energy-efficient drone detection on resource-constrained FPGA devices. We trained SSD object detection algorithm with a custom drone dataset. For inference, we employed low-precision quantization and adapted the width of the SSD CNN model. To improve throughput, we use dual-data rate operations for DSPs to effectively double the throughput with limited DSP counts. For different SSD algorithm models, we analyze accuracy or mean average precision (mAP) and evaluate the corresponding FPGA hardware utilization, DRAM communication, throughput optimization. Our proposed design achieves a high mAP of 88.42% on the multi-drone dataset, with a high energy-efficiency of 79 GOPS/W and throughput of 158 GOPS using Xilinx Zynq ZU3EG FPGA device on the Open Vision Computer version 3 (OVC3) platform. Our design achieves 2.7X higher energy efficiency than prior works using the same FPGA device, at a low-power consumption of 1.98 W.

Search-based planning with motion primitives is a powerful motion planning technique that can provide dynamic feasibility, optimality, and real-time computation times on size, weight, and power-constrained platforms in unstructured environments. However, optimal design of the motion planning graph, while crucial to the performance of the planner, has not been a main focus of prior work. This paper proposes to address this by introducing a method of choosing vertices and edges in a motion primitive graph that is grounded in sampling theory and leads to theoretical guarantees on planner completeness. By minimizing dispersion of the graph vertices in the metric space induced by trajectory cost, we optimally cover the space of feasible trajectories with our motion primitive graph. In comparison with baseline motion primitives defined by uniform input space sampling, our motion primitive graphs have lower dispersion, find a plan with fewer iterations of the graph search, and have only one parameter to tune.

This paper investigates servicing waypoints in a wide area using collaborative deployments of vehicles with heterogeneous range and mobility constraints. We formulate a joint planning problem for a single transport truck and multiple service drones in which the truck is constrained to a road and must deploy a team of range-constrained drones to visit waypoints. The need to deploy, collect, and redeploy drones over multiple flights introduces both route finding and scheduling aspects to this problem. We solve large problem instances by decoupling our approach into a service drone route finding phase and a transport truck scheduling phase. Numerical simulations explore the qualitative character of the driving schedule and the quantitative marginal value of adding additional drones to the team as a function of agent number and relative speed. The combination of road network constraints and range constraints make this problem especially relevant to wide area forestry, last-mile delivery, and ecological monitoring applications.

Many algorithms for control of multi-robot teams operate under the assumption that low-latency, global state information necessary to coordinate agent actions can readily be disseminated among the team. However, in harsh environments with no existing communication infrastructure, robots must form ad-hoc networks, forcing the team to operate in a distributed fashion. To overcome this challenge, we propose a task-agnostic, decentralized, low-latency method for data distribution in ad-hoc networks using Graph Neural Networks (GNN). Our approach enables multi-agent algorithms based on global state information to function by ensuring it is available at each robot. To do this, agents glean information about the topology of the network from packet transmissions and feed it to a GNN running locally which instructs the agent when and where to transmit the latest state information. We train the distributed GNN communication policies via reinforcement learning using the average Age of Information as the reward function and show that it improves training stability compared to task-specific reward functions. Our approach performs favorably compared to industry-standard methods for data distribution such as random flooding and round robin. We also show that the trained policies generalize to larger teams of both static and mobile agents.

The multi-robot coverage problem is an essential building block for systems that perform tasks like inspection, exploration, or search and rescue. We discretize the coverage problem to induce a spatial graph of locations and represent robots as nodes in the graph. Then, we train a Graph Neural Network controller that leverages the spatial equivariance of the task to imitate an expert open-loop routing solution. This approach generalizes well to much larger maps and larger teams that are intractable for the expert. In particular, the model generalizes effectively to a simulation of ten quadrotors and dozens of buildings in an urban setting. We also demonstrate the GNN controller can surpass planning-based approaches in an exploration task.

In this article, we address the problem of stochastic motion planning under partial observability, more specifically, how to navigate a mobile robot equipped with continuous range sensors, such as LIDAR. In contrast to many existing robotic motion planning methods, we explicitly consider the uncertainty of the robot state by modeling the system as a partially observable Markov decision process (POMDP). Recent work on general purpose POMDP solvers is typically limited to discrete observation spaces, and does not readily apply to the proposed problem due to the continuous measurements from LIDAR. In this article, we build upon an existing Monte Carlo tree search method, partially observable Monte Carlo planning (POMCP), and propose a new algorithm POMCP++. Our algorithm can handle continuous observation spaces with a novel measurement selection strategy. The POMCP++ algorithm overcomes overoptimism in the value estimation of a rollout policy by removing the implicit perfect state assumption at the rollout phase. We validate POMCP++ in theory by proving it is a Monte Carlo tree search algorithm. Through comparisons with other methods that can also be applied to the proposed problem, we show that POMCP++ yields significantly higher success rate and total reward.

Robot networks are susceptible to fail under the presence of malicious or defective robots. Resilient networks in the literature require high connectivity and large communication ranges, leading to high energy consumption in the communication network. This paper presents robot formations with guaranteed resiliency that use smaller communication ranges than previous results in the literature. The formations can be built on triangular and square lattices in the plane, and cubic lattices in the three-dimensional space. We support our theoretical framework with simulations.

This paper proposes a new reactive mission planning algorithm for multiple robots that operate in unknown environments. The robots are equipped with individual sensors that allow them to collectively learn and continuously update a map of the unknown environment. The goal of the robots is to accomplish complex tasks, captured by global co-safe Linear Temporal Logic (LTL) formulas. The majority of existing temporal logic planning approaches rely on discrete abstractions of the robot dynamics operating in known environments and, as a result, they cannot be applied to the more realistic scenarios where the environment is initially unknown. In this paper, we address this novel challenge by proposing the first reactive, and abstraction-free LTL planning algorithm that can be applied for complex mission planning of multiple robots operating in unknown environments. Our algorithm is reactive in the sense that temporal logic planning is adapting to the updated map of the environment and abstraction-free as it does not rely on designing abstractions of robot dynamics. Our proposed algorithm is complete under mild assumptions on the structure of the environment and the sensor models. Our paper provides extensive numerical simulations and hardware experiments that illustrate the theoretical analysis and show that the proposed algorithm can address complex planning tasks in unknown environments.

In this work, we introduce Mobile Wireless Infrastructure on Demand: a framework for providing wireless connectivity to multi-robot teams via autonomously reconfiguring ad-hoc networks. In many cases, previous multi-agent systems either assumed the availability of existing communication infrastructure or were required to create a network in addition to completing their objective. Instead our system explicitly assumes the responsibility of creating and sustaining a wireless network capable of satisfying end-to-end communication requirements of a team of agents, called the task team, performing an arbitrary objective. To accomplish this goal, we propose a joint optimization framework that alternates between finding optimal network routes to support data flows between the task agents and improving the performance of the network by repositioning a collection of mobile relay nodes referred to as the network team. We demonstrate our approach with simulations and experiments wherein wireless connectivity is provided to patrolling task agents.

Autonomous flight through unknown environments in the presence of obstacles is a challenging problem for micro aerial vehicles (MAVs). A majority of the current state-of-art research assumes obstacles as opaque objects that can be easily sensed by optical sensors such as cameras or LiDARs. However in indoor environments with glass walls and windows, or scenarios with smoke and dust, robots (even birds) have a difficult time navigating through the unknown space.In this paper, we present the design of a new class of micro aerial vehicles that achieves autonomous navigation and are robust to collisions. In particular, we present the Tiercel MAV: a small, agile, light weight and collision-resilient robot powered by a cellphone grade CPU. Our design exploits contact to infer the presence of transparent or reflective obstacles like glass walls, integrating touch with visual perception for SLAM. The Tiercel is able to localize using visual-inertial odometry (VIO) running on board the robot with a single downward facing fisheye camera and an IMU. We show how our collision detector design and experimental set up enable us to characterize the impact of collisions on VIO. We further develop a planning strategy to enable the Tiercel to fly autonomously in an unknown space, sustaining collisions and creating a 2D map of the environment. Finally we demonstrate a swarm of three autonomous Tiercel robots safely navigating and colliding through an obstacle field to reach their objectives.

We address the localization of robots in a multi-MAV system where external infrastructure like GPS or motion capture systems may not be available. Our approach lends itself to implementation on platforms with several constraints on size, weight, and power (SWaP). Particularly, our framework fuses the onboard VIO with the anonymous, visual-based robot-to-robot detection to estimate all robot poses in one common frame, addressing three main challenges: 1) the initial configuration of the robot team is unknown, 2) the data association between each vision-based detection and robot targets is unknown, and 3) the vision-based detection yields false negatives, false positives, inaccurate, and provides noisy bearing, distance measurements of other robots. Our approach extends the Coupled Probabilistic Data Association Filter [1] to cope with nonlinear measurements. We demonstrate the superior performance of our approach over a simple VIO-based method in a simulation with the measurement models statistically modeled using the real experimental data. We also show how onboard sensing, estimation, and control can be used for formation flight.

This paper focuses on exploration and occupancy mapping of unknown environments using a mobile robot. While a truncated signed distance field (TSDF) is a popular, efficient, and highly accurate representation of occupancy, few works have considered optimizing robot sensing trajectories for autonomous TSDF mapping. We propose an efficient approach for maintaining TSDF uncertainty and predicting its evolution from potential future sensor measurements without actually receiving them. Efficient uncertainty prediction is critical for long-horizon optimization of potential sensing trajectories. We develop a deterministic tree-search algorithm that evaluates the information gain between the TSDF distribution and potential observations along sequences of robot motion primitives. Efficient planning is achieved by branch-and-bound pruning of uninformative sensing trajectories. The effectiveness of our active TSDF mapping approach is evaluated in several simulated environments with complex visibility constraints.

In this work we propose long wave infrared (LWIR) imagery as a viable supporting modality for semantic segmentation using learning-based techniques. We first address the problem of RGB-thermal camera calibration by proposing a passive calibration target and procedure that is both portable and easy to use. Second, we present PST900, a dataset of 894 synchronized and calibrated RGB and Thermal image pairs with per pixel human annotations across four distinct classes from the DARPA Subterranean Challenge. Lastly, we propose a CNN architecture for fast semantic segmentation that combines both RGB and Thermal imagery in a way that leverages RGB imagery independently. We compare our method against the state-of-the-art and show that our method outperforms them in our dataset.

In this work, we derive a coordinate-free formulation of the coupled dynamics of a class of 6DOF aerial manipulators consisting of an underactuated quadrotor equipped with a 2DOF articulated manipulator, and demonstrate that the system is differentially flat with respect to the end effector pose. In particular, we require the center of mass of the entire system to be fixed in the end effector frame, suggesting a reasonable mechanical design criterion. We make use of an inertial decoupling transformation to demonstrate differential flatness, allowing us to plan dynamically feasible trajectories for the system in the space of the 6DOF pose of the end effector, which is ideal for achieving precise manipulator tasks. Simulation results validate the flatness-based planning methodology for our dynamic model, and its usefulness is demonstrated in a simulated aerial videography task.

We present an algorithm for the concurrent assignment and planning of collision-free trajectories (DC-CAPT) for robots whose kinematics can be modeled as Dubins cars, i.e., robots constrained in terms of their initial orientation and their minimum turning radius. Coupling the assignment and trajectory planning subproblems allows for a computationally tractable solution. This solution is guaranteed to be collision- free through the use of a single constraint: the start and goal locations have separation distance greater than some threshold. We derive this separation distance by extending a prior work that assumed holonomic robots. We demonstrate the validity of our approach, and show its efficacy through simulations and experiments where groups of robots executing Dubins curves travel to their assigned goal locations without collisions.

In this work, we address the motion planning problem for autonomous vehicles through a new lattice planning approach, called Feedback Enhanced Lattice Planner (FELP). Existing lattice planners have two major limitations, namely the high dimensionality of the lattice and the lack of modeling of agent vehicle behaviors. We propose to apply the Intelligent Driver Model (IDM) [1] as a speed feedback policy to address both of these limitations. IDM both enables the responsive behavior of the agents, and uniquely determines the acceleration and speed profile of the ego vehicle on a given path. Therefore, only a spatial lattice is needed, while discretization of higher order dimensions is no longer required. Additionally, we propose a directed-graph map representation to support the implementation and execution of lattice planners. The map can reflect local geometric structure, embed the traffic rules adhering to the road, and is efficient to construct and update. We show that FELP is more efficient compared to other existing lattice planners through runtime complexity analysis, and we propose two variants of FELP to further reduce the complexity to polynomial time. We demonstrate the improvement by comparing FELP with an existing spatiotemporal lattice planner using simulations of a merging scenario and continuous highway traffic. We also study the performance of FELP under different traffic densities.

Traditionally, controllers and state estimators in robotic systems are designed independently. Controllers are often designed assuming perfect state estimation. However, state estimation methods such as Visual Inertial Odometry (VIO) drift over time and can cause the system to misbehave. While state estimation error can be corrected with the aid of GPS or motion capture, these complementary sensors are not always available or reliable. Recent work has shown that this issue can be dealt with by synthesizing robust controllers using a data-driven characterization of the perception error, and can bound the system's response to state estimation error using a robustness constraint. We investigate the application of this robust perception-based approach to a quadrotor model using VIO for state estimation and demonstrate the benefits and drawbacks of using this technique in simulation and hardware. Additionally, to make tuning easier, we introduce a new cost function to use in the control synthesis which allows one to take an existing controller and "robustify" it. To the best of our knowledge, this is the first robust perception-based controller implemented in real hardware, as well as one utilizing a data-driven perception model. We believe this as an important step towards safe, robust robots that explicitly account for the inherent dependence between perception and control.

Multi-Robot Systems have been recently employed in different applications and have advantages over single-robot systems, such as increased robustness and task performance efficiency. We consider such assemblies specifically in the scenario of perimeter defense, where the task is to defend a circular perimeter by intercepting radially approaching targets. Possible intruders appear randomly at a fixed distance from the perimeter and with azimuthal location determined by some unknown probability density. Coordination among multiple defenders is a complex combinatorial optimization problem. In this work, we focus on the following two aspects: (i) estimating the probability density that describes the direction from which the next intruders are going to arrive, and (ii) partitioning of the space so that the defenders focus on capturing a disjoint subset of intruders. Results show that the proposed strategy increases the number of captures over a naive baseline strategy, especially in scenarios with non-uniform spatial distributions of intruder arrival. The proposed approach is also efficient and able to quickly adapt to time-varying intruder distributions.

We study strategies to deploy defenders/sensors to detect intruders that approach a targeted region. This scenario is formulated as a barrier coverage, which aims to minimize the number of unseen paths. The problem becomes challenging when the number of defenders is insufficient for a full coverage, requiring us to find the most effective location to deploy them. To this end, we use ideas from game theory to account for various paths that the intruders may take. Specifically, we propose an iterative algorithm to refine the set of candidate defender formations, which uses the payoff matrix to directly evaluate the utility of different formations. Given the set of candidate formations, a mixed Nash equilibrium gives a stochastic policy to deploy the defenders. The efficacy of the proposed strategy is demonstrated by a numerical analysis that compares our method with an existing graph-theoretic method.

We study the problem of inferring communication structures that can solve cooperative multi-agent planning problems while minimizing the amount of communication. We quantify the amount of communication as the maximum degree of the communication graph; this metric captures settings where agents have limited bandwidth. Minimizing communication is challenging due to the combinatorial nature of both the decision space and the objective; for instance, we cannot solve this problem by training neural networks using gradient descent. We propose a novel algorithm that synthesizes a control policy that combines a programmatic communication policy used to generate the communication graph with a transformer policy network used to choose actions. Our algorithm first trains the transformer policy, which implicitly generates a "soft" communication graph; then, it synthesizes a programmatic communication policy that "hardens" this graph, forming a neurosymbolic transformer. Our experiments demonstrate how our approach can synthesize policies that generate low-degree communication graphs while maintaining near-optimal performance.

Flying modular robots have the potential to rapidly form temporary structures. In the literature, docking actions rely on external systems and indoor infrastructures for relative pose estimation. In contrast to related work, we provide local estimation during the self-assembly process to avoid dependency on external systems. In this paper, we introduce ModQuad-Vi, a flying modular robot that is aimed to operate in outdoor environments. We propose a new robot design and vision-based docking method. Our design is based on a quadrotor platform with onboard computation and visual perception. Our control method is able to accurately align modules for docking actions. Additionally, we present the dynamics and a geometric controller for the aerial modular system. Experiments validate the vision-based docking method with successful results.

Effective communication is required for teams of robots to solve sophisticated collaborative tasks. In practice it is typical for both the encoding and semantics of communication to be manually defined by an expert; this is true regardless of whether the behaviors themselves are bespoke, optimization based, or learned. We present an agent architecture and training methodology using neural networks to learn task-oriented communication semantics based on the example of a communication-unaware expert policy. A perimeter defense game illustrates the system's ability to handle dynamically changing numbers of agents and its graceful degradation in performance as communication constraints are tightened or the expert's observability assumptions are broken.

In this paper we describe the Open Vision Computer (OVC) which was designed to support high speed, vision guided autonomous drone flight. In particular our aim was to develop a system that would be suitable for relatively small-scale flying platforms where size, weight, power consumption and computational performance were all important considerations. This manuscript describes the primary features of our OVC system and explains how they are used to support fully autonomous indoor and outdoor exploration and navigation operations on our Falcon 250 quadrotor platform.

We present an approach to depth estimation that fuses information from a stereo pair with sparse range measurements derived from a LIDAR sensor or a range camera. The goal of this work is to exploit the complementary strengths of the two sensor modalities, the accurate but sparse range measurements and the ambiguous but dense stereo information. These two sources are effectively and efficiently fused by combining ideas from anisotropic diffusion and semi-global matching.We evaluate our approach on the KITTI 2015 and Middlebury 2014 datasets, using randomly sampled ground truth range measurements as our sparse depth input. We achieve significant performance improvements with a small fraction of range measurements on both datasets. We also provide qualitative results from our platform using the PMDTec Monstar sensor. Our entire pipeline runs on an NVIDIA TX-2 platform at 5Hz on 1280×1024 stereo images with 128 disparity levels.

Accurate knowledge of geometric and inertia parameters are a necessity for precise and robust control of aerial vehicles. We propose a novel filter that is able to fuse motor speed, inertia, and pose measurements to estimate the vehicle's key dynamic properties online. The presented framework is able to estimate the multirotor's moment of inertia, mass, center of mass and each sensor module's relative position. Obtaining these estimates in-flight allow the multirotor to be precisely controlled even during tasks such as load transportation or after configuration changes on scene. We provide a nonlinear observability analysis, proving that the presented model is locally weakly observable. Experimental results validate the proposed approach, showing the ability to estimate the dynamic properties accurately and demonstrate its capability to do so even while additional loads are added. The framework is flexible and can easily be adapted to a wide range of applications, including self-calibration, object grasping, and single robot or multi-robot payload transportation.

In this paper, we present a learning approach to goal assignment and trajectory planning for unlabeled robots operating in 2D, obstacle-filled workspaces. More specifically, we tackle the unlabeled multi-robot motion planning problem with motion constraints as a multi-agent reinforcement learning problem with some sparse global reward. In contrast with previous works, which formulate an entirely new hand-crafted optimization cost or trajectory generation algorithm for a different robot dynamic model, our framework is a general approach that is applicable to arbitrary robot models. Further, by using the velocity obstacle, we devise a smooth projection that guarantees collision free trajectories for all robots with respect to their neighbors and obstacles. The efficacy of our algorithm is demonstrated through varied simulations. A video describing our method and results can be found here.

We envision a system that concisely describes the rules of air traffic control, assists human operators and supports dense autonomous air traffic around commercial airports. We develop a method to learn the rules of air traffic control from real data as a cost function via maximum entropy inverse reinforcement learning. This cost function is used as a penalty for a search-based motion planning method that discretizes both the control and the state space. We illustrate the methodology by showing that our approach can learn to imitate the airport arrival routes and separation rules of dense commercial air traffic. The resulting trajectories are shown to be safe, feasible, and efficient.

We present a novel flying modular platform capable of grasping and transporting objects. It is composed of four cooperative identical modules where each is based on a quadrotor within a cuboid frame with a docking mechanism. Pairs of modules are able to fly independently and physically connect by matching their vertical edges forming a hinge. Four one degree of freedom (DOF) connections results in a one DOF four-bar linkage that can be used to grasp external objects. In this paper, we propose a decentralized method that allows the Flying Gripper to control its position, attitude and aperture angle. In our experiments, we tested the hovering performance for different aperture angles and with a grasped object. The performance for a closing and opening motion was also verified.

Micro Aerial Vehicles have the potential to assist humans in real life tasks involving applications such as smart homes, search and rescue, and architecture construction. To enhance autonomous navigation capabilities these vehicles need to be able to create dense 3D maps of the environment, while concurrently estimating their own motion. In this paper, we are particularly interested in small vehicles that can navigate cluttered indoor environments. We address the problem of visual inertial state estimation, control and 3D mapping on platforms with Size, Weight, And Power (SWAP) constraints. The proposed approach is validated through experimental results on a 250 g, 22 cm diameter quadrotor equipped only with a stereo camera and an IMU with a computationally-limited CPU showing the ability to autonomously navigate, while concurrently creating a 3D map of the environment.

High speed navigation through unknown environments is a challenging problem in robotics. It requires fast computation and tight integration of all the subsystems on the robot such that the latency in the perception-action loop is as small as possible. Aerial robots add a limitation of payload capacity, which restricts the amount of computation that can be carried onboard. This requires efficient algorithms for each component in the navigation system. In this paper, we describe our quadrotor system which is able to smoothly navigate through mixed indoor and outdoor environments and is able to fly at speeds of more than 18 m/s. We provide an overview of our system and details about the specific component technologies that enable the high speed navigation capability of our platform. We demonstrate the robustness of our system through high speed autonomous flights and navigation through a variety of obstacle rich environments.

We introduce ModQuad, a novel flying modular robotic structure that is able to self-assemble in midair and cooperatively fly. The structure is composed by agile flying modules that can easily move in a three dimensional environment. The module is based on a quadrotor platform within a cuboid frame which allows it to attach to other modules by matching vertical faces. Using this mechanism, a ModQuad swarm is able to rapidly assemble flying structures in midair using the robot bodies as building units. In this paper, we focus on two important tasks for modular flying structures. First, we propose a decentralized modular attitude controller to allow a team of physically connected modules to fly cooperatively. Second, we develop a docking method that drives pairs of structures to be attached in midair. Our method precisely aligns, and corrects motion errors during the docking process. In our experiments, we tested and analyzed the performance of the cooperative flying method for multiple configurations. We also tested the docking method with successful results.

Current motion capture (MoCap) systems generally require markers and multiple calibrated cameras, which can be used only in constrained environments. In this work we introduce a drone-based system for 3D human MoCap. The system only needs an autonomously flying drone with an on-board RGB camera and is usable in various indoor and outdoor environments. A reconstruction algorithm is developed to recover full-body motion from the video recorded by the drone. We argue that, besides the capability of tracking a moving subject, a flying drone also provides fast varying viewpoints, which is beneficial for motion reconstruction. We evaluate the accuracy of the proposed system using our new DroCap dataset and also demonstrate its applicability for MoCap in the wild using a consumer drone.

We propose a novel approach for optimal trajectory tracking for unmanned aerial vehicles (UAV), using a linear model predictive controller (MPC) in combination with non-linear state feedback. The solution relies on fast onboard simulation of the translational dynamics of the UAV, which is guided by a linear MPC. By sampling the states of the virtual UAV, we create a control command for fast non-linear feedback, which is capable of performing agile maneuvers with high precision. In addition, the proposed pipeline provides an interface for a decentralized collision avoidance system for multi-UAY scenarios. Our solution makes use of the long prediction horizon of the linear MPC and allows safe outdoors execution of multi-UAV experiments without the need for in-advance collision-free planning. The practicality of the tracking mechanism is shown in combination with priority-based collision resolution strategy, which performs sufficiently in experiments with up to 5 UAVs. We present a statistical and experimental evaluation of the platform in both simulation and real-world examples, demonstrating the usability of the approach.

In this paper, we propose a novel architecture and a self-supervised policy gradient algorithm, which employs unsupervised auxiliary tasks to enable a mobile robot to learn how to navigate to a given goal. The dependency on the global information is eliminated by providing only sparse range-finder measurements to the robot. The partially observable planning problem is addressed by splitting it into a hierarchical process. We use convolutional networks to plan locally, and a differentiable memory to provide information about past time steps in the trajectory. These modules, combined in our network architecture, produce globally consistent plans. The sparse reward problem is mitigated by our modified policy gradient algorithm. We model the robots uncertainty with unsupervised tasks to force exploration. The novel architecture we propose with the modified version of the policy gradient algorithm allows our robot to reach the goal in a sample efficient manner, which is orders of magnitude faster than the current state of the art policy gradient algorithm. Simulation and experimental results are provided to validate the proposed approach.

We present a novel fruit counting pipeline that combines deep segmentation, frame to frame tracking, and 3D localization to accurately count visible fruits across a sequence of images. Our pipeline works on image streams from a monocular camera, both in natural light, as well as with controlled illumination at night. We first train a Fully Convolutional Network (FCN) and segment video frame images into fruit and non-fruit pixels. We then track fruits across frames using the Hungarian Algorithm where the objective cost is determined from a Kalman Filter corrected Kanade-Lucas-Tomasi (KLT) Tracker. In order to correct the estimated count from tracking process, we combine tracking results with a Structure from Motion (SfM) algorithm to calculate relative 3D locations and size estimates to reject outliers and double counted fruit tracks. We evaluate our algorithm by comparing with ground-truth human-annotated visual counts. Our results demonstrate that our pipeline is able to accurately and reliably count fruits across image sequences, and the correction step can significantly improve the counting accuracy and robustness. Although discussed in the context of fruit counting, our work can extend to detection, tracking, and counting of a variety of other stationary features of interest such as leaf-spots, wilt, and blossom.

This work addresses the design and implementation of a filter that estimates the orientation of the body-fixed z axis and the velocity of a quadrotor UAV from the inertial measurement unit (IMU) given a known yaw. The velocity and attitude estimation is possible since the filter employs a linear drag model measuring the drag forces on the quadrotor through the IMU. These forces are functions of the robot's velocity and attitude. In addition, the filter estimates the linear drag parameters and thrust coefficient for the propellers. These parameters may be fed back into a controller to improve tracking performance. Experimental results are used to validate the proposed approach.

The papers in this special section present recent advances in aerial swarm robotics, and aims to put together a cohesive set of research goals and visions toward realizing fully autonomous aerial swarm systems. One objective is to emphasize the three-way tradeoff among computational efficiency for large-scale swarms, stability, and robustness under uncertainty, and the optimal system performance. Aerial robotics has been one of the most active areas of research within the robotics community, and recently there have been many reports of promising results in aerial swarm systems. This is partly due to the commoditization of multicopter platforms, and communication, sensing, and processing hardware that has substantially lowered the barriers to entry to the field of aerial swarm robotics. Aerial swarms differ from swarms of ground-based vehicles in two major respects: Aerial robots or unmanned aerial vehicles (UAVs) operate in a three-dimensional space, and the dynamics of individual vehicles add an extra layer of complexity to the problems of path planning and trajectory design. Furthermore, the success of aerial swarms is predicated on the distributed and synergistic capabilities of individual and cooperative control, estimation, and decision making of aerial robots with limited resources, such as modest onboard computation and sensing capabilities and size, weight, and power constraints.

The use of aerial swarms to solve real-world problems has been increasing steadily, accompanied by falling prices and improving performance of communication, sensing, and processing hardware. The commoditization of hardware has reduced unit costs, thereby lowering the barriers to entry to the field of aerial swarm robotics. A key enabling technology for swarms is the family of algorithms that allow the individual members of the swarm to communicate and allocate tasks amongst themselves, plan their trajectories, and coordinate their flight in such a way that the overall objectives of the swarm are achieved efficiently. These algorithms, often organized in a hierarchical fashion, endow the swarm with autonomy at every level, and the role of a human operator can be reduced, in principle, to interactions at a higher level without direct intervention. This technology depends on the clever and innovative application of theoretical tools from control and estimation. This paper reviews the state of the art of these theoretical tools, specifically focusing on how they have been developed for, and applied to, aerial swarms. Aerial swarms differ from swarms of ground-based vehicles in two respects: they operate in a three-dimensional space and the dynamics of individual vehicles adds an extra layer of complexity. We review dynamic modeling and conditions for stability and controllability that are essential in order to achieve cooperative flight and distributed sensing. The main sections of this paper focus on major results covering trajectory generation, task allocation, adversarial control, distributed sensing, monitoring, and mapping. Wherever possible, we indicate how the physics and subsystem technologies of aerial robots are brought to bear on these individual areas.

We present a method for calibration-free, infrastructure-free localization in sensor networks. Our strategy is to estimate node positions and noise distributions of all links in the network simultaneously - a strategy that has not been attempted thus far. In particular, we account for biased, non-line-of-sight (NLOS) range measurements from ultra-wideband (UWB) devices that lead to multi-modal noise distributions, for which few solutions exist to date. Our approach circumvents cumbersome a-priori calibration, allows for rapid deployment in unknown environments, and facilitates adaptation to changing conditions. Our first contribution is a generalization of the classical multidimensional scaling algorithm to account for measurements that have multi-modal error distributions. Our second contribution is an online approach that iterates between node localization and noise parameter estimation. We validate our method in 3-dimensional networks, (i) through simulation to test the sensitivity of the algorithm on its design parameters, and (ii) through physical experimentation in a NLOS environment. Our setup uses UWB devices that provide time-of-flight measurements, which can lead to positively biased distance measurements in NLOS conditions. We show that our algorithm converges to accurate position estimates, even when initial position estimates are very uncertain, initial error models are unknown, and a significant proportion of the network links are in NLOS.

Robotic pouring is an important step in improving the safety, productivity and repeatability in the biotechnology industry and generally increasing the effectiveness of robotics in human based environments. In this work we present a method to autonomously dispense a precise amount of fluid using only visual feedback without using precision pouring instruments such as pipettes, syringes or pourers. We model circular and rectangular pouring container geometries. We prove that for square containers we can control the flow by only observing the fluid height in the receiving beaker. We show a systematic approach using a hybrid control scheme that is robust to the initial amount of fluid in the pouring container and inconsistent flow. Specifically we present (a) a model for pouring (b) a model based algorithm to drive a robot arm (c) visual feedback for regulating the pouring rate. We demonstrate this using the Rethink Robotics Sawyer manipulator and mvBluefox MLC202bc camera.

In this work, we propose a search-based planning method to compute dynamically feasible trajectories for a quadrotor flying in an obstacle-cluttered environment. Our approach searches for smooth, minimum-time trajectories by exploring the map using a set of short-duration motion primitives. The primitives are generated by solving an optimal control problem and induce a finite lattice discretization on the state space which can be explored using a graph-search algorithm. The proposed approach is able to generate resolution-complete (i.e., optimal in the discretized space), safe, dynamically feasibility trajectories efficiently by exploiting the explicit solution of a Linear Quadratic Minimum Time problem. It does not assume a hovering initial condition and, hence, is suitable for fast online re-planning while the robot is moving. Quadrotor navigation with online re-planning is demonstrated using the proposed approach in simulation and physical experiments and comparisons with trajectory generation based on state-of-art quadratic programming are presented.

We present a novel framework for integrity analysis of swarm robotic systems using the symmetric Kullback-Leibler Divergence. The objective is to understand a robot swarm's vulnerability to malicious intrusion and to develop the necessary computational tools that would detect the presence of malicious agents within the swarm. Using ensemble approaches for modeling and analyzing stochastic task allocation, we analyze the performance of the proposed strategy subject to different system parameters, and show how different design choices can facilitate early intrusion detection. We further evaluate the performance of our method in realistic scenarios through stochastic simulations for different team sizes. The main contribution is an analysis framework whose output can be used to avoid system-inherent design flaws and to decrease the damage that can be inflicted by an undetected attacker.

Urban transportation is being transformed by mobility-on-demand (MoD) systems. One of the goals of MoD systems is to provide personalized transportation services to passengers. This process is facilitated by a centralized operator that coordinates the assignment of vehicles to individual passengers, based on location data. However, current approaches assume that accurate positioning information for passengers and vehicles is readily available. This assumption raises privacy concerns. In this work, we address this issue by proposing a method that protects passengers' drop-off locations (i.e., their travel destinations). Formally, we solve a batch assignment problem that routes vehicles at obfuscated origin locations to passenger locations (since origin locations correspond to previous drop-off locations), such that the mean waiting time is minimized. Our main contributions are two-fold. First, we formalize the notion of privacy for continuous vehicle-to-passenger assignment in MoD systems, and integrate a privacy mechanism that provides formal guarantees. Second, we present a polynomial-time iterative version of the Hungarian algorithm to allocate a redundant number of vehicles to a single passenger. This algorithm builds on the insight that even during peak rush hour there are unoccupied (redundant) traveling vehicles. This strategy allows us to reduce the performance deterioration induced by the privacy mechanism. In particular, it enables the exploration of the trade-off between privacy levels, waiting time, and deployed fleet size. We evaluate our methods on a real, large-scale data set consisting of over 11 million taxi rides (specifying vehicle availability and passenger requests), recorded over a month's duration, in the area of Manhattan, New York. Based on current traffic statistics, our evaluations indicate that privacy can be achieved without incurring a significant loss of performance, and that this loss can be further controlled by varying operator or user preferences.

Recent work in the field of bio-inspired robotic systems has introduced designs for modular robots that are able to assemble into structures (e.g., bridges, landing platforms, fences) using their bodies as the building components. Yet, it remains an open question as to how to program large swarms of robotic modules so that the assembly task is performed as efficiently as possible. Moreover, the problem of designing assembly algorithms is compounded by the scale of these systems, and by the lack of centralized guidance in unstructured environments. The main contribution of this work is a decentralized algorithm to assemble structures with modular robots. Importantly, we coordinate the robots so that docking actions can be parallelized. We show the correctness of our algorithm, and we demonstrate its scalability and generality through multiple scenarios in simulation. Experiments on physical robots demonstrate the validity of our approach in real-world settings.

This paper presents an automated robotic system for generating semantic maps of inventory in retail environments. In retail settings, semantic maps are labeled maps of stores where each discrete section of shelving is assigned a department label describing the types of products on that shelf. Starting from a metric map of the store, the robot autonomously extracts the shelf boundaries, generates a distance-optimal tour of the store to view every shelf, and follows the tour while avoiding unmapped clutter and moving people. The robot creates a point cloud of the store using the data collected from this tour. We introduce a novel soft-object assignment algorithm to create a virtual map and a dynamic programming algorithm to segment this map. These algorithms use a priori information about the products to boost data from laser and camera sensors in order to recognize and semantically label objects. The primary contribution of this paper is the integration of multiple systems for automated path planning, navigation, object recognition, and semantic mapping. This paper represents an important contribution toward deploying mobile robots in dynamic human environments.

Algorithms for fixed-wing unmanned aerial systems (UAS) must integrate on-board sensor capabilities and vehicle maneuver constraints to reliably satisfy the objectives of persistent surveillance, path planning, and trajectory management. In many cases, the characteristic dimensions of sensor fields of view are comparable with the turning radius of the UAS platform. Consequently, when persistent, full area, time-critical coverage is required and the number of assets is limited, the complexity of path planning is increased, as the turn radius becomes comparable with or exceeds the sensor footprint. A technique is developed to integrate persistent surveillance mission requirements with sensor resolution and field-of-view to facilitate efficient path planning. Graph search techniques and spline-based methods are combined to develop computationally simple algorithms that converge to feasible paths with $G^{2}$ continuity. ( $G^{2}$ continuity in this context means the path, which is constructed from a concatenated series of $C^{2}$ curve segments, and its derivatives are continuous through two derivatives, but the magnitudes of second derivatives may differ at the boundaries, where these curve segments are joined to form the entire path. Curvature is continuous throughout.) Two example cases are provided. In the first, the turning radius is small in comparison with the sensor footprint, while in the second, the minimum turn radius is a critical parameter in the determination of a feasible path.

We consider the problem of distributing a large group of heterogeneous robots among a set of tasks that require specialized capabilities in order to be completed. We model the system of heterogeneous robots as a community of species, in which each species (robot type) is defined by the traits (capabilities) that it owns. In order to solve the distribution problem, we develop centralized as well as decentralized methods to efficiently control the heterogeneous swarm of robots. Our methods assume knowledge of the underlying task topology and are based on a continuous model of the system that defines transition rates to and from tasks, for each robot species. Our optimization of the transition rates is fully scalable with respect to the number of robots, number of species, and number of traits. Building on this result, we propose a real-time optimization method that enables an online adaptation of transition rates as a function of the state of the current robot distribution. We also show how the robot distribution can be approximated based on local information only, consequently enabling the development of a decentralized controller. We evaluate our methods by means of microscopic simulations and show how the performance of the latter is well predicted by the macroscopic equations. Importantly, our framework also includes a diversity metric that enables an evaluation of the impact of swarm heterogeneity on performance. The metric defines the notion of minspecies, i.e., the minimum set of species that are required to achieve a given goal. We show that two distinct goal functions lead to two specializations of minspecies, which we term as eigenspecies and coverspecies. Quantitative results show the relation between diversity and performance.

We develop a hybrid system architecture that enables a team of mobile robots to complete a task in a complex environment by self-organizing into a multihop ad hoc network and solving the concurrent communication and mobility problem. The proposed system consists of a two-layer feedback loop. An outer loop performs infrequent global coordination and a local inner loop determines motion and communication variables. This system provides the lightweight coordination and responsiveness of decentralized systems while avoiding local minima. This allows a team to complete a task in complex environments while maintaining desired end-to-end data rates. The behavior of the system is evaluated in experiments that demonstrate: 1) successful task completion in complex environments; 2) achievement of equal or greater end-to-end data rates as compared to a centralized system; and 3) robustness to unexpected events such as motion restriction.

We present a novel approach for solving pursuit-evasion problems where multiple pursuers with limited sensing capabilities are used to detect all possible mobile evaders in a given environment. We make no assumptions about the number, the speed, or the maneuverability of evaders. Our algorithm takes as input a map of the environment and sensor models for the pursuers. We then obtain a graph representation of an environment using a Čech Complex. Even with such a representation, the configuration space grows exponentially with the number of pursuers. In order to address this challenge, we propose an abstraction framework to partition the configuration space into sets of topologically similar configurations that preserve the space of possible evader locations. We validate our approach on several simulated environments with varying topologies and numbers of pursuers.

The so-called direct visual SLAM methods have shown a great potential in estimating a semidense or fully dense reconstruction of the scene, in contrast to the sparse reconstructions of the traditional feature-based algorithms. In this paper, we propose for the first time a direct, tightly-coupled formulation for the combination of visual and inertial data. Our algorithm runs in real-time on a standard CPU. The processing is split in three threads. The first thread runs at frame rate and estimates the camera motion by a joint non-linear optimization from visual and inertial data given a semidense map. The second one creates a semidense map of high-gradient areas only for camera tracking purposes. Finally, the third thread estimates a fully dense reconstruction of the scene at a lower frame rate. We have evaluated our algorithm in several real sequences with ground truth trajectory data, showing a state-of-the-art performance.

In this work, we explore a unique approach to harvest energy from active matter in order to independently power and control many untethered, mobile microrobots. More specifically, we investigate microstructure transport caused by stochastic collisions between freely swimming cells and rigid microstructure boundaries in a dense bacterial bath. We characterize the motion of asymmetrically shaped microstructures, specifically gears and chevrons, and demonstrate control of these microrobots by altering the motility of swimming cells with high intensity blue light. We demonstrate both rotational and translational motion control using a custom system for tracking and real-time automated exposure of local bath regions using visual feedback. These experiments are supported by a mathematical model and simulations which describe the dynamics of a microstructure propelled by many stochastic cell structure collisions. This model allows us to predict microrobot trajectories and to develop improved strategies for microrobot design and control.

This paper addresses the challenge of versatile aerial grasping utilizing suction while considering the limitations of an on-board vacuum pump. It builds upon our patented self-sealing suction cup technology, which allows the exertion of local pulling contact forces for grasping a wide range of objects. The novel self-sealing nature of the cups enables the gripper to be versatile, employing just one, several, or all of the cups for the grasp in a passively actuated manner. We begin by describing the design of the system and its components. Because aerial applications are typically sensitive to weight constraints, we used a micro-pump vacuum generator, which introduced new challenges for our system. To investigate and overcome those challenges, we tested the relationship between the cup's design and its leakage, activation force, and maximum holding force. In addition, we tested the performance of the individual gripper components, the aerial vehicle's ability to transfer force to the cups, the system's ability to grip inclined surfaces, and finally the vehicle's ability to grasp a multitude of objects using various numbers of cups. This included the grasping of one object, followed by the grasping of a second object while still holding the first object.

We address the problem of high speed autonomous navigation of quadrotor micro aerial vehicles with limited onboard sensing and computation. In particular, we propose a dual range planning horizon method to safely and quickly navigate quadrotors to specified goal locations in previously unknown and unstructured environments. In each planning epoch, a short-range planner uses a local map to generate a new trajectory. At the same time, a safe stopping policy is found. This allows the robot to come to an emergency halt when necessary. Our algorithm guarantees collision avoidance and demonstrates important advances in real-time planning. First, our novel short range planning method allows us to generate and re-plan trajectories that are dynamically feasible, comply with state and input constraints, and avoid obstacles in real-time. Further, previous planning algorithms abstract away the obstacle detection problem by assuming the instantaneous availability of geometric information about the environment. In contrast, our method addresses the challenge of using the raw sensor data to form a map and navigate in real-time. Finally, in addition to simulation examples, we provide physical experiments that demonstrate the entire algorithmic pipeline from obstacle detection to trajectory execution.

The combination of on-board sensors measurements with different statistical characteristics can be employed in robotics for localization and control, especially in GPS-denied environments. In particular, most aerial vehicles are packaged with low cost sensors, important for aerial robotics, such as camera, a gyroscope, and an accelerometer. In this work, we develop a visual inertial odometry system based on the Unscented Kalman Filter (UKF) acting on the Lie group SE(3), such to obtain an unique, singularity-free representation of a rigid body pose. We model this pose with the Lie group SE(3) and model the noise on the corresponding Lie algebra. Moreover, we extend the concepts used in the standard UKF formulation, such as state uncertainty and modeling, to correctly incorporate elements that do not belong to an Euclidean space such as the Lie group members. In this analysis, we use the parallel transport, which requires us to explicitly consider SE(3) as representing rigid bodies though the use of the affine connection. We present experimental results to show the effectiveness of the proposed approach for state estimation of a quadrotor platform.

The agility and ease of control make a quadrotor aircraft an attractive platform for studying swarm behavior, modeling, and control. The energetics of sustained flight for small aircraft, however, limit typical applications to only a few minutes. Adding payloads - and the mechanisms used to manipulate them - reduces this flight time even further. In this paper we present the flying monkey, a novel robot platform having three main capabilities: walking, grasping, and flight. This new robotic platform merges one of the world's smallest quadrotor aircraft with a lightweight, single-degree-of-freedom walking mechanism and an SMA-actuated gripper to enable all three functions in a 30 g package. The main goal and key contribution of this paper is to design and prototype the flying monkey that has increased mission life and capabilities through the combination of the functionalities of legged and aerial robots.

We are interested in a principled study of the impact of diversity in heterogeneous large-scale distributed robotic systems. In order to evaluate the implications of heterogeneity on performance, we consider the concrete problem of distributing a large group of robots among a set of tasks that require specialized capabilities in order to be completed. We model the system of heterogeneous robots as a community of species, where each species (robot type) is defined by the traits (capabilities) that it owns. We develop a continuous model of the system at a macroscopic level, and formulate an optimization problem that produces an optimal set of transition rates for each species, so that the desired trait distribution is reached as quickly as possible. In order to evaluate the effects of heterogeneity, we propose a diversity metric that defines the notion of eigenspecies. We show that our metric correlates with performance: the higher the cardinality of the eigenspecies, the harder it becomes to optimize the system. Our approach is validated over multiple levels of abstraction, and real robot results confirm its validity on physical platforms.

Unmanned aerial vehicles (UAVs) have the potential to significantly impact early detection and monitoring of plant diseases. In this paper, we present preliminary work in developing a UAV-mounted sensor suite for detection of citrus greening disease, a major threat to Florida citrus production. We propose a depth-invariant sensing methodology for measuring reflectance of polarized amber light, a metric which has been found to measure starch accumulation in greening-infected leaves. We describe the implications of adding depth information to this method, including the use of machine learning models to discriminate between healthy and infected leaves with validation accuracies up to 93%. Additionally, we discuss stipulations and challenges of use of the system with UAV platforms. This sensing system has the potential to allow for rapid scanning of groves to determine the spread of the disease, especially in areas where infection is still in early stages, including citrus farms in California. Although presented in the context of citrus greening disease, the methods can be applied to a variety of plant pathology studies, enabling timely monitoring of plant health-impacting scientists, growers, and policymakers.

This paper addresses assembly sequence planning for constructing planar structures of the common brick wall pattern collectively with mobile modular robots that have the same rectangular footprint. Here we present a new algorithm for target structures with internal holes that our previous algorithm was not able to address. Our new algorithm constructs a feasible assembly sequence where robots do not have to pass through narrow corridors while approaching their target positions. The algorithm is provably correct and complete and runs in time that is linear in the size of a target structure, that is, the number of its parts. We also present software implementing our algorithms and a set of numerical experiments using the software. Finally, we extend our algorithms to address other symmetric patterns formed by a collection of congruent rectangles on the plane.

In this paper we propose a hybrid architecture that allows a team of mobile robots to self-organize into a multi-hop ad-hoc network and solve the joint mobility and communication problem in complex environments to complete a given task. The system consists of an outer global planning loop and an inner local loop responsible for motion and network routing, arranged in a two-stage feedback system. This system is able to leverage the benefits of previous systems, while avoiding their drawbacks. This results in a lightweight responsive system that is able to operate in complex environments with minimal global coordination while maintaining a minimum end-to-end data rate between robots. Two main benefits of our approach are demonstrated through experimentation superior performance over existing systems and dynamic adjustment to unexpected events. We conclude with a demonstration of the system operating in a realistic scenario, in which the team patrols a set of hallways.

In this paper we consider an online planning problem for unmanned aerial vehicle (UAV) operations. Specifically, a UAV has the task of reaching a goal from a set of possible goals while minimizing the amount of energy required. Due to unforeseen disturbances, it is possible that initially attractive goals might end up being very expensive during the execution. Thus, two main problems are investigated here: i) how to predict and plan the motion of the UAV at run time to minimize its energy consumption and ii) when to schedule next replanning time to avoid unnecessary periodic re-evaluation executions. Our approach considers a nonlinear model of the system for which a model predictive controller is used to determine the desired control inputs for each possible goal. These control inputs are then used to estimate the energy required to reach the different goals. Finally, a self-triggered scheduling policy determines how long to wait before replanning the goal to aim for. The proposed framework is validated through simulations and experiments in which a quadrotor must choose and reach some goal while being subject to external disturbances.

In the last decade, consumer electronic devices such as smartphones, are packaged with small cameras, gyroscopes, and accelerometers, all sensors allowing autonomous deployment of aerial robots in GPS-denied environments. Our previous work [1], demonstrated the feasibility of using smartphones for autonomous flight. In many applications, there is a large interest to the use multiple autonomous aerial vehicles in a cooperative manner to speed up the operation of the mission. In this work, we present the first fully autonomous smartphone-based swarm of quadrotors. Multiple vehicles are able to plan safe trajectories avoiding inter-robot collisions, optimizing at the same time a given task and concurrently building in a cooperative manner a 3-D map of the environment. The sensing, sensor fusion, control, and planning are all done on an offthe- shelf Samsung Galaxy S5 smartphone using just the single camera and IMU available on the phone. The work allows any consumer with multiple smartphones to autonomously drive a swarm of multiple vehicles without GPS, by downloading an app, and have the swarm cooperatively map a 3-D environment.

In the last decade, multi-rotor Micro Aerial Vehicles (MAVs) have attracted great attention from robotics researchers. Offering affordable agility and maneuverability, multi-rotor aircrafts have become the most commonly used platforms for robotics applications. Amongst the most promising applications are inspection of power-lines, cell-towers, large and constrained infrastructures and precision agriculture. While GPS offers an easy solution for outdoor autonomy, using on-board sensors is the only solution for autonomy in constrained indoor environments. In this paper, we present our results on autonomous inspection of completely dark, featureless, symmetric dam penstocks using cameras and range sensors. We use a hex-rotor platform equipped with an IMU, four cameras and two lidars. One of the cameras tracks features on the walls using the on-board illumination to estimate the position along the tunnel axis unobservable to range sensors while all of the cameras are used for panoramic image construction. The two lidars estimate the remaining degrees of freedom (DOF). Outputs of the two estimators are fused using an Unscented Kalman Filter (UKF). A moderately trained operator defines waypoints using the Remote Control (RC). We demonstrate our results from Carters Dam, GA and Glen Canyon Dam, AZ which include panoramic images for cracks and rusty spot detection and 6-DOF estimation results with ground truth comparisons. To our knowledge ours is the only study that can autonomously inspect environments with no geometric cues and poor to no external illumination using MAVs.

In this work, we consider the labeled multi-robot planning problem. In this paradigm, a team of robots at fixed start positions must navigate to pre-specified and noninterchangable goal positions. While many algorithms have been proposed for finding optimal solutions to this problem, most methods assume that the robots are kinematic agents, whereas in reality, robots often have high-order dynamics that must be respected by their trajectories. Here, we propose a centralized method for generating trajectories for teams of robots with general nth-order dynamics navigating to labeled goals. Our algorithm is safe and complete and additionally allows for decoupled optimization of each robot's trajectory as a Quadratic Program with linear constraints. We present simulation results for teams of up to 20 robots.

We propose a new optimal trajectory generation technique on SE(3) which avoids known obstacles. We leverage techniques from differential geometry and Lie algebra to formulate a cost functional which is intrinsic to the geometric structure of this space and makes physical sense. We propose an approximation technique to generate trajectories on the subgroup SO(3) and use Semidefinite Programming (SDP) to approximate an NP-Hard problem with one which is tractable to compute. From this trajectory on the subgroup, the trajectory generation on the other dimensions of the group becomes a Quadratic Program (QP). For obstacle avoidance, we use a computational geometric technique to decompose the environment into overlapping convex regions to confine the trajectory. We show how this motion planning technique can be used to generate feasible trajectories for a space robot in SE(3) and describe controllers that enable the execution of the generated trajectory. We compare our method to other geometric techniques for calculating trajectories on SO(3) and SE(3), but in an obstacle-free environment.

In this paper we examine the problem of autonomously exploring and mapping an environment using a mobile robot. The robot uses a graph-based SLAM system to perform mapping and represents the map as an occupancy grid. In this setting, the robot must trade-off between exploring new area to complete the task and exploiting the existing information to maintain good localization. Selecting actions that decrease the map uncertainty while not significantly increasing the robot's localization uncertainty is challenging. We present a novel information-theoretic utility function that uses both Shannon's and Rényi's definitions of entropy to jointly consider the uncertainty of the robot and the map. This allows us to fuse both uncertainties without the use of manual tuning. We present simulations and experiments comparing the proposed utility function to state-of-the-art utility functions, which only use Shannon's entropy. We show that by using the proposed utility function, the robot and map uncertainties are smaller than using other existing methods.

We develop a computationally efficient control policy for active perception that incorporates explicit models of sensing and mobility to build 3D maps with ground and aerial robots. Like previous work, our policy maximizes an information-theoretic objective function between the discrete occupancy belief distribution (e.g., voxel grid) and future measurements that can be made by mobile sensors. However, our work is unique in three ways. First, we show that by using Cauchy-Schwarz Quadratic Mutual Information (CSQMI), we get significant gains in efficiency. Second, while most previous methods adopt a myopic, gradient-following approach that yields poor convergence properties, our algorithm searches over a set of paths and is less susceptible to local minima. In doing so, we explicitly incorporate models of sensors, and model the dependence (and independence) of measurements over multiple time steps in a path. Third, because we consider models of sensing and mobility, our method naturally applies to both ground and aerial vehicles. The paper describes the basic models, the problem formulation and the algorithm, and demonstrates applications via simulation and experimentation.

Discrete-time estimation of rigid body attitude and angular velocity without any knowledge of the attitude dynamics model, is treated using the discrete Lagrange-d'Alembert principle. Using body-fixed sensor measurements of direction vectors and angular velocity, a Lagrangian is obtained as the difference between a kinetic energy-like term that is quadratic in the angular velocity estimation error, and an artificial potential obtained from Wahba's function. An additional dissipation term that depends linearly on the angular velocity estimation error is introduced, and the discrete Lagrange-d'Alembert principle is applied to the Lagrangian with this dissipation. An implicit and an explicit first-order version of this discrete-time estimation scheme is presented. A comparison of this estimator is made with certain state-of-the-art attitude estimators in the absence of bias in sensor readings. Numerical simulations show that this estimator is robust and unlike extended Kalman filter-based schemes, its convergence does not depend on the gain values. In addition, the variational estimator is found to be more computationally efficient than these other estimators.

The fusion of IMU and RGB-D sensors presents an interesting combination of information to achieve autonomous localization and mapping using robotic platforms such as ground robots and flying vehicles. In this paper, we present a software framework for cooperative localization and mapping while simultaneously using multiple aerial platforms. We employ a monocular visual odometry algorithm to solve the localization task, where the depth data flow associated to the RGB image is used to estimate the scale factor associated with the visual information. The current framework enables autonomous onboard control of each vehicle with cooperative localization and mapping. We present a methodology that provides both a sparse map generated by the monocular SLAM and a multiple resolution dense map generated by the associated depth. The localization algorithm and both 3D mapping algorithms work in parallel improving the system real-time reliability. We present experimental results to show the effectiveness of the proposed approach using two quadrotors platforms.

Scaling down the size and mass of micro aerial vehicles (MAVs) increases their agility and their ability to operate in tight formations. In addition, smaller robots are safer and, as we will show in this paper, more robust to collisions. This paper addresses the development of a pico quadrotor measuring 11 cm from tip to tip, with a mass of 25g. To increase the robustness of the robot to collisions, the vehicle is equipped with a 2 gram carbon fiber cage that protects it from impact velocities in excess of 4 m/s and also permits recovery after collisions. We present the design of the electrical, mechanical and computational elements, as well as experimental results demonstrating trajectory following with feedback from an external motion camera system, recovery from collisions with walls and other robots, and formation flight.

We consider the problem of deploying a swarm of mobile robots into an unknown environment for attaining complete sensor coverage of the environment. The robots have limited and noisy sensing capabilities and no metric or global information available to them. Using tools from algebraic topology, we formally describe the sensor coverage as a simplicial complex, deploy robots through the complex using bearing-based local controllers, and attain coverage while identifying and removing sensor redundancies. Despite the highly limited sensing capabilities and complete lack of global localization and metric information, we demonstrate that the proposed algorithm is complete, always terminates in a finite-sized environment, is guaranteed to attain complete coverage and is robust to sensor failures. The algorithm presented in this paper was demonstrated through simulation and proves to effectively cover and explore unknown indoor environments.

There have been increasing interests in the robotics community in building smaller and more agile autonomous micro aerial vehicles (MAVs). In particular, the monocular visual-inertial system (VINS) that consists of only a camera and an inertial measurement unit (IMU) forms a great minimum sensor suite due to its superior size, weight, and power (SWaP) characteristics. In this paper, we present a tightly-coupled nonlinear optimization-based monocular VINS estimator for autonomous rotorcraft MAVs. Our estimator allows the MAV to execute trajectories at 2 m/s with roll and pitch angles up to 30 degrees. We present extensive statistical analysis to verify the performance of our approach in different environments with varying flight speeds.

Independent control of several, untethered microrobots remains a significant challenge. Although progress has been made in controlling multiple robots at scales below 100 mm, motion is dependent on external energy sources for propulsion and is often not fully decoupled. Here, we develop a method to dynamically pattern light at a resolution of 1 μm2per pixel in a closed loop real-time system using a digital micromirror device (DMD) system. We test the system on microbiorobots - bacteria-coated microplates - and demonstrate that we can selectively stimulate pre-defined regions of bioactuators. We verify previously developed dynamic models as a step toward developing individually addressable, self-propelled microrobots.

In this paper, we present a trajectory planning method to navigate a quadrotor with a cable-suspended payload through known obstacle-filled environments. We model the system as a hybrid dynamical system and formulate the trajectory generation problem as a Mixed Integer Quadratic Program (MIQP). Specifically, we address two novel challenges. First, we plan for a multi-body system, and obstacle avoidance must be guaranteed for the quadrotor, load, and the cable. Second, our method accommodates transitions between subsystems of the hybrid dynamical system, allowing for maneuvers that would otherwise be infeasible if the cable were constrained to remain taut. Numerical and experimental results validate the proposed approach for the full hybrid system.

Consumer grade technology seen in cameras and phones has led to the price/performance ratio of sensors and processors falling dramatically over the last decade. In particular, most devices are packaged with a camera, a gyroscope, and an accelerometer, important sensors for aerial robotics. The low mass and small form factor make them particularly well suited for autonomous flight with small flying robots, especially in GPS-denied environments. In this work, we present the first fully autonomous smartphone-based quadrotor. All the computation, sensing and control runs on an off-the-shelf smartphone, with all the software functionality in a smartphone app.We show how quadrotors can be stabilized and controlled to achieve autonomous flight in indoor buildings with application to smart homes, search and rescue, construction and architecture. The work allows any consumer with a smartphone to autonomously drive a quadrotor robot platform, even without GPS, by downloading an app, and concurrently build 3-D maps.

We study the problem of planning paths for a team of robots motivated by coverage, persistent monitoring and surveillance applications. The input is a set of target points in a polygonal environment that must be monitored using robots with omni-directional cameras. The goal is to compute paths for all robots such that every target is visible from at least one path. The cost of a path is given by the weighted combination of the length of the path (travel time) and the number of viewpoints along the path (measurement time). The overall cost is given by the maximum cost over all robot paths and the objective is to minimize the maximum cost. In its general form, this problem is NP-hard. In this paper, we present an optimal algorithm and a constant factor approximation for two special versions of the problem. In both cases, the paths are restricted to lie on a pre-defined curve in the polygon. We show that if the curve satisfies a special property, termed chain-visibility, then there exists an optimal algorithm for monitoring a given set of target locations. Furthermore, if we restrict the input polygon to the class of street polygons, then we present a constant-factor approximation which is applicable even if the set of target locations is the entire polygon. In addition to theoretical proofs, we also present results from simulation studies.

Micro Aerial Vehicles (MAVs) are becoming ubiquitous, but most experiments and demonstrations have been limited to slow flight except in open environments or in laboratories with motion capture systems. In this paper, we develop representations and algorithms for aggressive flight in cluttered environments. We incorporate the specific dynamics of the system and generate safe, feasible trajectories for fast navigation in real time. Specifically, we use a polyhedral decomposition of the visible free space and address the generation of safe trajectories that are within the space. Because of the limited field of view, we adopt a receding horizon control policy (RHCP) for planning over a finite time horizon, but with the guarantee that there exists a safe stopping control policy over a second time horizon from the planned state at the end of the horizon. Thus, the robot planning occurs over two horizons. While the robot executes the planned trajectory over the first time horizon, the map of obstacles is refreshed allowing the planner to generate a refined plan, once again over two time horizons. The key algorithmic contribution of the paper lies in the fast planning algorithm that is able to incorporate robot dynamics while guaranteeing safety. The algorithm is also optimal in the sense that the receding horizon control policy is based on minimizing the trajectory snap. Central to the algorithm is a novel polyhedral representation that allows us to abstract the trajectory planning problem as a problem of finding a path through a sequence of convex regions in configuration space.

This paper considers situations in which a team of mobile sensor platforms autonomously explores an environment to detect and localize an unknown number of targets. Individual sensors may be unreliable, failing to detect objects within the field-of-view, returning false positive measurements to clutter objects, and being unable to disambiguate true targets. In this setting, data association is difficult. We utilize the PHD filter for multitarget localization, simultaneously estimating the number of objects and their locations within the environment without the need to explicitly consider data association. Using sets of potential actions generated at multiple length scales for each robot, the team selects the joint action that maximizes the expected information gain over a finite time horizon. This is computed as the mutual information between the set of targets and the binary events of receiving no detections, effectively hedging against uninformative actions in a computationally tractable manner. We frame the controller as a receding-horizon problem. We demonstrate the real-world applicability of the proposed autonomous exploration strategy through hardware experiments, exploring an office environment with a team of ground robots. We also conduct a series of simulated experiments, varying the planning method, target cardinality, environment, and sensor modality. Note to Practitioners-Teams of small robots have the potential to automate many information gathering tasks, relaying data back to a base station or human operator from multiple vantage points within an environment. The information gathering tasks we consider in this work are those in which the number of objects being sought is not known at the onset of exploration. Such tasks are common in security and surveillance, where the number of such objects is often zero; search and rescue, where, for example, the number of people trapped due to a natural disaster can be large; or smart building/smart city applications, where the data collection needs may be on an even larger scale. This paper seeks to address the problem of automating this data collection process, so that a team of mobile sensor platforms are able to autonomously explore a given environment in order to determine the number of objects of interest and their locations, while avoiding any explicit data association, i.e., matching individual measurements to targets.

We present the methodology, algorithms, system design, and experiments addressing the self-assembly of large teams of autonomous robotic boats into floating platforms. Identical self-propelled robotic boats autonomously dock together and form connected structures with controllable variable stiffness. These structures can self-reconfigure into arbitrary shapes limited only by the number of rectangular elements assembled in brick-like patterns. An O(m2) complexity algorithm automatically generates assembly plans which maximize opportunities for parallelism while constructing operator-specified target configurations with m components. The system further features an O(n3) complexity algorithm for the concurrent assignment and planning of trajectories from n free robots to the growing structure. Such peer-to-peer assembly among modular robots compares favorably to a single active element assembling passive components in terms of both construction rate and potential robustness through redundancy. We describe hardware and software techniques to facilitate reliable docking of elements in the presence of estimation and actuation errors, and we consider how these local variable stiffness connections may be used to control the structural properties of the larger assembly. Assembly experiments validate these ideas in a fleet of 0.5 m long modular robotic boats with onboard thrusters, active connectors, and embedded computers.

We address the fundamental problem of goal-directed path planning in an uncertain environment represented as a probability (of occupancy) map. Most methods generally use a threshold to reduce the grayscale map to a binary map before applying off-the-shelf techniques to find the best path. This raises the somewhat ill-posed question, what is the right (optimal) value to threshold the map? We instead suggest a persistent homology approach to the problem-a topological approach in which we seek the homology class of trajectories that is most persistent for the given probability map. In other words, we want the class of trajectories that is free of obstacles over the largest range of threshold values. In order to make this problem tractable, we use homology in ℤ2coefficients (instead of the standard ℤ coefficients), and describe how graph search-based algorithms can be used to find trajectories in different homology classes. Our simulation results demonstrate the efficiency and practical applicability of the algorithm proposed in this paper.paper.

In this paper we investigate the problem of navigation for a planar mobile robot tethered to a base by a flexible cable of length L. Obstacles present in the environment, coupled with the cable length constraint, makes the problem highly non-trivial. We adopt a topological approach along with graph search-based techniques to solve this problem, wherein we use the notion of a homotopy augmented graph to capture the information about the homotopy class of the cable. This lets us plan traversable optimal trajectories from an initial robot position and cable configuration to a final position of the robot. We demonstrate the algorithm by planning trajectories in several cluttered environments and with different cable lengths. As a demonstration of practical applicability, using a dynamic simulation testbed we simulate a robot-cable system following a planned trajectory.

We formulate the position-based visual servoing problem for a quadrotor equipped with a monocular camera and an IMU relying only on features on planes and lines in order to fly above and perch on arbitrarily oriented lines. We show that we are able to compute the orientation of an arbitrarily oriented line, the speed of the robot and its position with respect to the target line using two points at a known distance on the line. The direction of the velocity is derived from optical flow induced by features on a plane in the background Finally, we demonstrate fully autonomous flight and perching using a small 230 gram quadrotor with all the computations running on the robot.

This paper addresses the self-assembly of a large team of autonomous boats into floating platforms. We describe the design of individual boats, the systems concept, the algorithms, the software architecture and experimental results with prototypes that are 1:12 scale realizations of modified ISO shipping containers, with the goal of demonstrating self-assembly into large maritime structures such as air strips, bridges, harbors or sea bases. Each container is a robotic module capable of holonomic motion that can dock in a brick pattern to form arbitrary shapes. Over 60 modules were built of varying capability. The docking mechanism is designed to be robust to large disturbances that can be expected in the high seas. The docking mechanism also incorporates adjustable stiffness so that the conglomerate can comply to waves representative of sea state three, and have the ability to dynamically stiffen as required. The component modules for autonomous assembly, docking and simultaneous collision-free planning as well as the software architecture are presented along with the description of experimental verification.

This paper considers the problem of decentralized goal assignment and trajectory generation for multi-robot networks when only local communication is available, and proposes an approach based on methods related to switched systems and set invariance. A family of Lyapunov-like functions is employed to encode the (local) decision making among candidate goal assignments, under which the agents pick the assignment which results in the shortest total distance to the goals. An additional family of Lyapunov-like barrier functions is activated in the case when the optimal assignment may lead to colliding trajectories, thus maintaining system safety while preserving the convergence guarantees. The proposed switching strategies give rise to feedback control policies which are scalable as the number of agents increases, and therefore are suitable for applications including first-response deployment of robotic networks under limited information sharing. Simulations demonstrate the efficacy of the proposed method.

We present a modular and extensible approach to integrate noisy measurements from multiple heterogeneous sensors that yield either absolute or relative observations at different and varying time intervals, and to provide smooth and globally consistent estimates of position in real time for autonomous flight. We describe the development of algorithms and software architecture for a new 1.9kg MAV platform equipped with an IMU, laser scanner, stereo cameras, pressure altimeter, magnetometer, and a GPS receiver, in which the state estimation and control are performed onboard on an Intel NUC 3rdgeneration i3 processor. We illustrate the robustness of our framework in large-scale, indoor-outdoor autonomous aerial navigation experiments involving traversals of over 440 meters at average speeds of 1.5 m/s with winds around 10 mph while entering and exiting buildings.

Microscale robots offer an unprecedented opportunity to perform tasks at resolutions approaching 1 μm, but the great majority of research to this point focuses on actuation and control. Potential applications for microrobots can be considerably expanded by integrating sensing, signal processing and feedback into the system. In this work, we demonstrate that technologies from the field of synthetic biology may be directly integrated into microrobotic systems to create cell-based programmable mobile sensors, with signal processors and memory units. Specifically, we integrate genetically engineered, ultraviolet light-sensing bacteria with magnetic microrobots, creating the first controllable biological microrobot that is capable of exploring, recording and reporting on the state of the microscale environment. We demonstrate two proof-of-concept prototypes: (a) an integrated microrobot platform that is able to sense biochemical signals, and (b) a microrobot platform that is able to deploy biosensor payloads to monitor biochemical signals, both in a biological environment. These results have important implications for integrated micro-bio-robotic systems for applications in biological engineering and research.

This paper addresses the dynamics, control, planning, and visual servoing for micro aerial vehicles to perform high-speed aerial grasping tasks. We draw inspiration from agile, fast-moving birds, such as raptors, that detect, locate, and execute high-speed swoop maneuvers to capture prey. Since these grasping maneuvers are predominantly in the sagittal plane, we consider the planar system and present mathematical models and algorithms for motion planning and control, required to incorporate similar capabilities in quadrotors equipped with a monocular camera. In particular, we develop a dynamical model directly in the image space, show that this is a differentially-flat system with the image features serving as flat outputs, outline a method for generating trajectories directly in the image feature space, develop a geometric visual controller that considers the second order dynamics (in contrast to most visual servoing controllers that assume first order dynamics), and present validation of our methods through both simulations and experiments.

Accurate localization is a problem in environments such as tunnels or pipes due to the hostile conditions, dimensions and the general lack of distinctive visual and/or structural features. Standard indoor localization techniques (e.g. visual SLAM) do not work well in pipelines given the lack of exploitable visual features, while outdoor techniques (GPS in particular) do not work inside metal pipes. In this paper, we present a RF odometry-like method to localize a robot along a pipe. Using a radio-frequency signal transmitter and a receiver, we propose and implement the complete setup in order to obtain periodic received-signal fadings and base a localization system on the periodicity of these fadings. There are two main advantages of the proposed system. First, the sensors are easy to install and can be used with any (ground, aerial) robot. Second, the fadings obtained are periodic, avoiding cumulative errors in localization.

We present a compositional motion planning framework for multi-robot systems based on an encoding to satisfiability modulo theories (SMT). In our framework, the desired behavior of a group of robots is specified using a set of safe linear temporal logic (LTL) properties. Our method relies on a library of motion primitives, each of which corresponds to a controller that ensures a particular trajectory in a given configuration. Using the closed-loop behavior of the robots under the action of different controllers, we formulate the motion planning problem as an SMT solving problem and use an off-the-shelf SMT solver to generate trajectories for the robots. Our approach can also be extended to synthesize optimal cost trajectories where optimality is defined with respect to the available motion primitives. Experimental results show that our framework can efficiently solve complex motion planning problems in the context of multi-robot systems.

Providing reliable end-to-end communication for teams of robots requires the integration of novel routing techniques, motion planning algorithms, and transport level communication protocols. In this paper we look at existing robust routing solutions that provide redundancy at the routing layer and develop the Multi-Confirmation Transmission Protocol (MCTP) to take advantage of that redundancy at the transport level. The resulting system that integrates robust routing and MCTP is evaluated in experiments performed in complex environments. The integrated system is observed to provide a robust architecture that allows for near lossless communication while operating in a complex environment with less traffic than standard confirmation protocols.

Vision-based formation control of multiple agents, such as mobile robots or fully autonomous cars, has recently received great interest due to its application in robotic networks and automated highways. This paper addresses the cooperative motion coordination of leader-follower formations of nonholonomic mobile robots, under visibility and communication constraints in known polygonal obstacle environments. We initially consider the case of N = 2 agents moving in L-F fashion and propose a feedback control strategy under which L ensures obstacle avoidance for both robots, while F ensures visibility maintenance with L and intervehicle collision avoidance. The derived algorithms are based on set-theoretic methods to guarantee visibility maintenance, dipolar vector fields to maintain the formation shape, and the consideration of the formation as a tractor-trailer system to ensure obstacle avoidance. We furthermore show how the coordination and control design extends to the case of N > 2 agents, and provide simulation results, which demonstrate the efficacy of the control solutions. The proposed algorithms do not require information exchange among robots, but are instead based on information locally available to each agent. In this way, the desired tasks are executed and achieved in a decentralized manner, with each robot taking care of converging to a desired configuration, while maintaining visibility with its target.

We propose a methodology of applying PoMDPs at a sufficiently high abstraction of a high-dimensional continuous-time partially observable hybrid system. In particular, we develop a two-layer hybrid controller, where the higher-level PoMDP-based hybrid controller learns the boundaries between various modes and appropriately switches between them. The modes partition the state-space and represent a closed-loop hybrid system with a lower-level hybrid controller. We apply this methodology onto the problem of bipedal walking on varying terrain, where the gradient change in the terrain is only partially observable (due to poor and noisy sensors.) We develop three lower-level hybrid controllers that result in robust walking on level ground, up and down ramps. The higher-level PoMDP-based hybrid controller then learns the boundary between these controllers and is used to perform appropriate controller switching. With only a coarse, discrete estimate of walking speed, the controller enables traversing terrain both with long sustained constant slopes, and with rapid changes in slope. Simulation results are presented on a 26-dimensional planar bipedal robot model that incorporates contact forces and friction.

Opening and navigating through doors remains a challenging problem, particularly in cluttered environments and for spring-loaded doors. Passing through doors, especially spring-loaded doors, requires making and breaking contacts with the door and preventing the door from closing while passing through. In this work, we present a planning framework that handles non-spring and spring-loaded doors, in cluttered or confined workspaces, planning the approach to the door, pushing or pulling it open, and passing through. Because the problem is solved in a combined search space, the planner yields an overall least-cost path. The planner is able to insert a transition between robot-door contacts at any point along the plan. We utilize a compact graph-based representation of the problem to keep planning times low. We precompute the force workspace of the end-effectors to eliminate checks against joint torque limits at plan time. We have validated our solution in both simulation and real-world experiments on the PR2 mobile manipulation platform; the robot is able to successfully open a variety of spring-loaded and non-spring-loaded doors by pushing and pulling.

This paper addresses the task of searching for an unknown number of static targets within a known obstacle map using a team of mobile robots equipped with noisy, limited field-of-view sensors. Such sensors may fail to detect a subset of the visible targets or return false positive detections. These measurement sets are used to localize the targets using the Probability Hypothesis Density, or PHD, filter. Robots communicate with each other on a local peer-to-peer basis and with a server or the cloud via access points, exchanging measurements and poses to update their belief about the targets and plan future actions. The server provides a mechanism to collect and synthesize information from all robots and to share the global, albeit time-delayed, belief state to robots near access points. We design a decentralized control scheme that exploits this communication architecture and the PHD representation of the belief state. Specifically, robots move to maximize mutual information between the target set and measurements, both self-collected and those available by accessing the server, balancing local exploration with sharing knowledge across the team. Furthermore, robots coordinate their actions with other robots exploring the same local region of the environment.

This paper describes an approach to motion generation for quadrotor micro-UAV's navigating cluttered and partially known environments. We pursue a graph search method that, despite the high dimensionality of the problem, the complex dynamics of the system and the continuously changing environment model is capable of generating dynamically feasible motions in real-time. This is enabled by leveraging the differential flatness property of the system and by developing a structured search space based on state lattice motion primitives. We suggest a greedy algorithm to generate these primitives off-line automatically, given the robot's motion model. The process samples the reachability of the system and reduces it to a set of representative, canonical motions that are compatible with the state lattice structure, which guarantees that any incremental replanning algorithm is able to produce smooth dynamically feasible motion plans while reusing previous computation between replans. Simulated and physical experimental results demonstrate real-time replanning due to the inevitable and frequent world model updates during micro-UAV motion in partially known environments.

In this paper, we consider the development of a rotorcraft micro aerial vehicle (MAV) system capable of vision-based state estimation in complex environments. We pursue a systems solution for the hardware and software to enable autonomous flight with a small rotorcraft in complex indoor and outdoor environments using only onboard vision and inertial sensors. As rotorcrafts frequently operate in hover or nearhover conditions, we propose a vision-based state estimation approach that does not drift when the vehicle remains stationary. The vision-based estimation approach combines the advantages of monocular vision (range, faster processing) with that of stereo vision (availability of scale and depth information), while overcoming several disadvantages of both. Specifically, our system relies on fisheye camera images at 25 Hz and imagery from a second camera at a much lower frequency for metric scale initialization and failure recovery. This estimate is fused with IMU information to yield state estimates at 100 Hz for feedback control. We show indoor experimental results with performance benchmarking and illustrate the autonomous operation of the system in challenging indoor and outdoor environments.

A quadrotor with a cable-suspended load with eight degrees of freedom and four degrees underactuation is considered and the system is established to be a differentially-flat hybrid system. Using the flatness property, a trajectory generation method is presented that enables finding nominal trajectories with various constraints that not only result in minimal load swing if required, but can also cause a large swing in the load for dynamically agile motions. A control design is presented for the system specialized to the planar case, that enables tracking of either the quadrotor attitude, the load attitude or the position of the load. Stability proofs for the controller design and experimental validation of the proposed controller are presented.

For applications such as targeting therapeutic nanocarriers to specific tissues, as well as for the more general problem of assessing binding affinity between proteins, it is important to discover tools which may be used for both micromanipulation as well as force microscopy. In this work, we measure picoscale forces using microscale robots, using antibody-coated microbead probes as an interface. We develop a model for a detailed understanding of the relationship between robot velocity and applied force, and apply this to the measurement of binding forces between antibody-coated beads and protein-coated glass slides. We also verify binding between specific antibody/protein combinations relative to untargeted control combinations, and we measure the rate at which this binding occurs. This offers a truly novel technique for measuring binding forces between proteins.

This paper considers the problem of finding optimal time parameterized trajectories for N unlabeled robots navigating through a cluttered environment to N unlabeled goal locations where success is defined as every goal being reached by any robot. We propose a complete computationally-tractable algorithm for simultaneously finding trajectories and assignment of goal locations. This method is then demonstrated to have an upper complexity bound of that scales polynomially in the number of robots, O(N3). The trajectories generated are guaranteed to be minimum length and collision free, while the assignment policy minimizes the maximum distance travelled. The key idea in the paper comes from the coupling between the optimal assignment, the properties of the resulting paths, and the set of valid priority assignments to the robots. These benefits result from structure in the solution to the optimal assignment to create a partial ordering of the robots, which in turn allows safe trajectories to be easily generated. Finally, we demonstrate the performance of the algorithm through simulations with tens and hundreds of robots operating in cluttered and confined environments.

Autonomous micro aerial robots can operate in three-dimensional unstructured environments, and offer many opportunities for environmental monitoring, search and rescue, and first response. I will describe the challenges in developing small, agile robots and our recent work in addressing these challenges. I will also discuss the deployment of large numbers of aerial robots, focusing on the control and planning problems with applications to cooperative manipulation and transport, construction, and exploration and mapping.

We present the mathematical framework and algorithms for multi-robot topological exploration of unknown environments in which the main goal is to identify the different topological classes of trajectories and thus efficiently distribute the task of exploration among different groups of robots. We consider two-dimensional configuration spaces. At any point in time, the robots' map consists of known, partially-mapped obstacles. The unknown, yet-to-be-explored area is mapped to a single point, thus giving us a quotient space. The topological classes on the quotient space allows us to define topological classes of trajectories connecting a robot pose to the unknown region in the original configuration space. Robots explore this configuration space choosing different homology classes when confronted by obstacles or walls. We illustrate the basic idea with simulations of small teams of robots. Experiments with a single robot illustrate the applicability of the method to robots that have small sensor footprints and limited computational resources. We also provide comparisons with a standard frontier-based algorithm.

We are interested in the multiple robot surveillance problem where robots must allocate waypoints to be visited among themselves and plan paths through different waypoints while avoiding obstacles. Furthermore, the robots are allocated specific times to reach their respective goal locations and as a result they have to decide which robots have to visit which waypoints. Such a problem has the challenge of computing the allocation of waypoints across robots, ordering for these waypoints and dynamical feasibility of the paths between waypoints. We present an algorithm that runs a series of graph searches to solve the problem and provide theoretical analysis that our approach yields an optimal solution. We present simulated results as well as experiments on two UAVs that validate the capability of our algorithm. For a single robot, we can solve instances having 10–15 waypoints and for multiple robots, instances having five robots and 10 waypoints can be solved.

This paper addresses the kinematics of cooperative transport of payloads suspended by multiple aerial robots with cables. In such problems, it is important to determine the positions of the aerial robots to achieve a specified position and orientation of the payload. In general, this inverse kinematics problem has no solutions for the case with one or two robots and infinitely many solutions for three or more robots. However, in the case with three robots, when the tensions of the cables are also specified, this inverse kinematics problem is shown to have a finite number of solutions. In order to obtain all possible solutions, an efficient analytic algorithm based on dialytic elimination is presented in this paper. Case studies with an equilateral triangle payload and a general payload are used for demonstration. In addition, a numerical procedure is developed to determine the set of allowable tensions. Finally, an approach for stability analysis is developed, and the stability of all equilibrium configurations is analyzed.

The focus of this study is on the design of feedback control laws for swarms of robots that are based on models from fluid dynamics. We apply an incompressible fluid model to solve a pattern generation task. Possible applications of an efficient solution to this task are surveillance and the cordoning off of hazardous areas. More specifically, we use the smoothed-particle hydrodynamics (SPH) technique to devise decentralized controllers that force the robots to behave in a similar manner to fluid particles. Our approach deals with static and dynamic obstacles. Considerations such as finite size and nonholonomic constraints are also addressed. In the absence of obstacles, we prove the stability and convergence of controllers that are based on the SPH method. Computer simulations and actual robot experiments are shown to validate the proposed approach.

Homotopy classes of trajectories, arising due to the presence of obstacles, are defined as sets of trajectories that can be transformed into each other by gradual bending and stretching without colliding with obstacles. The problem of exploring/finding the different homotopy classes in an environment and the problem of finding least-cost paths restricted to a specific homotopy class (or not belonging to certain homotopy classes) arises frequently in such applications as predicting paths for unpredictable entities and deployment of multiple agents for efficient exploration of an environment. In [Bhattacharya, Kumar, Likhachev, AAAI 2010] we have shown how homotopy classes of trajectories on a two-dimensional plane with obstacles can be classified and identified using the Cauchy Integral Theorem and the Residue Theorem from Complex Analysis. In more recent work [Bhattacharya, Likhachev, Kumar, RSS 2011] we extended this representation to three-dimensional spaces by exploiting certain laws from the Theory of Electromagnetism (Biot-Savart law and Ampere's Law) for representing and identifying homotopy classes in three dimensions in an efficient way. Using such a representation, we showed that homotopy class constraints can be seamlessly weaved into graph search techniques for determining optimal path constrained to certain homotopy classes or forbidden from others, as well as for exploring different homotopy classes in an environment. (This is a condensed, non-technical overview of work previously published in the proceedings of Robotics: Science and Systems, 2011 conference [Bhattacharya, Likhachev, Kumar, RSS 2011].)

We propose an architecture and algorithms for maintaining end-to-end network connectivity for autonomous teams of robots. By adopting stochastic models of point-to-point wireless communication and computing robust solutions to the network routing problem, we ensure reliable connectivity during robot movement in complex environments. We fully integrate the solution to network routing with the choice of node positions through the use of randomized motion planning techniques. Experiments demonstrate that our method succeeds in navigating a complex environment while ensuring that end-to-end communication rates meet or exceed prescribed values within a target failure tolerance.

We present an algorithm for the generation of optimal trajectories for teams of heterogeneous quadrotors in three-dimensional environments with obstacles. We formulate the problem using mixed-integer quadratic programs (MIQPs) where the integer constraints are used to enforce collision avoidance. The method allows for different sizes, capabilities, and varying dynamic effects between different quadrotors. Experimental results illustrate the method applied to teams of up to four quadrotors ranging from 65 to 962 grams and 21 to 67 cm in width following trajectories in three-dimensional environments with obstacles with accelerations approaching 1g.

This paper addresses the problem of controlling a leader-follower (L - F) formation of two unicycle mobile robots moving under visibility constraints in a known obstacle environment. Visibility constraints are realized as inequality state constraints that determine a visibility set K. Maintaining visibility is translated into controlling the robots so that system trajectories starting in K always remain in K. We provide the conditions under which visibility is maintained, as well as a feedback control scheme that forces F to converge and remain into a set of desired configurations w.r.t. L while maintaining visibility. We also propose a cooperative control scheme for the motion of the formation in a known obstacle environment, so that both collision avoidance and maintaining visibility are ensured. The proposed control schemes are decentralized, in the sense that there is no direct communication between the robots. The efficacy of our algorithms is evaluated through simulations.

We address the problem of synthesizing planar, bimanual, whole-arm grasps by developing the abstraction of an open chain gripper, an open, planar chain of rigid links and revolute joints contacting a planar, polygonal object, and introducing the concept of a generalized contact. Since two generalized contacts suffice for planar grasps, we leverage previous work on caging and immobilization for two contact grasps to construct an algorithm which synthesizes contact configurations for stable grasping. Simulations show that our methodology can be applied to grasp a wide range of planar objects without relying on special-purpose end-effectors. Representative experiments with the PR2 humanoid robot illustrate that this approach is practical.

In this paper, we propose a stochastic differential equation-based exploration algorithm to enable exploration in three-dimensional indoor environments with a payload constrained micro-aerial vehicle (MAV). We are able to address computation, memory, and sensor limitations by considering only the known occupied space in the current map. We determine regions for further exploration based on the evolution of a stochastic differential equation that simulates the expansion of a system of particles with Newtonian dynamics. The regions of most significant particle expansion correlate to unexplored space. After identifying and processing these regions, the autonomous MAV navigates to these locations to enable fully autonomous exploration. The performance of the approach is demonstrated through numerical simulations and experimental results in single and multi-floor indoor experiments.

Transport of individual cells or chemical payloads on a subcellular scale is an enabling tool for the study of cellular communication, cell migration, and other localized phenomena. We present a magnetically actuated robotic system for the fully automated manipulation of cells and microbeads. Our strategy uses autofluorescent robotic transporters and fluorescently labeled microbeads to aid tracking and control in optically obstructed environments. We demonstrate automated delivery of microbeads infused with chemicals to specified positions on neurons.

We address formation control for a team of quadrotors in which the robots follow a specified group trajectory while safely changing the shape of the formation according to specifications. The formation is prescribed by shape vectors which dictate the relative separations and bearings between the robots, while the group trajectory is specified as the desired trajectory of a leader or a virtual robot in the group. Each robot plans its trajectory independently based on its local information of neighboring robots which includes both the neighbor's planned trajectory and an estimate of its state. We show that the decentralized trajectory planners (a) result in consensus on the planned trajectory for predefined shapes and (b) achieve safe reconfiguration when changing shapes.

We address the problem of synthesizing spatial, bimanual, whole-arm grasps by developing the abstractions of an open chain gripper, an open, spatial chain of rigid links and joints between the links, contacting a polyhedral object, and a generalized contact. We show that every general polyhedron can be immobilized by three generalized contacts. We leverage previous work on immobilization to construct an algorithm that synthesizes contact configurations for stable grasping. Our methodology can be applied to grasp a wide range of objects without relying on special-purpose end-effectors as shown in simulations and experiments with a PR2 humanoid robot.

In this paper, a system architecture to provide end-to-end network connectivity for autonomous teams of robots is discussed. The core of the proposed system is a cyber-physical controller whose goal is to ensure network connectivity as robots move to accomplish their assigned tasks. Due to channel quality uncertainties inherent to wireless propagation, we adopt a stochastic model where achievable rates are modeled as random variables. The cyber component of the controller determines routing variables that maximize the probability of having a connected network for given positions. The physical component determines feasible robot trajectories that are restricted to safe configurations which ensure these probabilities stay above a minimum reliability level. Local trajectory planning algorithms are proposed for simple environments and leveraged to obtain global planning algorithms to handle complex surroundings. The resulting integrated controllers are robust in that end-to-end communication survives with high probability even if individual point-to-point links are likely to fail with significant probability. Experiments demonstrate that the global planning algorithm succeeds in navigating a complex environment while ensuring that end-to-end communication rates meet or exceed prescribed values within a target failure tolerance.

We present an approach to designing scalable, decentralized control policies that produce a desired collective behavior in a spatially inhomogeneous robotic swarm that emulates a system of chemically reacting molecules. Our approach is based on abstracting the swarm to an advection-diffusion-reaction partial differential equation model, which we solve numerically using smoothed particle hydrodynamics (SPH), a meshfree technique that is suitable for advection-dominated systems. The parameters of the macroscopic model are mapped onto the deterministic and random components of individual robot motion and the probabilities that determine stochastic robot task transitions. For very large swarms that are prohibitively expensive to simulate, the macroscopic model, which is independent of the population size, is a useful tool for synthesizing robot control policies with guarantees on performance in a top-down fashion. We illustrate our methodology by formulating a model of rabbiteye blueberry pollination by a swarm of robotic bees and using the macroscopic model to select control policies for efficient pollination.

Oil skimmers towed by two vehicles have been widely used for skimming of oil on the water surface. In this paper, we address the cooperative control of two autonomous surface vehicles for oil skimming and cleanings. We model the skimmer as a flexible, floating rope of constant length as well as discrete segmented model. We derive the equations governing the rope dynamics from first principles and demonstrate their application through simulations. We have performed field experiments with two autonomous surface vehicles that substantiate the proposed model and provides estimates of constants underlying the model. We propose a method for controlling the shape of the rope, and derive the conditions that maximize skimming efficiency.

We address the controller design and the trajectory generation for a quadrotor maneuvering in three dimensions in a tightly constrained setting typical of indoor environments. In such settings, it is necessary to allow for significant excursions of the attitude from the hover state and small angle approximations cannot be justified for the roll and pitch. We develop an algorithm that enables the real-time generation of optimal trajectories through a sequence of 3-D positions and yaw angles, while ensuring safe passage through specified corridors and satisfying constraints on velocities, accelerations and inputs. A nonlinear controller ensures the faithful tracking of these trajectories. Experimental results illustrate the application of the method to fast motion (5-10 body lengths/second) in three-dimensional slalom courses.

This video features advanced control capabilities for a quadrotor helicopter. We use an Ascending Technologies Hummingbird quadrotor [1] in conjunction with a Vicon [2] motion capture system in a flight arena [3] . The video shows a single quadrotor flying through windows with only three inches of clearance on any side of the platform and perching on vertical, declined, and inclined walls. Robustness to failed perching is demonstrated when it recovers to a stable hover from a failed perching attempt. Next, the video shows control along optimal trajectories through desired waypoints which includes flying through static and thrown hoops. Seen next is a gripper designed for the quadrotor to pickup objects and perch on horizontal surfaces. Finally, multiple quadrotors are shown cooperatively manipulating payloads to which they are rigidly attached.

For such biomedical applications as single cell manipulation and targeted delivery of chemicals, it is important to fabricate microstructures that can be powered and controlled without a tether in fluidic environments. In this work, we describe the construction and operation of micronsized, biocompatible ferromagnetic microtransporters driven by external magnetic fields capable of exerting forces at the pico Newton scale. We develop microtransporters using a simple, single step micro fabrication technique that allows us to produce large numbers in the same step. We also fabricate microgels to deliver drugs. We demonstrate that the microtransporters can be navigated to separate individual targeted cells with micron-scale precision, and deliver microgels without disturbing the cells in the neighborhood and the local microenvironment.

This paper examines the dynamic interplay between decentralized controllers and mesh networking protocols for controlling groups of robots. A proportional controller is used to maintain robots in a formation based on estimates of the robots' states observed through the network. The state information is propagated through the network using a flooding algorithm, which introduces topology-dependent time delays. The coupled interaction of information flow over the network with the dynamics of the robots is modeled as a linear dynamical system. With this model it is shown that systems made up of robots with stable first order dynamics are stable for all network update times, positive feedback gains, and connected communication graphs. With higher order robot dynamics it is found that stability is a complex and counter intuitive function of feedback gain and network update time. A performance metric is proposed for analyzing the convergence rate of the multi-robot system. Experiments with flying quadrotor robots verify the predictions of the model and the performance metric.

In this paper, we consider the problem of autonomous navigation with a micro aerial vehicle (MAV) in indoor environments. In particular, we are interested in autonomous navigation in buildings with multiple floors. To ensure that the robot is fully autonomous, we require all computation to occur on the robot without need for external infrastructure, communication, or human interaction beyond high-level commands. Therefore, we pursue a system design and methodology that enables autonomous navigation with real time performance on a mobile processor using only onboard sensors. Specifically, we address multi-floor mapping with loop closure, localization, planning, and autonomous control, including adaptation to aerodynamic effects during traversal through spaces with low vertical clearance or strong external disturbances. We present experimental results with ground truth comparisons and performance analysis.

In this paper, we consider the problem of autonomous navigation with a micro aerial vehicle (MAV) in indoor environments. In particular, we are interested in autonomous navigation in buildings with multiple floors. To ensure that the robot is fully autonomous, we require all computation to occur on the robot without need for external infrastructure, communication, or human interaction beyond high-level commands. Therefore, we pursue a system design and methodology that enables autonomous navigation with real time performance on a mobile processor using only onboard sensors. Specifically, we address multi-floor mapping with loop closure, localization, planning, and autonomous control, including adaptation to aerodynamic effects during traversal through spaces with low vertical clearance or strong external disturbances. We present experimental results with ground truth comparisons and performance analysis.

We consider the problem of deploying robots in formations that ensure network connectivity between a fixed base station and a set of independent agents wandering in the environment. We adopt a communications model that requires line-of-sight and then solve for robot placements by finding mutually-visible configurations in a polygonal decomposition of the environment map. Both the static deployment case and the case of finding deployments that minimize total robot movement are considered. We provide algorithms for the moving agent case, consider their performance on various discretizations for a range of problem sizes, and discuss our experimental implementation of the presented ideas.

We address the problem of developing feedback controllers for a group of robots with second-order dynamics in an obstacle-filled, D-dimensional environment. Our control algorithm takes into account communication constraints, obstacle avoidance, and inter-robot collision avoidance, by synthesizing a piecewise smooth vector field for safe navigation. First, the feasible free joint configuration space is tessellated into polytopes that account for the desired constraints. We search the graph of these polytopes to find a discrete path to the goal polytope. We then use a novel navigation function-based feedback controller that drives the system from one polytope to the next and eventually to the goal. The controller exploits the fact that two adjoining polytopes in the planned discrete path together form a star-shaped object that is obstacle free; this enables the design of navigation function-based controller for kinematic and dynamic fully actuated robots without spurious minima. We sequentially compose these controllers to drive the state to the goal. For a polygonal space, the algorithm we propose is complete. We present successful simulation results of the algorithm on a group of ground vehicles and quadrotors performing a cooperative navigation task in constrained environments.

This paper presents the locomotion approach of a novel quadruped robot which is able to carry various effectors for achieving manufacturing tasks in large workspaces. Equipped with lockers on some of the passive joints and clamping devices at the end of its limbs, this quadruped uses eight actuators for achieving manufacturing tasks as well as locomotion tasks. In the following sections, we first present the proposed robot and its two working modes. Then, the locking strategy of the robot is formulated as an optimization problem. Also, a practical method for managing the limbs swinging movement is addressed. At last, the presented approach is applied on two concrete examples. Possessing a low degree of kinematic redundancy, the proposed quadruped shows a reasonable locomotion capacity which allows it to achieve locomotion with respect to some extra constrains in its workspaces.

In this paper, we formulate a distributed, energy-aware control policy aimed at enabling persistent surveillance of a specified region of interest by teams of networked robots. Central to our formulation is the fundamental idea that as an agent participating in coverage approaches a low energy reserve the team should cooperatively adjust the coverage formation to allow the agent to return to a designated base station, where it can recharge before rejoining the effort. Towards this end, we build upon recent efforts in employing Centroidal Voronoi Tessellation (CVT)-based coverage control laws by defining a policy that exploits a power-dependent weighting scheme that embeds an agent's trade-off to achieve its coverage mission and to maintain a desired energy reserve to guarantee its own safety. Stability of the proposed approach is considered, and we show that coupling our continuous controller with a straightforward switching mechanism guarantees every agent will return to its base station safely. Simulation results are presented to verify and demonstrate the utility of the proposed control scheme.

This paper addresses mechanics, design, estimation and control for aerial grasping. We present the design of several light-weight, low-complexity grippers that allow quadrotors to grasp and perch on branches or beams and pick up and transport payloads. We then show how the robot can use rigid body dynamic models and sensing to verify a grasp, to estimate the the inertial parameters of the grasped object, and to adapt the controller and improve performance during flight. We present experimental results with different grippers and different payloads and show the robot's ability to estimate the mass, the location of the center of mass and the moments of inertia to improve tracking performance.

Group food retrieval in some ant species serves as a useful paradigm for multirobot collective transport strategies that are decentralized, scalable, and do not require a priori information about the payload. We present a comprehensive overview of group retrieval in ants and investigate this phenomenon in Aphaenogaster cockerelli in order to extract the ants' roles during transport, the rules that govern their actions, and the individual forces that they apply to guide a food item to their nest. To measure these forces, we fabricated elastic structures with calibrated stiffness properties, induced ants to retrieve the structures, and tracked the resulting deformations with a camera. We then developed a hybrid system model of the ant behaviors that were observed in the experiments. We conducted simulations of the behavioral model that incorporate a quasi-static model of planar manipulation with compliant attachment points. Our simulations qualitatively replicate individual ant activity as well as certain macroscopic features of the transport.

There are many examples in nature where large groups of individuals are able to maintain 3-D formations while navigating in complex environments. This paper addresses the development of a framework and robot controllers that enable a group of aerial robots to maintain a formation with partial state information while avoiding collisions. The central concept is to develop a low-dimensional abstraction of the large teams of robots, facilitate planning, command, and control in a low-dimensional space, and to realize commands or plans in the abstract space by synthesizing controllers for individual robots that respect the specified abstraction. We derive the basic theory and present experimental and simulation results with a team of rotor crafts.

This paper describes a test-bed for planar micro and mesoscale manipulation tasks and a framework for planning based on quasi-static models of mechanical systems with intermittent frictional contacts. We show how planar peg-in-the-hole assembly tasks can be designed using randomized motion planning techniques with Mason's models for quasi-static manipulation. Simulation and experimental results are presented in support of our methodology. We develop this further into a systematic approach to incorporating uncertainty into planning manipulation tasks with frictional contacts. We again consider the canonical problem of assembling a peg into a hole at the mesoscale using probes with minimal actuation but with visual feedback from an optical microscope. We consider three sources of uncertainty. First, because of errors in sensing position and orientation of the parts to be assembled, we must consider uncertainty in the sensed configuration of the system. Second, there is uncertainty because of errors in actuation. Third, there are geometric and physical parameters characterizing the environment that are unknown. We discuss the synthesis of robust planning primitives using a single degree-of-freedom probe and the automated generation of plans for mesoscale manipulation. We show simulation and experimental results of our work.

In this study, we consider the problem of safely steering a planar nonholonomic cart around obstacles to reach a goal state. We achieve this by the decomposition of the free workspace into triangular tori and generation of local smooth feedback laws that drive the robot from one cell to an adjoining cell. These control laws exploit the fact that for nonholonomic systems, one can generate smooth controllers to reach a particular subset in the configuration space, even though smooth feedback laws cannot be obtained to reach a particular state. These local controllers are then sequenced using discrete motion planning algorithms like A* or incremental D* to reach the goal. We demonstrate the practical efficacy of this methodology by applying it to two experimental platforms: (1) a differential drive robot in which inertial effects are negligible and (2) a hexapedal robot in which inertial effects are significant but difficult to model. In both cases, we use the abstraction of a planar kinematic cart with process noise to develop feedback controllers. We present successful implementation of the controllers to navigate the hexapedal robot in both static and dynamic environments with obstacles.

We address the problem of controlling a formation of robots in a cluttered environment. Instead of explicitly controlling the relative positions between the robots and the environment, we construct a lower-dimensional abstraction of the group that establishes a boundary for the group. We then synthesize feedback controllers that allow the abstracted group to navigate a two-dimensional environment to a desired goal position, while automatically adapting the shape of the boundary as well as the position and orientation of the group to avoid collisions between the virtual boundary and the environment. In contrast to previous approaches, we address the planning and control problems concurrently and are naturally able to establish bounds on the positions of the robots through the abstraction. The complexity of the method is independent of the number of robots which promises scalability to large teams.

Abstract — The DPC algorithm developed in our previous work is an efficient way of computing optimal trajectories for multiple robots in a distributed fashion with timeparameterized constraints on the distances between pairs of robots. In the present work we extend DPC to the problem of multiple task execution. While this extended problem inherits all the objectives, complexities and constraints of the basic DPC algorithm, each robot is also given an unordered set of tasks that it has to execute before it reaches its goal. There is no specific order imposed on the tasks assigned to a particular robot. The algorithm decides the order of execution of the tasks such that an optimal solution is attained while the time-parametrized distance constraints are satisfied along with successful execution of the tasks. We solve this problem by designing a “State-task Graph ” that represents a product of the state-space graph and the task graph. We then develop an efficient heuristic function for performing searches in this graph. I.

In this note, we present initial results towards developing a distributed algorithm for repairing topological holes in the sensor cover of a mobile robot team. Central to our approach is the melding of recent advances in the application of computational homology (a sub-discipline of algebraic topology) to static sensor networks with relative metric information (i.e. relative pose). More precisely, we consider a greedy, hybrid (discrete-continuous) algorithm whereby a desired Cěch complex, the simplicial complex that captures the underlying topology of the sensing cover, is iteratively generated using local rules (between multi-hop neighbors) and agents are driven towards achieving this topology via a gradient-ascent simplicial control law. Convergence of the proposed algorithm is established as a function of the convergence of the underlying simplicial control law, and the relationship of the latter to the spectrum of the combinatorial Laplacian is considered. Simulation results for teams operating in ℝ2are presented.

In this paper we explore methods for the online mapping of received radio signal strength with mobile robots and localizing the source of the radio signal. By utilizing Gaussian processes, we are able to build an online model of the signal-strength map that can, in turn, be used to provide the current maximum likelihood estimate of the source location. Furthermore, using the estimate of the source location, the Gaussian process model allows for prediction of received signal strength with confidence bounds in regions of the environment that have not been explored. Finally, we develop a control law for collecting samples of the signal strength with mobile robots that allows for online estimation of the radio signal source.

Different robotic tasks can be solved by controlling a robot to circulate along curves. These include, for example, border inspection and surveillance, multirobot manipulation, and pattern generation. In a previous, work we have proposed a vector field approach for robot convergence and circulation along time-varying curves embedded in N-dimensional spaces. In the present work we instantiate this approach for three-dimensional spaces and, for the first time, show the efficacy of this method to control actual robots. Besides new theoretical analysis when constant speed control is applied, we present experimental results with aerial (quadrotors) and ground (differential-driven) robot.

This paper describes a motion planning algorithm that accounts for uncertainty in the dynamics of vehicles. This noise is a function of the type of controller employed on the vehicle and the characteristics of the terrain and can cause the robot to deviate from a planned trajectory and collide with obstacles. Our motion planning algorithm finds trajectories that balance the trade-off between conventional performance measures such as time and energy versus safety. The key is a characterization of the vehicle's ability to follow planned paths, which allows the algorithm to explicitly calculate probabilities of successful traversal for different trajectory segments. We illustrate the method with a six-legged Rhex-like robot by experimentally characterizing different gaits (controllers) on different terrains and demonstrating the hexapod navigating a multi-terrain environment.

In this paper, we describe how signaling networks and actuation in bacterial cells and biomolecular networks of bacteria can be used to develop an integrated micro-bio-robotic system. SU8 microstructures blotted with swarmer cells of Serratia Marcescens in a monolayer are propelled by the bacteria in the absence of any environmental stimulus. We call such microstructures with bacteria Micro Bio Robots (MBRs) and the uncontrolled motion in the absence of stimuli self actuation. Our paper has two primary contributions. First, we demonstrate the control of MBRs using self-actuation, DC electric fields and ultra-violet radiation, and develop experimentally validated mathematical model for the MBRs. This model allows us to use self-actuation and electrokinetic actuation to steer the MBR to any position and orientation in a planar micro channel. Second, we describe the development of biosensors for the MBRs. This is done by attaching genetically engineered Escherichia coli cells that are capable of sensing nonmetabolizable lactose analog methyl-β-D-thiogalactoside (TMG). We describe the fabrication process for MBRs and show experimental results demonstrating sensing, actuation and control.

Goal-directed path planning is one of the basic and widely studied problems in the field of mobile robotics. Homotopy classes of trajectories, arising due to the presence of obstacles, are defined as sets of trajectories that can be transformed into each other by gradual bending and stretching without colliding with obstacles. The problem of finding least-cost paths restricted to a specific homotopy class or finding least-cost paths that do not belong to certain homotopy classes arises frequently in such applications as predicting paths for dynamic entities and computing heuristics for path planning with dynamic constraints. In the present work, we develop a compact way of representing homotopy classes and propose an efficient method of graph search-based optimal path planning with constraints on homotopy classes. The method is based on representing the environment of the robot as a complex plane and making use of the Cauchy Integral Theorem. We prove optimality of the method and show its efficiency experimentally.

We propose a method for synthesizing decentralized feedback controllers for a team of multiple heterogeneous agents navigating a known environment with obstacles. The controllers are designed to drive agents with limited team state information to goal sets while avoiding collisions and maintaining specified proximity constraints. The method, its successful application to nonholonomic agents in dynamic simulation and experimentation, and its limitations are presented in this paper.

We present a two-dimensional, vision-based force sensor, capable of sensing muN level forces. There are currently no reliable, off-the-shelf, commercially-available force sensors to measure forces at this scale, that can be easily integrated into standard microrobotic test-beds. Our design consists of a planar, elastic mechanism with known force-deflection characteristics. A CCD camera is used to track the deformation of the mechanism as it is used to manipulate objects in a micro/meso-scale robotic manipulation test-bed. By observing the displacements of select points in the mechanism, the manipulation forces can be estimated. The modeling, design, microfabrication, calibration and experimental validation of the force sensor are presented with a brief discussion of an application to a robotic manipulation tasks such as microassembly.

This paper presents a systematic analysis of the motion of microscale structures actuated by flagellated bacteria. We perform the study both experimentally and theoretically. We use a blotting procedure to attach flagellated bacteria to a buoyancy-neutral plate called a microbarge. The motion of the plate depends on the distribution of the cells on the plate and the stimuli from the environment. We construct a stochastic mathematical model for the system, based on the assumption that the behavior of each bacterium is random and independent of that of its neighbors. The main finding of the paper is that the motion of the barge plus bacteria system is a function of a very small set of parameters. This reduced-dimensional model can be easily estimated using experimental data. We show that the simulation results obtained from the model show an excellent match with the experimentally-observed motion of the barge.

We present a decentralized, scalable approach to assembling a group of heterogeneous parts into different products using a swarm of robots. While the assembly plans are predetermined, the exact sequence of assembly of parts and the allocation of subassembly tasks to robots are determined by the interactions between robots in a decentralized fashion in real time. Our approach is based on developing a continuous abstraction of the system derived from models of chemical reactions and formulating the strategy as a problem of selecting rates of assembly and disassembly. These rates are mapped onto probabilities that determine stochastic control policies for individual robots, which then produce the desired aggregate behavior. This top-down approach to determining robot controllers also allows us to optimize the rates at the abstract level to achieve fast convergence to the specified target numbers of products. Because the method incorporates programs for assembly and disassembly, changes in demand can lead to reconfiguration in a seamless fashion. We illustrate the methodology using a physics-based simulator with examples involving 15 robots and two types of final products.

We present a novel approach to controlling the locomotion of a wheel by changing its shape, leading to applications to the synthesis and closed-loop control of gaits for modular robots. A dynamic model of a planar, continuous deformable ellipse in contact with a ground surface is derived. We present two alternative approaches to controlling this system and a method for mapping the gaits to a discrete rolling polygon. Mathematical models and dynamic simulation of the continuous approximation and the discrete n-body system, and experimental results obtained from a physical modular robot system illustrate the accuracy of the dynamic models and the validity of the approach.

We study radio signal propagation in indoor environments using low-power devices leveraging the Zigbee and Bluetooth specifications. We present results from experiments where two robots equipped with radio signal devices and enabled to control and localize autonomously in an indoor hallway and laboratory environment densely sample RSSI at various times over several days. We show that simulated RSSI measurements using existing radio signal models and experimentally gathered RSSI measurements match closely, suggesting that for robotics applications requiring predicted RSSI, low-power radio signal devices are a well-posed sensing modality.

We present a scalable approach to dynamically allocating a swarm of homogeneous robots to multiple tasks, which are to be performed in parallel, following a desired distribution. We employ a decentralized strategy that requires no communication among robots. It is based on the development of a continuous abstraction of the swarm obtained by modeling population fractions and defining the task allocation problem as the selection of rates of robot ingress and egress to and from each task. These rates are used to determine probabilities that define stochastic control policies for individual robots, which, in turn, produce the desired collective behavior. We address the problem of computing rates to achieve fast redistribution of the swarm subject to constraint(s) on switching between tasks at equilibrium. We present several formulations of this optimization problem that vary in the precedence constraints between tasks and in their dependence on the initial robot distribution. We use each formulation to optimize the rates for a scenario with four tasks and compare the resulting control policies using a simulation in which 250 robots redistribute themselves among four buildings to survey the perimeters.

We propose a method for synthesizing decentralized feedback controllers for a team of multiple heterogeneous agents navigating a known environment with obstacles. The controllers are designed to drive agents with limited team state information to goal sets while avoiding collisions and maintaining specified proximity constraints. The method, its successful application to nonholonomic agents in dynamic simulation, and its limitations are presented in this paper.

Manipulation and assembly tasks are typically characterized by many nominally rigid bodies coming into frictional contacts, possibly involving impacts. Manipulation tasks are difficult to model because uncertainties associated with friction and assembly tasks are particularly hard to analyze because of the interplay between process tolerance and geometric uncertainties due to manufacturing errors. Manipulation at the meso (hundred microns to millimeters) and micro (several microns to tens of microns) scale is even harder for several reasons. It is difficult to measure forces at the micro-netwon level reliably using off-the-shelf force sensors and good force-feedback control schemes have not proved successful. It is hard to manufacture general-purpose end effectors at this scale and it is even more difficult to grasp and manipulate parts at the micro and meso level than it is at the macro level. Finally, the lack of good models of the mechanics of contact interactions at this scale means that model-based approaches to planning and control are difficult.

In this paper, we present a scalable, decentralized task allocation algorithm for a group of unknown number of unmanned aerial vehicles (UAVs), which are equipped with GPS receivers, synchronized clocks and radars with a finite, but known operating distance to identify neighbors. The algorithm assigns subgroups of UAVs, whose initial positions are randomly scattered in a bounded space, to a finite set of independent tasks. The key features of the proposed algorithm are: (1) the algorithm does not require any communication between the UAVs; (2) the task allocation is achieved in finite time. The analysis and results in the simplified 2D simulation environment respectively prove and verify the correctness of the proposed algorithm.

We present a decentralized approach to multi- robot manipulation where the team of robots surround and trap an object and transport it, by dragging or pushing, to the goal configuration in an environment with obstacles. The proposed feedback controllers are obtained by sequentially composing vector fields or behaviors and are decentralized in the sense that robots do not exchange each other's state information. Rather, cooperative manipulation is achieved by relying solely on each robot's local information and a global knowledge of the task. We present computer simulations and experimental results obtained using our multi-robot testbed.

Distributed task assignment for multiple agents raises fundamental and novel problems in control theory and robotics. A new challenge is the development of distributed algorithms that dynamically assign tasks to multiple agents, not relying on a priori assignment information. We address this challenge using market-based coordination protocols where the agents are able to bid for task assignment with the assumption that every agent has knowledge of the maximum number of agents that any given task can accommodate. We show that our approach always achieves the desired assignment of agents to tasks after exploring at most a polynomial number of assignments, dramatically reducing the combinatorial nature of discrete assignment problems. We verify our algorithm through both simulation and experimentation on a team of non-holonomic robots performing distributed formation stabilization and group splitting and merging.

We address the problem of pattern generation in obstacle-filled environments by a swarm of mobile robots. Decentralized controllers are devised by using the Smoothed Particle Hydrodynamics (SPH) method. The swarm is modelled as an incompressible fluid subjected to external forces. Actual robot issues such as finite size and nonholonomic constraints are also addressed. Collision avoidance guarantees are discussed. Finally, in the absence of obstacles, we prove for the first time stability and convergence of controllers based on the SPH.

We develop a framework for controlling a team of robots to maintain and improve a communication bridge between a stationary robot and an independently exploring robot in a walled environment. We make use of two metrics for characterizing the communication: the Fiedler value of the weighted Laplacian describing the communication interactions of all the robots in the system, and the k-connectivity matrix that expresses which robots can interact through k or less intermediary robots. At each step, we move in such a way as to improve the Fiedler value as much as possible while keeping the number of intermediary robots between the two robots of interest below a desired value. We demonstrate the use of this framework in a scenario where the hop-count constraint cannot be satisfied, but show that communication quality is maintained anyways.

In this paper, we study the time-optimal trajectory planning of a sensor attached to an unmanned aerial vehicle (UAV) to provide complete 3-dimensional coverage with applications to urban environments with 2.5-dimensional features. The basic approach is to approximate the features of interest with a set of non planar coverage surfaces and to design a motion plan that guarantees the coverage surface is swept completely with a conical-field-of-view sensor. We establish a lower bound on time for a UAV to achieve complete coverage and derive the analytical coverage plan whose duration is a constant times this lower bound. Our hardware-in-the-loop simulation results verify the effectiveness of the proposed algorithm.

This paper discusses the design and current capabilities of a new software tool, dVC, capable of simulating planar systems of bodies experiencing unilateral contacts with friction. Since different problems require different levels of accuracy, dVC provides user-selectable body types (rigid or locally-compliant), motion models (first-order, quasi-static, dynamic), and several state-of-the-art time-stepping methods. One can also choose to include friction between each body and the plane of motion. To support optimal and robust part design, dVC also allows on-the-fly changes to parameters of the geometric and physical models. The results obtained for three representative planar problems are presented: the design of a passive part-orienting device, the planning of a mesoscale assembly operation, and the design of a grasp strategy.

We present a methodology for characterizing and synthesizing swarm behaviors using both a macroscopic model that represents a swarm as a continuum and a microscopic model that represents individual robots. We develop a systematic approach for synthesizing behaviors at the macroscopic level that can be realized on individual robots at the microscopic level. Our methodology is inspired by a dynamical model of ant house hunting [1], a decentralized process in which a colony attempts to emigrate to the best site among several alternatives. The model is hybrid because the colony switches between different sets of behaviors, or modes, during this process. At the macroscopic level, we are able to synthesize controllers that result in the deployment of a robotic swarm in a predefined ratio between distinct sites. We then derive hybrid controllers for individual robots using only local interactions and no communication that respect the specifications of the global continuous behavior. Our simulations demonstrate that our synthesis procedure yields a correct microscopic model from the macroscopic description with guarantees on performance at both levels

Classical path planning does not address many of the challenges of robotic systems subject to differential constraints. While there have been many recent efforts to develop motion planning algorithms for systems with differential constraints (MPD), very little has been said about the existence of exact algorithms. In other words, the decidability of MPD problems is still an open question. In this paper, we propose a partial answer to this question limiting ourselves to special cases where the trajectory functions of the systems under the finite-dimensional piecewise-continuous controls have a closed-form polynomial formulation. We define an abstract formulation for the MPD problem based on the concept of a control space. We provide an incremental decision algorithm to answer the decidability question and present sufficient conditions for problems to which this algorithm can be applied. Decidability results for several non trivial MPD problems are presented. For example, we show that the question of existence of a trajectory for a Dubin's car with a polygonal rigid body between two specified positions and orientations in a polygonal environment with a fixed and finite number of discontinuities in curvature is decidable.

We develop decentralized controllers for a team of disk-shaped robots to converge to and circulate along the boundary of a desired two-dimensional geometric pattern specified by a smooth function with collision avoidance. The proposed feedback controllers rely solely on each robot's range and bearing sensors which allow them to obtain information about positions of neighbors within a given range. This is relevant for applications such as perimeter surveillance or containing hazardous regions where limited bandwidth must be preserved for situational awareness. The computational complexity of the decentralized controller for each agent is linear in the number of neighboring agents, making it scalable to robot swarms. We establish stability and convergence properties of the controllers and verify the feasibility of the method through computer simulations.

While there is extensive work on motion planning and control for navigation tasks with guarantees, there is no systematic way for human operators to modify the resulting plans without losing the guarantees. In this paper we propose a systematic way of composing behaviors resulting from human inputs with behaviors derived from navigation functions. The proposed controller is based on a new class of navigation function based controllers that possess weak input-to-state stability properties. The resulting system has analytically guaranteed safety and convergence properties. The feasibility of the proposed methodology is demonstrated through simulation examples and hardware experiments

We present an approach for the dynamic assignment and reassignment of a large team of homogeneous robotic agents to multiple locations with applications to search and rescue, reconnaissance and exploration missions. Our work is inspired by experimental studies of ant house hunting and empirical models that predict the behavior of the colony that is faced with a choice between multiple candidate nests. We design stochastic control policies that enable the team of agents to distribute themselves between multiple candidate sites in a specified ratio. Additionally, we present an extension to our model to enable fast convergence via switching behaviors based on quorum sensing. The stability and convergence properties of these control policies are analyzed and simulation results are presented.

We consider the task of controlling a large team of nonholonomic ground robots with an unmanned aerial vehicle in a decentralized manner that is invariant to the number of ground robots. The central idea is the development of an abstraction for the team of ground robots that allows the aerial platform to control the team without any knowledge of the specificity of individual vehicles. This happens in much the same way as a human operator can control a single robot vehicle by simply commanding the forward and turning velocities without a detailed knowledge of the specifics of the robot. The abstraction includes a gross model of the shape of the formation of the team and information about the position and orientation of the team in the plane. We derive controllers that allow the team of robots to move in formation while avoiding collisions and respecting the abstraction commanded by the aerial platform. We provide simulation and experimental results using a team of indoor mobile robots and a three-dimensional, cable-controlled, parallel robot which serves as our indoor unmanned aerial platform.

This paper describes a test-bed for planar micro manipulation tasks and a framework for planning based on quasi-static models of mechanical systems with frictional contacts. We show how planar peg-in-the-hole assembly tasks can be designed using randomized motion planning techniques with Mason's models for quasi-static manipulation. Finally, we present simulation and experimental results in support of our methodology

The roller racer, an undulatory locomotion system, is a toy which can be propelled forward by sitting on it and only oscillating the steering handle. A nonholonomic dynamic model and controllability analysis of the roller racer was first published by Krishnaprasad and Tsakiris in 1998. The model is derived from the usual assumption that all the wheels obey sticking (non-slipping) constraints, i.e., rolling without slipping. Controllability analysis shows that under these assumptions, the roller racer cannot be stopped once started. Yet physical prototypes do not exhibit this characteristic. In this paper, a high-fidelity model of the roller racer is presented by considering the finite static friction between the wheels and the ground, i.e., slipping will occur when constraint force exceeds the maximal allowable frictional force. It is proved that the system could be stopped from any state with only the steering angle control. Furthermore, based on group symmetry and motion primitives, a planner is designed to achieve motions between any two given positions and orientations with zero velocities. Experiments also show that front wheel slipping stops the system faster than joint frictions

Communication is essential for coordination in most cooperative control and sensing paradigms. In this paper, we present an experimental study of strategies for maintaining end-to-end communication links for tasks such as surveillance and search and rescue where team connectivity is essential for providing situational awareness to a base station. We consider the differences between monitoring point-to-point signal strength versus data throughput and present experimental results with our multi-robot testbed in outdoor environments

We develop decentralized controllers for a swarm of robots to generate a desired two-dimensional geometric pattern specified by a smooth function while maintaining specified relative distance constraints. The controllers are decentralized in the sense that the robots do not exchange or sense each other's state information. However, we assume that the robots have range sensors allowing them to obtain information about distances to neighbors within a known range. We establish stability and convergence properties of the controllers

We address the problem of modeling and controlling a swarm of fully actuated point-like robots in three dimensions by generalizing the planar framework from (C. Belta and V. Kumar, 2004). We define a nine-dimensional abstraction of the swarm that has a product structure of the six-dimensional Euclidean group and a three-dimensional shape, and is independent of the number of robots. The group captures the pose of an ellipsoid spanning the swarm with semiaxes given by the shape variables. The overall abstract description is invariant to robot permutations. In addition, the shape is also invariant to left actions of the group. This description allows one to define and control the behavior of the swarm at a high level, with automatic generation of individual robot control laws. We present simulation results for controlling swarms of rotorcrafts

We address the synthesis of controllers for large groups of robots and sensors, tackling the specific problem of controlling a swarm of robots to generate patterns specified by implicit functions of the form s(x, y) = 0. We derive decentralized controllers that allow the robots to converge to a given curve S and spread along this curve. We consider implicit functions that are weighted sums of radial basis functions created by interpolating from a set of constraint points, which give us a high degree of control over the desired 2D curves. We describe the generation of simple plans for swarms of robots using these functions and illustrate our approach through simulations and real experiments.

The TRIKKE is a three-wheeled, human-powered scooter that can be propelled by a combination of cyclic motion of its handlebar and swaying motion of the rider. This paper addresses the modeling, dynamics and control of the TRIKKE and the development of a robotic platform called the ROBOTRIKKE that is derived from similar principles. The TRIKKE can be modeled as a modified roller-racer with an unstable steering arrangement. The model of the TRIKKE reduces to the roller-racer [8] in the absence of this steering arrangement. We prove that under certain conditions on the geometric parameters of the system, the TRIKKE and roller-racer systems cannot be stopped after motion starting from rest using the steering control as the sole input. As a consequence, the ideal model is severely limited from the point of view of controllability. We demonstrate the validity of our model through comparison with experimental measurements on a small-scale robotic prototype of the TRIKKE. We present closed-loop control results for tracking (on average) a straight line trajectory using visual feedback from an overhead camera.

This paper concerns the problem of actively searching for and localizing ground features by a coordinated team of air and ground robotic sensor platforms. The approach taken builds on well known Decentralized Data Fusion (DDF) methodology. In particular, it brings together established representations developed for identification and linearized estimation problems to jointly address feature detection and localization. This provides transparent and scalable integration of sensor information from air and ground platforms. As in previous studies, an Information-theoretic utility measure and local control strategy drive the robots to uncertainty reducing team configurations. Complementary characteristics in terms of coverage and accuracy are revealed through analysis of the observation uncertainty for air and ground on-board cameras. Implementation results for a detection and localization example indicate the ability of this approach to scalably and efficiently realize such collaborative potential.

We describe the development and assessment of a computer controlled wheelchair called the SMARTCHAIR. A shared control framework with different levels of autonomy allows the human operator to stay in complete control of the chair at each level while ensuring her safety. The framework incorporates deliberative motion plans or controllers, reactive behaviors, and human user inputs. At every instant in time, control inputs from these three different sources are blended continuously to provide a safe trajectory to the destination, while allowing the human to maintain control and safely override the autonomous behavior. In this paper, we present usability experiments with 50 participants and demonstrate quantitatively the benefits of human-robot augmentation.

In this work, we present a framework for the development of a planar mobile robot controller based on image plane feedback. We show that the design of such a motion controller can be accomplished in the image plane by making use of a subset of the parameters that relate the image plane to the ground plane, while still leveraging the simplifications offered by modeling the system as a differentially flat system. Our method relies on a waypoint-based trajectory generator, with all the waypoints specified in the image, as seen by an overhead observer. We present some results from simulation as well as from experiments that validate the ideas presented in this work and discuss some ideas for future work.

This paper addresses the problem of coordinating aerial and ground vehicles in tasks that involve exploration, identification of targets and maintaining a connected communication network. We focus on the problem of localizing vehicles in urban environments where GPS signals are often unreliable or unavailable. We first describe our multi-robot testbed and the control software used to coordinate ground and aerial vehicles. We present the results of experiments in air-ground localization analyzing three complementary approaches to determining the positions of vehicles on the ground. We show that the coordination of aerial vehicles with ground vehicles is necessary to get accurate estimates of the state of the system.

We present the design and gait generation for an experimental ROLLERBLADER. The ROLLERBLADER is a robot with a central platform mounted on omnidirectional casters and two 3 degree-of-freedom legs. A passive rollerblading wheel is attached to the end of each leg. The wheels give rise to nonholonomic constraints acting on the robot. The legs can be picked up and placed back on the ground allowing a combination of skating and walking gaits. We present two types of gaits for the robot. In the first gait, we allow the legs to be picked up and placed back on the ground while in the second, the wheels are constrained to stay on the ground at all tunes. Experimental gait results for a prototype robot are also presented.

Communication is essential for coordination in most cooperative control and sensing paradigms. In this paper, we investigate the construction of a map of radio signal strength that can be used to plan multirobot tasks and also serve as useful perceptual information. We show how nominal models of an urban environment, such as those obtained by aerial surveillance, can be used to generate strategies for exploration and present preliminary experimental results with our multi-robot testbed.

We describe the development and assessment of a computer controlled wheelchair equipped with a suite of sensors and a novel interface, called the SMARTCHAIR. The main focus of this paper is a shared control framework which allows the human operator to interact with the chair while it is performing an autonomous task. At the highest level, the autonomous system is able to plan paths using high level deliberative navigation behaviors depending on destinations or waypoints commanded by the user. The user is able to locally modify or override previously commanded autonomous behaviors or plans. This is possible because of our hierarchical control strategy that combines three independent sources of control inputs: deliberative plans obtained from maps and user commands, reactive behaviors generated by stimuli from the environment, and user-initiated commands that might arise during the execution of a plan or behavior. The framework we describe ensures the user's safety while allowing the user to be in complete control of a potentially autonomous system.

We introduce computational support tools for the analysis and design of systems with multiple frictional contacts, with a focus on applications to part feeding and assembly processes. The tools rely on dynamic models of the processes. We describe two approaches to modeling, the Stewart-Trinkle model (1996) and the Song-Pang-Kumar model (2003), that allow the designer to experiment with different geometric, material and dynamic properties and optimize the design for performance. In order to accommodate contact transitions, we introduce a smooth cone model for friction. We illustrate the models and the design process by describing the design optimization of a part feeder.

In this paper, we consider the problem of cooperatively localizing a formation of networked mobile robots/vehicles in SE(2), and adapting the formation to reduce localization errors. First, we propose necessary and sufficient conditions to establish when a team of robots with heterogeneous sensors can be completely localized. We present experimental measurements of range and bearing with omni-directional cameras to motivate a simple model for noisy sensory information. We propose a measure of quality of team localization, and show how this measure directly depends on a sensing graph. Finally, we show how the formation and the sensing graph can be adapted to improve the measure of performance for team localization and for localization of targets through experiments and simulations.

The paper investigates the stability properties of mobile agent formations which are based on leader following. We derive nonlinear gain estimates that capture how leader behavior affects the interconnection errors observed in the formation. Leader-to-formation stability (LFS) gains quantify error amplification, relate interconnection topology to stability and performance, and offer safety bounds for different formation topologies. Analysis based on the LFS gains provides insight to error propagation and suggests ways to improve the safety, robustness, and performance characteristics of a formation.

This paper shows how a large number of robots can be coordinated by designing control laws on a small dimensional manifold, independent on the number and ordering of the robots. The small dimensional description of the team has a product structure of a Lie group, which captures the dependence of the ensemble on world frame, and a shape manifold, which is an intrinsic description of the team. We design decoupled controls for group and shape. The individual control laws which are mapped to the desired collective behavior can be realized by feedback depending only on the current state of the robot and the state on the small dimensional manifold, so that the robots have to broadcast their states and only have to listen to some coordinating agent with small bandwidth.

This paper proposes a methodology that uses hybrid systems to model multiple robots in the execution of cooperative tasks. Basically, each robot is represented by a hybrid automation and the cooperative task execution is modeled by the composition of several automata. We describe in details our approach to perform the composition of these automata and demonstrate the effectiveness of the proposed methodology modeling a cooperative manipulation task.

This paper addresses the problem of multi-robots object transportation by using the concept of object closure. Once object closure is achieved, the robots can cooperatively drag or flow the trapped object to the desired goal. In this paper, we address the concept CC-closure object used for efficient testing of object closure. Properties of the CC-closure object and testing algorithm are described. Finally, an example of object closure constructed from both bodies and hands of mobile-manipulators is discussed and an experiment is presented for illustrating the proposed concept.

We develop the dynamic model for a planar rollerblader. The robot consists of a rigid platform and two planar, two degree-of-freedom legs with in-line skates at the foot. The dynamic model consists of two unicycles coupled through the rigid body dynamics of the planar platform. We derive the Lagrangian reduction for the rollerblading robot. We show the generation of some simple gaits that allow the platform to move forward and rotate by using cyclic motions of the two legs.

Robot programmers can often quickly program a robot to approximately execute a task under specific environment conditions. However, achieving robust performance under more general conditions is significantly more difficult. We propose a framework that starts with an existing control system and uses reinforcement feedback from the environment to autonomously improve the controller's performance. We use the policy gradient reinforcement learning (PGRL) framework, which estimates a gradient (in controller space) of improved reward, allowing the controller parameters to be incrementally updated to autonomously achieve locally optimal performance. Our approach is experimentally verified on a Cye robot executing a room entry and observation task, showing significant reduction in task execution time and robustness with respect to un-modelled changes in the environment.

Robotic airships have several beneficial properties such as low operation cost, low noise, and low speed flight capability. We present in this paper the design and verification of a feedback control algorithm for waypoint to waypoint navigation of an outdoor blimp. A rapidly exploring random tree (RRT) is used for the validation of the blimp system control law. We describe an implementation of an algorithm that systematically searches the set of all disturbances to validate viability of the control law in the presence of winds. Experimental results with a simulator show that the RRT method can be effective in verifying controller design under unpredictable but bounded disturbances.

We apply a novel motion planning and control methodology, which is based on a non-smooth navigation function, to a point mobile robot moving amongst moving obstacles. The chattering introduced by the discontinuous potential field is suppressed using non-smooth backstepping. The combined controller guarantees global asymptotic convergence and collision avoidance. This controller is particularly suitable for real time implementation on systems with limited computational resources. The effectiveness of the proposed scheme is verified through computer simulations.

We build on previous work, on the development of a computer controlled wheelchair equipped with a suite of sensors and a novel interface for human-robot interaction. In this paper, we present experimental results and usability studies for the wheelchair. The architecture for human-robot interaction is hierarchical, with the lowest level corresponding to trajectory control, the intermediate level being behavioral and the highest level involving the composition of behaviors and navigation. Our experimental results illustrate the benefits of a shared-control paradigm where the human operator selects the appropriate behavior(s) or goals while the software is responsible for executing behaviors and generating safe trajectories. Experiments with human users highlight advantages of augmentation in wheelchairs.

We address the problem of controlling a team of robots subject to constraints on relative positions. We adopt the general framework of leader-follower control in which a network of controllers is used to control the position and orientation of the team and its shape. We propose two improvements to this scheme. First, we introduce cooperative leader-following where the motion of a robot is determined not only by its leader, but also by other robots including their followers. Second, we allow constraints that are induced by limitations on ranges of sensors and wireless network cards. Our approach is based on potential field controllers for each robot and the on-line modification of these controllers to accommodate motion constraints induced by other robots in the group. We present experimental results with a team of three car-like robots equipped with omnidirectional cameras and 802.11b network cards.

In this paper the problem of controlling the motion of a nonholonomic vehicle along a desired trajectory using observations from an overhead camera is considered. The control problem is formulated in the image plane. We show that the system in the image plane is differentially flat and use this property to generate effective control strategies using only visual feedback. Simulation results illustrate the methodology and show robustness to errors in the camera calibration parameters.

This paper describes the modeling language CHARON for modular design of interacting hybrid systems. The language allows specification of architectural as well as behavioral hierarchy and discrete as well as continuous activities. The modular structure of the language is not merely syntactic, but is exploited by analysis tools and is supported by a formal semantics with an accompanying compositional theory of refinement. We illustrate the benefits of CHARON in the design of embedded control software using examples from automated highways concerning vehicle coordination.

We develop a method for generating smooth trajectories for a set of mobile robots. Given two end configurations, by tuning one parameter, the user can choose an interpolating trajectory from a continuum of curves varying from that corresponding to maintaining a rigid formation to motion of the robots toward each other The idea behind our method is to change the original constant kinetic energy metric in the configuration space and can be summarized into three steps. First, the energy of the motion as a rigid structure is decoupled from the energy of motion along directions that violate the rigid constraints. Next, the metric is "shaped" by assigning different weights to each term. Finally, geodesic flow is constructed for the modified metric. The optimal motions generated on the manifolds of rigid body displacements in 3D space or in plane, and the uniform rectilinear motion of each robot corresponding to a totally uncorrelated approach are particular cases of our general treatment.

Proposes a methodology for coordinating multi-robot teams in the execution of cooperative tasks. It is based on a dynamic role-assignment mechanism in which the robots assume and exchange roles during cooperation. We model the role assignment under a hybrid systems framework, using a hybrid automaton to represent roles, transitions and controllers. Using a multi-robot simulator, the methodology is demonstrated in a cooperative transportation task, in which a group of robots must find and cooperatively transport several objects scattered in the environment.

The problem of motion planning in environments with both known static obstacles and unpredictable dynamic constraints is considered. A methodology is introduced in which the motion plan for the static environment is modified on-line to accommodate the unpredictable constraints in such a way that the completeness properties of the original motion plan are preserved. At the heart of the approach is the idea that navigation functions are indeed Lyapunov functions; and that the traditional method of forcing the robot to track the negative gradient of field is not the only input which stabilizes the system. This extra freedom in selecting the input is used to accommodate the dynamic constraints. A computational method for selecting the appropriate inputs is given. The method is used to solve two sample problems. The constraints in these cases are used to model collisions with other robots and, in the second example, a team of robots traveling information. Finally, some preliminary work on extending the approach to nonholonomic systems is presented.

Addresses the problem of human robot interaction with application to the design of assistive devices. We describe the design and development of a prototype of a smart wheelchair that can be commanded by a rider. Specifically, we focus on (a) the vision-based human interaction interface; (b) the suite of sensors on the chair, and (c) the software architecture and the control algorithms used to control the chair.

We describe a framework for controlling and coordinating a group of robots for cooperative manipulation tasks. The framework enables a decentralized approach to planning and control. It allows the robots to approach the object, organize themselves into a formation that will trap the object, and then transport the object to the desired destination. Our controllers and planners are derived from simple potential fields and the hierarchical composition of potential fields. We show how these potential field based controllers and planners benefit complex group interactions, specifically for manipulating and transporting objects in the plane. Theoretically, we show how we can derive results on formation stability with potential field based controllers in many cases. Simulation results demonstrate successful application to a wide range of examples without showing sensitivity to parameters. Because the framework is decentralized at both trajectory generation level and the estimation and control agent level, our framework can potentially scale to groups of tens and hundreds of robots.

A new type of stability of leader follower formations is defined, based on input-to-state stability (ISS) properties of cascade interconnections. Formation ISS links leader input to internal state of the formation and characterizes the way this input affects performance. The effect of feedforward and feedback inter-agent communication is then investigated in this framework and it is indicated how the structure of interconnections and the amount of available information can affect stability performance.

We describe a framework for cooperative control of a group of nonholonomic mobile robots that allows us to build complex systems from simple controllers and estimators. The resultant modular approach is attractive because of the potential for reusability. Our approach to composition also guarantees stability and convergence in a wide range of tasks. There are two key features in our approach: 1) a paradigm for switching between simple decentralized controllers that allows for changes in formation; 2) the use of information from a single type of sensor, an omnidirectional camera, for all our controllers. We describe estimators that abstract the sensory information at different levels, enabling both decentralized and centralized cooperative control. Our results include numerical simulations and experiments using a testbed consisting of three nonholonomic robots.

We describe a framework and control algorithms for coordinating multiple mobile robots with manipulators focusing on tasks that require grasping, manipulation and transporting large and possibly flexible objects without special purpose fixtures. Because each robot has an independent controller and is autonomous, the coordination and synergy are realized through sensing and communication. The robots can cooperatively transport objects and march in a tightly controlled formation, while also having the capability to navigate autonomously. We describe the key aspects of the overall hierarchy and the basic algorithms, with specific applications to our experimental testbed consisting of three robots. We describe results from many experiments that demonstrate the ability of the system to carry flexible boards and large boxes as well as the system's robustness to alignment and odometry errors.

Develops a method for generating smooth trajectories for mobile robots in formation. The problem of trajectory generation is cast in terms of designing optimal curves on the Euclidean group, SE(3). Specifically, the method generates the trajectory that minimizes the total energy associated with the translations and rotations of the robots, while maintaining a rigid formation. When the mobile robots are nonholonomic, trajectories that allow rigid formations to be maintained must satisfy appropriate constraints. An efficient non-iterative algorithm to obtain near-optimal trajectories is described. Finally, the approach is illustrated with examples involving formations of aircrafts.

Proposes an architecture for tightly coupled multi-robot coordination that is well suited to cooperative manipulation tasks. At all times, a robot is identified as a leader, while the others are designated as followers. The assignment of roles and the coordination between the robots is guaranteed by communication protocols and control algorithms. The key feature is the flexibility that allows changes in leadership and assignment of roles during the execution of a task. We describe the experimental implementation and demonstration in a cooperative transportation task, in which two and three heterogeneous robots cooperate to carry a large object in an environment containing obstacles.

This paper considers the problem of vision-based control of a nonholonomic mobile robot. We describe the design and implementation of real-time estimation and control algorithms on a car-like robot platform using a single omni-directional camera as a sensor without explicit use of odometry. We provide experimental results for each of these vision-based control objects. The algorithms are packaged as control modes and can be combined hierarchically to perform higher level tasks involving multiple robots.

In this paper multirate numerical integration techniques are introduced as a tool for simulating robotic systems. In contrast with traditional simulation techniques where a single global time step is used, multirate methods seek a gain in efficiency by using larger step sizes for the slow varying components and smaller step sizes for components with rapidly changing solutions. We argue that many robotic systems inherently possess different time scales, and therefore can benefit from multirate techniques. We have developed a multirate version of the popular Adams predictor-corrector methods, which has a variety of modern features. We present results on the accuracy, stability and efficiency of the algorithm along with simulation results.

We describe a framework for controlling a group of nonholonomic mobile robots equipped with range sensors. The vehicles are required to follow a prescribed trajectory while maintaining a desired formation. By using the leader-following approach, we formulate the formation control problem as a hybrid (mode switching) control system. We then develop a decision module that allows the robots to automatically switch between continuous-state control laws to achieve a desired formation shape. The stability properties of the closed-loop hybrid system are studied using the Lyapunov theory. We do not use explicit communication between robots; instead we integrate optimal estimation techniques with nonlinear controllers. Simulation and experimental results verify the validity of our approach.

In this paper we consider the dynamics and control of whole arm grasping systems. We develop a control scheme that employs a minimal set of inputs to control the trajectory of the system while using the surplus inputs to control the interaction forces in order to maintain the unilateral constraints at both rolling and sliding contacts. Since the number of surplus inputs is less than the number of output force variables, we propose a controller that controls the critical contact force components. We emphasize the dynamic models and algorithms for computing contact forces, which are crucial to the development of the control algorithms. Finally, we show how compliant contact models and an integrated simulation approach are used to overcome the difficulties with uniqueness and existence of solutions. A planar whole arm manipulation system is used as an example to illustrate the basic ideas.

We present a novel modeling framework and control algorithms for multiple mobile manipulators cooperatively grasping and transporting an object. The planning and control tasks are decentralized, and the framework explicitly incorporates the protocols used to coordinate the robots in the team. The framework is flexible in the sense that it scales with the number of robots and controllers. Preliminary experimental results for teams of two-three robots are shown, while simulation results are used to illustrate the extensions of our approach to larger teams.

We describe a framework for coordinating multiple robots in cooperative manipulation tasks in which vision is used for establishing relative position and orientation and maintaining formation. The two key contributions are a cooperative scheme for localizing the robots based on visual imagery that is more robust than decentralized localization, and a set of control algorithms that allow the robots to maintain a prescribed formation (shape and size). The ability to maintain a prescribed formation allows the robots to "trap" objects in their midst, and to "flow" the formation to a desired position. We derive the cooperative localization and control algorithms and present experimental results that illustrate the implementation and the performance of these algorithms.

This paper addresses the control of a team of nonholonomic mobile robots navigating in a terrain with obstacles while maintaining a desired formation and changing formations when required, using graph theory. We model the team as a triple, (g, r, H), consisting of a group element g that describes the gross position of the lead robot, a set of shape variables r that describe the relative positions of robots, and a control graph H that describes the behaviors of the robots in the formation. Our framework enables the representation and enumeration of possible control graphs and the coordination of transitions between any two formations.

In this paper, we survey the field of robotic grasping and the work that has been done in this area over the last two decades, with a slight bias toward the development of the theoretical framework and analytical results in this area.

We discuss a game theoretic approach to the design of closed loop feedback laws to solve sensor based motion planning problems for mobile robots. Our approach provides a framework for dealing with environmental uncertainty, and, by explicitly accounting for the sensor dynamics, a formal way of combining exploratory and goal directed motions. We focus on methods of devising optimal feedback laws, using finite dimensional parametrizations, under the worst case uncertainty for several illustrative examples.

We propose a novel approach to dynamic simulation of grasps that overcomes difficulties due to inconsistencies in the forward dynamics problem. The key idea in the paper is a minimalist model of the contact compliance and the use of this model in situations when the performance of the rigid body model is not satisfactory. Our general framework allows for online diagnostics that enable the automatic switching between models to maximize efficiency while avoiding ambiguous situations. We illustrate the basic ideas by simulating the dynamics of several whole arm grasps.

We develop two quality measures for the analysis and optimization of grasps and fixtures. The first metric is a measure of stability and is obtained by analyzing the effect of small perturbations in the position of the grasped object or small disturbance forces on the grasped object The second metric is a measure of robustness to errors in the positions of the fixture elements. In addition, we explore the effect that the stiffness of the fixture elements has on these two quality measures and propose a design optimization approach to maximize stability and robustness to errors.

Addresses the control of a team of robots navigating in a terrain with obstacles while maintaining a desired formation and changing formations when required. We model the team as a triple consisting of a group element that describes the gross motion of the team, a set of shape variables that describe the relative positions of robots, and a control graph that describes the behaviors of the robots in the formation. We assume that a lead robot is equipped with the appropriate sensors and plans the gross motion (path) for the team. This path is derived from optimal control theory. All other robots are coordinated by continuous controllers that are prescribed by the control graph. Our framework allows us to enumerate the number of control graphs and the possible transitions between them. Further, we describe an algorithm that allows the team of robots to move between any two formations, while avoiding obstacles. We illustrate the methodology with examples involving teams of 5 and 6 robots in the presence of obstacles.

The main goal of the paper is to present a framework and basic control algorithms for coordinating a small team of robots in tasks that involve grasping large objects and transporting them in a two-dimensional environment with obstacles. Each robot consists of a mobile platform and an arm. Some arms are passive while others are active. Our previous work (1998) addressed the manipulator design, the architecture, and the communication between robots. In this paper we address the control of the manipulators, the coordination between the platforms for a team of two or three cooperating robots, and the handling of flexible large objects.

The main objective is to generate kinematic models for the head and neck movements. The motivation comes from our study of individuals with quadriplegia and the need to design rehabilitation aiding devices such as robots and teletheses that can be controlled by head-neck movements. It is then necessary to develop mathematical models for the head and neck movements. Two identification methods have been applied to study the kinematics of head-neck movements of able-body as well as neck-injured subjects. In particular, sagittal plane movements are well modeled by a planar two-revolute-joint linkage. In fact, the motion in joint space seems to indicate that sagittal plane movements may be classified as a single DOF motion. Finally, a spatial three-revolute-joint system has been employed to model 3D head-neck movements.

We develop a geometric framework for the stability analysis of multifingered grasps and propose a measure of grasp stability for arbitrary perturbations and loading conditions. The measure requires a choice of metric on the group of rigid body displacements. We show that although the stability of a grasp itself does not depend on the choice of metric, comparison of the stability of different grasps depends on the metric. Finally, we provide some insight into the choice of metrics for stability analysis.

We investigate feedback laws used to control multiple robots moving together in a formation. We propose a method for controlling formations that uses only local sensor-based information, in a leader-follower motion. We use methods of feedback linearization to exponentially stabilize the relative distance and orientation of the follower, and show that the zero dynamics of the system are also (asymptotically) stable. We demonstrate in simulation the use of these algorithms to control six robots moving around an obstacle. These types of control laws can be used to control arbitrarily large numbers of robots moving in very general types of formations.

This paper addresses the design and control of autonomous mobile platforms manipulating and transporting a grasped object. Each platform is equipped with a manipulator arm. In order to hold an object in a stable grasp, the arms have to apply and maintain appropriate contact forces to ensure force closure. We describe an architecture and a real time control system that allow the coordinated control of multiple, heterogeneous, mobile manipulators. In this architecture, a lead robot plans, based on available sensory information, and follows a suitable trajectory. The other robots follow a desired formation with respect to the leader while maintaining a stable grasp. There are three main contributions in this paper. We present the real time control system and our approach to rapidly prototype the control code. We describe a novel, forklift-like arm and a control scheme that allows us to easily control the Cartesian stiffness or impedance. Finally, we describe the details of an experimental testbed and results that demonstrate the robustness of the control scheme.

This paper investigates methods for computing a smooth motion that interpolates a given set of positions and orientations of a rigid body. To make the interpolation independent of the representation of the motion, we use the coordinate-free framework of differential geometry. Inertial and body-fixed reference frames must be chosen to describe the position and orientation of the rigid body. We show that trajectories that are independent of the choice of these frames can be obtained by using the exponential map. Since these trajectories may exhibit rapid changes in velocity or its higher derivatives, a method for finding the maximally smooth interpolating curve is developed. Trajectories computed by both methods are compared on an example.

We present a general framework for motion planning of robots in the presence of obstacles and other robots. We use variational calculus and optimization to find optimal open loop and closed loop plans in the presence of uncertainty. The plans are based on world models with set-valued uncertainty associated with the positions and shape of the obstacles. The open loop plans are generated by an efficient method that allows successive refinements of a nominal motion plan and accommodates finer levels of granularity as additional information becomes available. The closed loop plans are control policies that are based on given sensor models. The optimal plan is a control policy that performs the best in the worst-case situation. We discuss how the open loop and closed loop plans can be viewed as optimal strategies in the framework of two-person, zero-sum, non-cooperative games.

This paper investigates methods for computing a smooth motion that interpolates a given set of positions and orientations. The position and orientation of a rigid body can be described with an element of the group of spatial rigid body displacements, SE(3). To find a smooth motion that interpolates a given set of positions and orientations is therefore the same as finding an interpolating curve between the corresponding elements of SE(3). To make the interpolation on SE(3) independent of the representation of the group, we use the coordinate-free framework of differential geometry. It is necessary to choose inertial and body-fixed reference frames to describe the position and orientation of the rigid body. We first show that trajectories that are independent of the choice of these frames can be obtained by using the exponential map on SE(3). However, these trajectories may exhibit rapid changes in the velocity or higher derivatives. The second contribution of the paper is a method for finding the maximally smooth interpolating curve. By adapting the techniques of the calculus of variations to SE(3), necessary conditions are derived for motions that are equivalent to cubic splines in the Euclidean space. These necessary conditions result in a boundary value problem with interior-point constraints. A simple and efficient numerical method for finding a solution is then described. Finally, we discuss the dependence of the computed trajectories on the metric on SE(3) and show that independence of the trajectories from the choice of the reference frames can be achieved by using a suitable metric.

This paper addresses the problem of generating smooth trajectories between an initial and a final position and orientation in space. The main idea is to define a functional depending on velocity or its derivatives that measures smoothness of trajectories and find a trajectory that minimizes this functional. In order to ensure that the computed trajectories are independent of the parametrization of positions and orientations, we use the notions of Riemannian metric and covariant derivative from differential geometry and formulate the problem as a variational problem on the Lie group of spatial rigid body displacements. We show that by choosing an appropriate measure of smoothness, the trajectories can be made to satisfy boundary conditions on the velocities or higher order derivatives. Dynamically smooth trajectories can be obtained by incorporating the inertia of the system into the definition of the Riemannian metric. We state the necessary conditions for the shortest distance, minimum acceleration and minimum jerk trajectories.

We address the problem of motion planning for nonholonomic cooperating mobile robots manipulating and transporting objects while holding them in a stable grasp. We present a general approach based on the calculus of variations that allows us to obtain optimal trajectories and actuator forces/torques for any manoeuvre in the presence of obstacles. In addition, geometric constraints such as joint limits, kinematic constraints such as nonholonomic velocity constraints and dynamic constraints can be easily incorporated into the planning scheme. The application of the method is illustrated by computing motion plans for several examples.

This paper addresses the optimal control and selection of gaits in a class of nonholonomic locomotion systems that exhibit group symmetries. We study optimal gaits for the snakeboard, a representative example of this class of systems. We employ Lagrangian reduction techniques to simplify the optimal control problem, and describe a general framework and an algorithm to obtain numerical solutions to this problem. This paper represents an initial study in using optimal control techniques to study optimality of gaits and issues involving gait transitions. The general framework provided in this paper can easily be applied to other examples of biological and robotic locomotion.

We study the 6/spl times/6 Cartesian stiffness matrix. We show that the stiffness of a rigid body subjected to conservative forces and moments is described by a (0,2) tensor which is the Hessian of the potential function. The key observation of the paper is that since the Hessian depends on the choice of an affine connection in the task space, so will the Cartesian stiffness matrix. Further, the symmetry of the Hessian and thus of the stiffness matrix depends on the symmetry of the connection. The connection that is implicit in the definition of the Cartesian stiffness matrix through the joint stiffness matrix (Salisbury, 1980) is made explicit and shown to be symmetric. In contrast, the direct definition of the Cartesian stiffness matrix in Griffis (1993), Ciblak and Lipkin (1994) and Howard et al. (1996) is shown to be derived from an asymmetric connection. A numerical example is provided to illustrate the main ideas of the paper.

This paper studies human dual arm manipulation tasks and develops a computational model that predicts the trajectories and the force distribution for the coordination of two arms moving an object between two given positions and orientations in a horizontal plane. Our ultimate goal is to understand the dynamics of dual arm coordination in order to develop better robot control algorithms. Our computational model is based on the hypothesis proposed by Uno et al. (1989) who suggest that human movements minimize the integral of the norm of the rate of change of actuator torques. We compare the experimental trajectories and force distributions with this computational model. The first important observation is that the trajectories show a significant degree of repeatability across trials and across subjects. Next, we observe that the trajectories in the sagittal and frontal plane are characterized by asymmetric features that are hard to model using such integral cost functions. Finally, we show that the internal forces play an important role in trajectory generation. While these are repeatable across trials, they vary significantly from subject to subject.

When two or more arms are used to manipulate a large object, it is preferable not to have a rigid grasp in order to gain more dexterity in manipulation. It may therefore be necessary to control contact motion between the object and the effector(s) on one or more arms. This paper addresses the dynamic control of two arms cooperatively manipulating a large object with rolling contacts. In the framework presented here, the motion of the object as well as the loci of the contact point either on the surface of each effector or on the object can be directly controlled. The velocity and acceleration equations for three-dimensional rolling contacts are derived in order to obtain a dynamic model of the system. A nonlinear feedback control algorithm that decouples and linearizes the system is developed. This is used to demonstrate the control of rolling motion along each arm and the adaptation of grasps to varying loads.

Presents a general approach for coordinating the legs of a multi-legged statically stable walking machine on an uneven terrain. The approach yields an optimized motion plan for several "body-lengths" that allows us to select footholds and sequence the legs of the walking machine. We assume that a terrain map is available but that this map may be characterized by uncertainty. We also assume the optimality of the motion plan can be measured by a suitable metric. The method of ordinal optimization is used to find a motion plan that is guaranteed to be in a desired percentile with a given confidence level. Depending on the available computational resources we can improve our confidence level and/or get closer to the optimal plan. Finally, our approach allows us to trade off speed with safety, and speed with optimality.

We address the motion planning for "fixtureless" material-handling with multiple manipulators on nonholonomic carts. The mobile manipulators possess the ability to manipulate and transport objects while holding them in a stable grasp. We present a general approach that allows generation of optimal trajectories and actuator inputs for any given maneuver. Constraints such as limitations on the turning radii of the mobile manipulators or bounds on their separation can be easily incorporated into the planning scheme. Numerical solutions for several maneuvers including abrupt turns, parallel parking in cluttered environments and changes in formation are computed. Finally, we present experimental results with two mobile manipulators.

This paper addresses the general problem of generating smooth trajectories between an initial and a final position and orientation. A functional depending an velocity and its higher derivatives involving a left invariant Riemannian metric on SE(3) is used to measure the smoothness of a trajectory. The problem of determining a smooth trajectory between two points is formulated as a variational problem on SE(3). The authors derive necessary conditions for the shortest distance and minimum jerk trajectories and solve the resulting two-point boundary value problem.

A grasped object is defined to be in equilibrium if the sum of all forces and moments acting on a body equals zero. An equilibrium grasp may be stable or unstable. Force closed grasps are a well-known subset of equilibrium grasps, and they are known to be stable. However, not all stable grasps are force closed, including many common and easily obtainable grasps. In this paper, we classify the categories of equilibrium grasps and establish a general framework for the determination of the stability of a grasp. In order to analyze the stability of grasps with multiple contacts, we first model the compliance at each contact. We develop expressions for the changes in contact forces as a function of the rigid body relative motion between the fingers and the grasped object. The stability of a grasp is shown to depend on the local curvature of the contacting bodies, as well as the magnitude and arrangement of the contact forces. We then derive results providing simple criteria to determine the stability of a grasped object, including the special but important limiting case of rigid bodies where the contact compliance is zero.

We address the stability and performance of enveloping grasps. An enveloping grasp is defined in which robot fingers or arms grasp an object by making contact at points other than just the fingertips or end-effectors. Previous work has demonstrated that enveloping grasps are more robust. However their analysis is difficult because they are usually statically indeterminate and overconstrained. In this paper we develop a method for modeling the stiffness of a grasp. It is based on a model of compliance at each contact and a model of the joint compliance at each joint. It incorporates the kinematics of three-dimensional contact in addition to the kinematics of the robot arm/finger. In particular the methodology is used to show that force closure does not necessarily imply stability.

This paper addresses the determination of joint motions for a given end effector displacement in kinematically redundant manipulators. It is well known that velocity control schemes, such as pseudoinverse control, do not guarantee repeatability. But it is possible to obtain repeatable solutions by controlling the manipulator joints so that they behave like elastic couplings. The joint configuration adopted by the manipulator at a given point in task space is one which minimizes the potential energy of the system and is unique. In this paper, we investigate the existence and uniqueness of solutions (joint displacements) for a given end effector displacement. We identify singularities in the algorithm and their physical significance. Finally, we show that it is important to consider the stability of the manipulator at a given configuration and discuss practical considerations in implementing such algorithms.

Problems in robotics and biomechanics such as trajectory planning or resolution of redundancy can be effectively solved using optimal control. Such systems are often subject to unilateral constraints. Examples include tasks involving contacts (e.g., walking, running, multifingered or multiarm manipulation), and other tasks that may not involve contacts but in which the system state or the inputs must satisfy inequality conditions (e.g., limits on actuator forces). This paper shows how problems of optimal control in robotics that involve unilateral constraints can be efficiently solved by first formulating the constrained optimal control problem as an unconstrained problem of the calculus of variations and then solving it using an integral formulation. This method has several advantages over the Pontryagin minimum principle which is traditionally employed to solve such problems. An example of two-arm manipulation with inequality constraints due to Coulomb friction is used to demonstrate the formulation of the problem and the algorithms.

Presents a collaborative effort to design and implement a cooperative material handling system by a small team of human and robotic agents in an unstructured indoor environment. The authors' approach makes fundamental use of the human agents' expertise for aspects of task planning, task monitoring, and error recovery. The authors' system is neither fully autonomous nor fully teleoperated. It is designed to make effective use of the human's abilities within the present state of the art of autonomous systems. The authors' robotic agents refer to systems which are each equipped with at least one sensing modality and which possess some capability for self-orientation and/or mobility. The authors' robotic agents are not required to be homogeneous with respect to either capabilities or function. The authors' research stresses both paradigms and testbed experimentation. Theory issues include the requisite coordination principles and techniques which are fundamental to a cooperative multiagent system's basic functioning. The authors have constructed an experimental distributed multiagent-architecture testbed facility. The required modular components of this testbed are currently operational and have been tested individually. The authors' current research focuses on the agents' integration in a scenario for cooperative material handling.

This paper addresses determination of trajectories and force distribution for cooperative manipulation with two arms through optimizing an integral cost function that depends an the actuator forces. We compare the calculated trajectories with the measurements on human subjects performing planar manipulation tasks. Our findings suggest that the trajectories and forces used by humans can be predicted by minimizing the integral of the rate of change of actuator torques over the trajectory. Good match is shown for a class of manipulation tasks in which the person-to-person variability is small. The theoretical foundation for computing the optimal solutions is briefly presented and the advantages of using such schemes for robotic systems are discussed.

It is well established that all form closed grasps are stable. However, not all stable grasps are form closed, including many common and easily obtainable grasps. This is especially true in whole arm manipulators and multi-fingered hands where, in many cases, it is difficult or impossible to effect a form closed grasp. Thus it is necessary to establish a general framework for the determination of stability for any equilibrium grasp. In order to analyze the stability of grasps with multiple contacts, we first model the compliance at each contact. We develop expressions for the changes in contact forces as a function of the rigid body relative motion between the fingers and a grasped planar object. The stability of a grasp is shown to depend on the local curvature of the contacting bodies as well as the magnitude and arrangement of the contact forces. We then derive results providing simple criteria to determine the stability of almost any grasped planar object, including the special but important limiting case of rigid bodies where the contact compliance is zero.>

In a decentralized multi-robot system, each robot is controlled by an independent controller and the information obtained by each robot through its proprioceptive sensing devices is not shared with other robots. If the coordination of multiple robots in a cooperative manipulation task is to be accomplished, each robot must be able to exhibit controlled interaction with other robots and objects with minimal information. In this paper, the authors address the problem of maintaining rolling contact between a robot arm and an external, moving object. The authors assume that a nominal model of the motion of the moving object is available. The dynamic system that characterizes the motion of the robot arm and the moving object is acatastatic and nonholonomic. The authors design a nonlinear feedback for such a system that successfully maintains rolling contact. Simulation results are presented to demonstrate the effectiveness of the system.>

A proof-of-concept prototype walking chair for the disabled is proposed with the objective of demonstrating the feasibility of a completely new approach to mobility. Our prototype system consists of a chair equipped with wheels and legs and is capable of walking on uneven terrain and circumventing obstacles. The important design considerations, the system design and an experimental prototype of a chair, are discussed. Redundancy in actuation enables the online optimization of tractive forces which enhances the adaptability of the system.>

The optimization of trajectories and actuator torques for a dual arm manipulation system is considered. Given the starting and final configurations, we find the trajectories that minimize: (a) the integral of the norm of the vector of derivatives of the actuator forces; and (b) the integral of the norm of the actuator forces. In this way both kinematic and actuator redundancy are resolved. The optimization problem reduces to solving a two-point boundary valve problem for coupled, nonlinear differential equations. The effect of different parameters such as preload and inertia are investigated and the results are compared with those obtained using other well-known cost functions.>

When multiple arms are used to manipulate a large object, it is beneficial and sometimes necessary to maintain and control contacts between the object and the effector (the contacting surface of an arm) through force closure. Rolling and/or sliding can occur at these contacts, and the system is, in general, characterized by holonomic as well as nonholonomic (including unilateral) constraints. In this paper, the control of planar rolling contacts is investigated. Multi-arm manipulation systems are typically redundant. In the authors' approach, a minimal set of inputs is employed to control the trajectory of the system while the surplus inputs control the contact conditions. The trajectory includes the gross motion of the object as well as the rolling motion at each contact. A nonlinear feedback scheme for simultaneous control of motion as well as contact conditions is presented. A new algorithm which adapts a two-effector grasp with rolling contacts to external loads and the trajectory is developed. Simulations and experimental results are used to illustrate the salient features in control and planning.>

In a versatile, intelligent, multirobot system, each robot is controlled by independent controllers and information obtained by a robot through proprioceptive sensing (joint position, velocity, torques etc.) is not shared with other robots. If the coordination of robots in a cooperative manipulation task is to be accomplished, each robot must be able to exhibit controlled interaction with other robots and objects with minimal information. The authors address the problem of maintaining rolling contact between am actively controlled mechanical linkage and an external moving surface, assuming that a nominal model of the gross motion of the moving surface is available. Thus the control problem involves a dynamic system that is acatastic and nonholonomic. Nonlinear feedback is used to decouple and linearize the time-varying system. Examples and results from computer simulations are used to show that the normal and tangential forces as well as the relative rolling motion can be controlled satisfactorily.

A compact inverse rate kinematics solution for serial-chain redundant n-jointed robots is presented. The serial-chain inverse rate kinematics problem is dual to the inverse statics problem in in-parallel systems such as cooperating robots, in which the task of distributing the load wrench between multiple subsystems is typically under-specified. Therefore, the method proposed earlier in J.H. Kim and V. Kumar (1991) and V. Kumar (1990) for obtaining the wrench distribution in multirobot systems can be applied to redundant serial-chain manipulators. The solution thus obtained is one in which a weighted-norm of the joint rates vector is minimized. The advantages of this method are the ability to accommodate any type of joint and the availability of analytical, closed-form solutions for the joint rates. A geometric interpretation of the solution is provided using screw system theory.>

The authors address force control in overconstrained dynamic systems with special emphasis on robot control. Previous approaches to force control are studied, and many of these are shown to be unsuitable for dynamic force control. Practical and theoretical considerations for designing force control algorithms are discussed. Experimental and simulation results that validate the theoretical findings are presented for a single-degree-of-freedom pneumatic force controller.>

The authors present a unified formulation for the control problem of multiple arm systems which accommodates both holonomic and nonholonomic constraints. Several unique control properties of nonholonomic systems are discussed. Several useful results regarding input-output linearization and the zero dynamics in such systems are proved. The analysis and controller design are discussed for multi-arm systems. Results from computer simulations are presented to demonstrate the control algorithms. Simulation results illustrate that rolling and sliding can be effectively controlled.>

An object much larger than a robot end-effector can not be rigidly grasped. Instead, it is necessary to support the object with multiple effectors. The effectors need not be grippers-they may be surfaces on the arm(s). Multiple arm manipulation without rigid grasps is investigated. The interaction between a robot and the object is characterized by unilateral constraints. The closed chain topology, redundancy in actuation, and the nonlinear coupled equations of motion make the problem formidable. The approach used is to utilize a minimal set of inputs to control the trajectory. The surplus inputs are used to control the contact conditions (rolling, sliding, etc.).>

The mechanics, design, control, and experimental results for a 3-DOF (degree-of-freedom) in-parallel, pneumatically actuated manipulator are presented. The manipulator consists of two platforms connected by three serial chains. It is naturally compliant in translation along the approach direction as well as in rotations about axes perpendicular to the approach direction, while it is stiff in other directions. The kinematic design allows the three relative DOF between the two plates of the manipulator to accommodate uncertainties and sustain impacts while contacting and interacting with unknown environments. Experimental results show that, with a sampling rate of 100 Hz, the position control bandwidth of a single actuator servo system is 11 Hz, while the force control bandwidth is 10 Hz. These figures also reflect the bandwidth and the performance of the complete system.>

The instantaneous kinematics are discussed of a hybrid manipulation system that combines the traditional serial chain geometry with parallelism in actuation. Such a system is characterized by closed chains in the structure and redundancy in actuation. This redundancy is shown to be dual to the kinematic redundancy in serial chain robot manipulators. The presence of redundancy in the system allows the specification of force set-points that will ensure an optimal load distribution for force control or hybrid control schemes. In addition, the singularities in the inverse kinematics and statics equations, which are typical of robotic systems with closed chains, are analyzed. Coordination algorithms for the computation of optimal force distribution that minimize joint torques while avoiding singularities with reasonable computation efficiency are the authors' main focus. In particular, a planar dual-arm manipulation system is used as an example and is analyzed in some detail.>