We present a pipeline for geomorphological analysis that uses structure from motion (SfM) and deep learning on close-range aerial imagery to estimate spatial distributions of rock traits (size, roundness, and orientation) along a tectonic fault scarp. The properties of the rocks on the fault scarp derive from the combination of initial volcanic fracturing and subsequent tectonic and geomorphic fracturing, and our pipeline allows scientists to leverage UAS-based imagery to gain a better understanding of such surface processes. We start by using SfM on aerial imagery to produce georeferenced orthomosaics and digital elevation models (DEM). A human expert then annotates rocks on a set of image tiles sampled from the orthomosaics, and these annotations are used to train a deep neural network to detect and segment individual rocks in the entire site. The extracted semantic information (rock masks) on large volumes of unlabeled, high-resolution SfM products allows subsequent structural analysis and shape descriptors to estimate rock size, roundness, and orientation. We present results of two experiments conducted along a fault scarp in the Volcanic Tablelands near Bishop, California. We conducted the first, proof-of-concept experiment with a DJI Phantom 4 Pro equipped with an RGB camera and inspected if elevation information assisted instance segmentation from RGB channels. Rock-trait histograms along and across the fault scarp were obtained with the neural network inference. In the second experiment, we deployed a hexrotor and a multispectral camera to produce a DEM and five spectral orthomosaics in red, green, blue, red edge, and near infrared. We focused on examining the effectiveness of different combinations of input channels in instance segmentation.

Flying modular robots have the potential to rapidly form temporary structures. In the literature, docking actions rely on external systems and indoor infrastructures for relative pose estimation. In contrast to related work, we provide local estimation during the self-assembly process to avoid dependency on external systems. In this paper, we introduce ModQuad-Vi, a flying modular robot that is aimed to operate in outdoor environments. We propose a new robot design and vision-based docking method. Our design is based on a quadrotor platform with onboard computation and visual perception. Our control method is able to accurately align modules for docking actions. Additionally, we present the dynamics and a geometric controller for the aerial modular system. Experiments validate the vision-based docking method with successful results.

We present a novel fruit counting pipeline that combines deep segmentation, frame to frame tracking, and 3D localization to accurately count visible fruits across a sequence of images. Our pipeline works on image streams from a monocular camera, both in natural light, as well as with controlled illumination at night. We first train a Fully Convolutional Network (FCN) and segment video frame images into fruit and non-fruit pixels. We then track fruits across frames using the Hungarian Algorithm where the objective cost is determined from a Kalman Filter corrected Kanade-Lucas-Tomasi (KLT) Tracker. In order to correct the estimated count from tracking process, we combine tracking results with a Structure from Motion (SfM) algorithm to calculate relative 3D locations and size estimates to reject outliers and double counted fruit tracks. We evaluate our algorithm by comparing with ground-truth human-annotated visual counts. Our results demonstrate that our pipeline is able to accurately and reliably count fruits across image sequences, and the correction step can significantly improve the counting accuracy and robustness. Although discussed in the context of fruit counting, our work can extend to detection, tracking, and counting of a variety of other stationary features of interest such as leaf-spots, wilt, and blossom.

Unmanned aerial vehicles (UAVs) have the potential to significantly impact early detection and monitoring of plant diseases. In this paper, we present preliminary work in developing a UAV-mounted sensor suite for detection of citrus greening disease, a major threat to Florida citrus production. We propose a depth-invariant sensing methodology for measuring reflectance of polarized amber light, a metric which has been found to measure starch accumulation in greening-infected leaves. We describe the implications of adding depth information to this method, including the use of machine learning models to discriminate between healthy and infected leaves with validation accuracies up to 93%. Additionally, we discuss stipulations and challenges of use of the system with UAV platforms. This sensing system has the potential to allow for rapid scanning of groves to determine the spread of the disease, especially in areas where infection is still in early stages, including citrus farms in California. Although presented in the context of citrus greening disease, the methods can be applied to a variety of plant pathology studies, enabling timely monitoring of plant health-impacting scientists, growers, and policymakers.

A key component of robotic path planning for monitoring dynamic events is reliable navigation to the right place at the right time. For persistent monitoring applications (e.g., over months), marine robots are beginning to make use of the environment for propulsion, instead of depending on traditional motors and propellers. These vehicles are able to realize dramatically higher endurance by exploiting wave and wind energy, however the path planning problem becomes difficult as the vehicle speed is no longer directly controllable. In this paper, we examine Gaussian process models to predict the speed of the Wave Glider autonomous surface vehicle from observable environmental parameters. Using training data from an on-board sensor, and wave parameter forecasts from the WAVEWATCH III model, our probabilistic regression models create an effective method for predicting Wave Glider speed for use in a variety of path planning applications.

Marine phenomena such as algal blooms can be detected using in situ measurements onboard autonomous underwater vehicles (AUVs), but understanding plankton ecology and community structure requires retrieval and analysis of water specimens. This process requires shipboard or manual sample collection, followed by onshore lab analysis which is time-consuming. Better understanding of the relationship between the observable environmental features and organism abundance would allow more precisely targeted sampling and thereby save time. In this work, we present an approach to learn and improve models that predict this relationship. Coupled with recent advances in AUV technology allowing selective retrieval of water samples, this constitutes a new paradigm in biological sampling. We use organism abundance models along with spatial models of environmental features learned immediately after AUV deployments to compute spatial distributions of organisms in the coastal ocean purely from in situ AUV data. We use Gaussian process regression along with the unscented transform to fuse the two models, obtaining both the mean and variance of the organism abundance estimates. The uncertainty in organism abundance predictions is used in a sampling strategy to selectively acquire new water specimens that improves the organism abundance models. Simulation results are presented demonstrating the advantage of performing hierarchical probabilistic regression. After the validation through simulation, we show predictions of organism abundance from models learned on lab-analyzed water sample data, and AUV survey data.

With the advent of Autonomous Underwater Vehicles (AUVs) and other mobile platforms, marine robotics have had substantial impact on the oceanographic sciences. These systems have allowed scientists to collect data over temporal and spatial scales that would be logistically impossible or prohibitively expensive using traditional ship-based measurement techniques. Increased dependence of scientists on such robots has permeated scientific data gathering with future field campaigns involving these platforms as well as on entire infrastructure of people, processes and software, on shore and at sea. Recent field experiments carried out with a number of surface and underwater platforms give clues to how these technologies are coalescing and need to work together. We highlight one such confluence and describe a future trajectory of needs and desires for field experiments with autonomous marine robotic platforms. Our 2010 inter-disciplinary experiment in the Monterey Bay involved multiple platforms and collaborators with diverse science goals. One important goal was to enable situational awareness, planning and collaboration before, during and after this large-scale collaborative exercise. We present the overall view of the experiment and describe an important shore-side component, the Oceanographic Decision Support System (ODSS), its impact and future directions leveraging such technologies for field experiments.

This paper presents an oceanographic toolchain that can be used to generate multi-vehicle robotic surveys for large-scale dynamic features in the coastal ocean. Our science application targets Harmful Algal Blooms (HABs) which have significant societal impact to coastal communities yet are poorly understood ecologically. Bloom patches can be large spatially (in kms) and unpredictable in their extent. To understand their ecology, we need to be able to bring back water samples from the `right' places and times for lab analysis. In doing so, we target hotspots representative of intense biogeochemical activity for such sampling. Our approach uses remote sensing data to detect such hotspots using ocean color as a proxy, and advectively projects these patches spatio-temporally using surface current data from HF Radar stations. Experiments with satellite and Radar data sets are promising for large, coherent blooms. We show how these predictions can be used to select an appropriate sampling trajectory for an AUV.

This paper presents the design of a novel robotic system capable of long-term benthic sampling along a transect. The robot is built to traverse back and forth along a mechanical guide-rail at the bottom of a water body. We present results from localization tests with the robot in a laboratory tank and a shallow swimming pool. A pilot deployment was made at a marina to accurately observe the rate and direction of water flow across a section of the marina inlet. Results from the experiment demonstrate the potential of this platform for monitoring various aquatic phenomena of interest.

We study the collective transport of robots (CTR) problem. A large number of commodity mobile robots are to be moved from one location to another by a single operator. Joysticking each one or carrying them physically is impractical. None of the robots are particularly sophisticated in their ability to plan or reason. Prior work on flocking and formation control has addressed the transport of a robot group that maintains its integrity by explicitly controlling coherence. We show how flocking emerges as a consequence of each robot contending for space near the human operator. A coherent flock can be made to follow a leader in this manner thereby solving the CTR problem. We also present the design of a hand-worn IMU-based gesture interface which allows the human operator to issue simple commands to the group. A preliminary experimental evaluation of the system shows robust CTR with different leader behaviors.

Dynamic positioning is an important application for marine vehicles that do not have the luxury of anchoring or mooring themselves. Such vehicles are usually large and have arrays of thrusters that allow for controllability in the sway as well as the surge and yaw axes. Most smaller boats however, are underactuated and do not possess control in the sway direction. This makes the control problem significantly more challenging. We address the station keeping problem for a small autonomous surface vehicle (ASV) with significant windage. The vehicle is required to hold station at a given position. We describe the design of a weighted controller that uses wind feed-forward to complement a line-of-sight guidance controller to achieve satisfactory performance under slow-varying moderate wind conditions. We test the control system in simulation and in field trials with a twin-propeller ASV. Experiments show that the controller works very well in moderate wind conditions allowing the ASV to keep station with a position error of approximately one vehicle length.