Unmanned aerial vehicles (UAVs) are becoming more common, presenting the need for effective human-robot communication strategies that address the unique nature of unmanned aerial flight. Visual communication via drone flight paths, also called gestures, may prove to be an ideal method. However, the effectiveness of visual communication techniques is dependent on several factors including an observer's position relative to a UAV. Previous work has studied the maximum line-of-sight at which observers can identify a small UAV [1]. However, this work did not consider how changes in distance may affect an observer's ability to perceive the shape of a UAV's motion. In this study, we conduct a series of online surveys to evaluate how changes in line-of-sight distance and gesture size affect observers' ability to identify and distinguish between UAV gestures. We first examine observers' ability to accurately identify gestures when adjusting a gesture's size relative to the size of a UAV. We then measure how observers' ability to identify gestures changes with respect to varying line-of-sight distances. Lastly, we consider how altering the size of a UAV gesture may improve an observer's ability to identify drone gestures from varying distances. Our results show that increasing the gesture size across varying UAV to gesture ratios did not have a significant effect on participant response accuracy. We found that between 17 m and 75 m from the observer, their ability to accurately identify a drone gesture was inversely proportional to the distance between the observer and the drone. Finally, we found that maintaining a gesture's apparent size improves participant response accuracy over changing line-of-sight distances.

Unmanned Aerial Vehicle (UAV) flight paths have been shown to communicate meaning to human observers, similar to human gestural communication. This paper presents the results of a UAV gesture perception study designed to assess how observer viewpoint perspective may impact how humans perceive the shape of UAV gestural motion. Robot gesture designers have demonstrated that robots can indeed communicate meaning through gesture; however, many of these results are limited to an idealized range of viewer perspectives and do not consider how the perception of a robot gesture may suffer from obfuscation or self-occlusion from some viewpoints. This paper presents the results of three online user-studies that examine participants' ability to accurately perceive the intended shape of two-dimensional UAV gestures from varying viewer perspectives. We used a logistic regression model to characterize participant gesture classification accuracy, demonstrating that viewer perspective does impact how participants perceive the shape of UAV gestures. Our results yielded a viewpoint angle threshold from beyond which participants were able to assess the intended shape of a gesture's motion with 90% accuracy. We also introduce a perceptibility score to capture user confidence, time to decision, and accuracy in labeling and to understand how differences in flight paths impact perception across viewpoints. These findings will enable UAV gesture systems that, with a high degree of confidence, ensure gesture motions can be accurately perceived by human observers.

Several independent approaches exist for state estimation and control of multirotor unmanned aerial systems (UASs) that address specific and constrained operational conditions. This work presents a complete end-to-end pipeline that enables precise, aggressive and agile maneuvers for multirotor UASs under real and challenging outdoor environments. We leverage state-of-the-art optimal methods from the literature for trajectory planning and control, such that designing and executing dynamic paths is fast, robust and easy to customize for a particular application. The complete pipeline, built entirely using commercially available components, is made open-source and fully documented to facilitate adoption. We demonstrate its performance in a variety of operational settings, such as hovering at a spot under dynamic wind speeds of up to 5– 6m/s (12–15mi/h) while staying within 12cm of 3D error. We also characterize its capabilities in flying high-speed trajectories outdoors, and enabling fast aerial docking with a moving target with planning and interception occurring in under 8s.

Power-over-tether aircraft is an effective tool for persistent spatiotemporal monitoring of environmental phenomena. This paper presents the design and evaluation of flight trajectories for the tethered aircraft unmanned system (TAUS) sensing a dynamic temperature field. TAUS is a novel power-over-tether-based unmanned aerial system (UAS) configured for long-term, high throughput atmospheric monitoring. It is unique in that it provides position control while measuring atmospheric properties on-board the aircraft and with sensors along the tether. We validated the robotic system by conducting outdoor experiments to characterize the sensor performance against a meteorological tower. We found minimal sensing error at the corresponding altitude relative to the ground truth installation. We then used the experimental data to simulate four trajectories (Lawn-mower, Spiral, Star, and Flower) on power-tethered and untethered system models to evaluate performance factors related to trajectory selection. The analysis of the simulated data indicated that the power-tethered Star trajectory performed well concerning key performance factors when measuring changing atmospheric fields.

Type annotations connect variables to domain-specific types. They enable the power of type checking and can detect faults early. In practice, type annotations have a reputation of being burdensome to developers. We lack, however, an empirical understanding of how and why they are burdensome. Hence, we seek to measure the baseline accuracy and speed for developers making type annotations to previously unseen code. We also study the impact of one or more type suggestions. We conduct an empirical study of 97 developers using 20 randomly selected code artifacts from the robotics domain containing physical unit types. We find that subjects select the correct physical type with just 51% accuracy, and a single correct annotation takes about 2 minutes on average. Showing subjects a single suggestion has a strong and significant impact on accuracy both when correct and incorrect, while showing three suggestions retains the significant benefits without the negative effects. We also find that suggestions do not come with a time penalty. We require subjects to explain their annotation choices, and we qualitatively analyze their explanations. We find that identifier names and reasoning about code operations are the primary clues for selecting a type. We also examine two state-of-the-art automated type annotation systems and find opportunities for their improvement.

This project seeks to generate small Unmanned Aerial System (sUAS) flight paths that are broadly understood by the general population and can communicate states about both the sUAS and its understanding of the world. Previous work in sUAS flight paths has sought to communicate intent, destination, or emotion of the system without focusing on concrete states (e.g., low battery, landing, etc.). This work leverages biologically-based flight paths and experimental methodologies from human-human and human-humanoid robot interactions to assess the understanding of avian flight paths to communicate sUAS states to novice users. If successful, this work should inform: the human-robot interaction community about the perception of flight paths, sUAS manufacturers on how their systems could communicate with both operators and bystanders, and end users on ways to communicate with others when flying systems in public spaces. General design implications and future directions of work are suggested to build on the results here, which suggest that novice users gravitate towards labels they understand (draw attention and landing) while avoiding more technical labels (lost sensor).

Prescribed fires can lessen wildfire severity and control invasive species, but they can also be risky and costly. Unmanned aerial systems can reduce those drawbacks by, for example, dropping ignition spheres to ignite the most hazardous areas. Existing systems, however, lack awareness of the fire vectors to operate autonomously, safely, and efficiently. In this work we address that limitation, introducing an approach that integrates a lightweight fire simulator and a planner for trajectories and ignition sphere drop waypoints. Both components are unique in that they are amenable to input from the system's sensors and the fire crew to increase fire awareness. We conducted a preliminary study that confirms that such inputs improve the accuracy of the fire simulation to counter the unpredictability of the target environment. The field study of the system showed that the fire-aware planner generated safe trajectories with effective ignitions leveraging the fire simulator predictions.

Unmanned Aerial Vehicles (UAVs) can charge Wireless Rechargeable Sensor Networks (WRSNs) in remote or hard to access locations. However, the charging efficiency is heavily affected by the distance between the wireless transmitter and receiver. This efficiency impacts the possible power level increase of each charged node. Most charging algorithms require full knowledge of sensor nodes' power levels to identify the nodes to charge. Collecting this power information adds overhead to the network and limits scalability. We propose and implement Charging with Power Transfer Efficiency Compensation (CPTEC), an algorithm that charges a WRSN without the need for a priori knowledge of the nodes' power levels. We show that CPTEC compensates for efficiency drops, due to landing alignments, making it practical for real-world power transfer scenarios. Our results show that CPTEC is able to perform with a median at ≈ 72% of the optimal performance of a full knowledge algorithm that assumes maximum power transfer efficiency, while other work drops to ≈ 22%. Under constant maximum efficiency CPTEC performs ≈ 90% of the optimal full knowledge case.

Sensor payloads suspended from parachutes are often used in atmospheric profiling applications. They drift freely and often end up landing in inaccessible regions that make their retrieval challenging or impossible. In this paper, we develop and evaluate an approach using a multirotor unmanned aerial system to autonomously retrieve the parachute while it is still in the air. The system relies only on the initial conditions of the parachute-payload system and feedback from the vehicle's onboard cameras to track and then intercept the parachute mid-air in under 40 seconds on average. We present the results from our field experiments where we demonstrate the feasibility of the system and discuss its applicability to long-term payload transportation systems.

Using an Unmanned Aerial Systems (UAS) to autonomously deploy soil sensors enables their installation in otherwise hard to access locations. In this paper, we present a system that integrates a UAS and a digging mechanism which can carry, secure, and install a small sensor into dirt effectively and efficiently. The integrated system includes 1) a low profile, light-weight, inexpensive auger mechanism, 2) a sensor carrying and deploying mechanism with low power consumption, and 3) sensors and software that control and evaluate the auger performance during digging. When tested on a suite of target soils and a target depth of 120mm, the system achieved a success rate of 100% for indoor tests and 92.5% for outdoors, verifying the potential of the approach.

Type annotations provide a link between program variables and domain-specific types. When combined with a type system, these annotations can enable early fault detection. For type annotations to be cost-effective in practice, they need to be both accurate and affordable for developers. We lack, however, an understanding of how burdensome type annotation is for developers. Hence, this work explores three fundamental questions: 1) how accurately do developers make type annotations; 2) how long does a single annotation take; and, 3) if a system could automatically suggest a type annotation, how beneficial to accuracy are correct suggestions and how detrimental are incorrect suggestions? We present results of a study of 71 programmers using 20 random code artifacts that contain variables with physical unit types that must be annotated. Subjects choose a correct type annotation only 51% of the time and take an average of 136 seconds to make a single correct annotation. Our qualitative analysis reveals that variable names and reasoning over mathematical operations are the leading clues for type selection. We find that suggesting the correct type boosts accuracy to 73%, while making a poor suggestion decreases accuracy to 28%. We also explore what state-of-the-art automated type annotation systems can and cannot do to help developers with type annotations, and identify implications for tool developers.

Changes to robotic systems as they are updated or upgraded often affect the flow of control and sensor data. Developers and users spend a significant amount of time tracing the impact of these changes that could otherwise have negative impacts on the robot's performance and behavior. Changes to the rates at which data is published from sensors, controllers, and other parts of the system are particularly subtle and difficult to detect. These rate changes, even if minor (e.g. lowering the frame rate of a camera), can propagate throughout the system and have broad impacts. In this work, we develop and implement an approach to help identify the set of components whose rate may be impacted by a system change. The approach builds on the insight that certain code patterns render component's outgoing data rate independent of the component's incoming data rate. We use that insight to reduce the number of components reported as affected by the change to minimize the number of components that must be reevaluated by the developer. A study of an implementation of the approach on three ROS systems shows that it can reduce the size of the impact set by up to 41% in cases when the changes have broad data impacts. The analysis is performed at compile time and only adds a third more to the compilation time.

Developing surface classification models manually requires significant time and detracts from the goal of automating systems. We create a system that automatically collects the data using an Unmanned Aerial Vehicle (UAV), extracts features, trains a large number of classifiers, selects the best classifier, and programs the UAV with that classifier. Motivating our work is a prior project [1] that manually developed a surface classifier using an accelerometer; to verify our system functionality, we replicate those results with our new automated system and improve on those results, providing a four-surface classifier with a 75% classification rate and a hard/soft classifier with a 100% classification rate. We further verify our system through a field experiment that collects and classifies new data, proving its end-to-end functionality. Overall, our system reduces the time and machine learning expertise needed by the user to develop new time-series classifiers usable by the UAV. The general form of our system provides a valuable tool for automation of classifier creation and is released as an open-source tool [2].

This work presents a study of robot software using the Robot Operating System (ROS), focusing on detecting inconsistencies in physical unit manipulation. We discuss how dimensional analysis, the rules governing how physical quantities are combined, can be used to detect inconsistencies in robot software that are otherwise difficult to detect. Using a corpus of ROS software with 5.9M lines of code, we measure the frequency of these dimensional inconsistencies and find them in 6% (211 / 3,484) of repositories that use ROS. We find that the inconsistency type `Assigning multiple units to a variable' accounts for 75% of inconsistencies in ROS code. We identify the ROS classes and physical units most likely to be involved with dimensional inconsistencies, and find that the ROS Message type geometry_msgs::Twist is involved in over half of all inconsistencies and is used by developers in ways contrary to Twist's intent. We further analyze the frequency of physical units used in ROS programs as a proxy for assessing how developers use ROS, and discuss the practical implications of our results including how to detect and avoid these inconsistencies.

Systems interacting with the physical world operate on quantities measured with physical units. When unit operations in a program are inconsistent with the physical units' rules, those systems may suffer. Existing approaches to support unit consistency in programs can impose an unacceptable burden on developers. In this paper, we present a lightweight static analysis approach focused on physical unit inconsistency detection that requires no end-user program annotation, modification, or migration. It does so by capitalizing on existing shared libraries that handle standardized physical units, common in the cyber-physical domain, to link class attributes of shared libraries to physical units. Then, leveraging rules from dimensional analysis, the approach propagates and infers units in programs that use these shared libraries, and detects inconsistent unit usage. We implement and evaluate the approach in a tool, analyzing 213 open-source systems containing +900,000 LOC, finding inconsistencies in 11% of them, with an 87% true positive rate for a class of inconsistencies detected with high confidence. An initial survey of robot system developers finds that the unit inconsistencies detected by our tool are 'problematic', and we investigate how and when these inconsistencies occur.

Systems that interact with the physical world use software that represents and manipulates physical quantities. To operate correctly, these systems must obey the rules of how quantities with physical units can be combined, compared, and manipulated. Incorrectly manipulating physical quantities can cause faults that go undetected by the type system, likely manifesting later as incorrect behavior. Existing approaches for inconsistency detection require code annotation, physical unit libraries, or specialized programming languages. We introduce Phriky-Units, a static analysis tool that detects physical unit inconsistencies in robotic software without developer annotations. It does so by capitalizing on existing shared libraries that handle standardized physical units, common in the cyber-physical domain, to link class attributes of shared libraries to physical units. In this work, we describe how Phriky-Units works, provide details of the implementation, and explain how Phriky-Units can be used. Finally we present a summary of an empirical evaluation showing it has an 87% true positive rate for a class of inconsistencies we detect with high-confidence.

Monitoring and predicting water quality poses significant challenges. Collecting enough information to characterize bodies of water is a critical bottleneck. Collecting data and samples from the surface all the way to the bottom over a short period of time would give water scientists the best spatio-temporal picture. In this paper, we present a small, light-weight, inexpensive water sensing and sampling robot, the “Waterbug”, capable of descending to depths up to 10m, collecting sensor information and a water sample, and returning to the surface. The water sampler also has limited capability to adjust buoyancy to hold depth for the purpose of measuring environmental conditions at specific locations in the water column. It is small enough that a single scientist could carry several in a backpack or it could be deployed by other robotic systems. The low cost of the node makes it feasible for blanket deployment. No tools are required for field servicing and the sample collection chamber is a common syringe that can be swapped quickly for redeployment. The main challenge was developing the system model and algorithm for achieving neutral buoyancy in the presence of system and initial condition variance. Over a range of conditions, we were able to achieve an 80% success rate for meeting the neutral buoyancy criteria and a 100% success rate in capturing a sample and returning to the surface.

Robotic systems often have complex configuration spaces that, when poorly set, can cause failures. In this work we take advantage of the close synergy between user and robot in co-robotic systems to better diagnose and overcome configuration failures. We leverage users' understanding of the system to mark failures they observe while the system is in operation. A marked failure indicates that the robot either “did not do something when it should have” or “did something when it should not have”. The failure marking is coupled with an automated analysis approach that identifies code predicates involving configuration parameters that may be relevant to each failure type, ranks the parameters according to their potential to be associated with the failure, and suggests adjustments based on the run-time outcome of those predicates. We present the approach, its implementation, and a preliminary study on a configurable unmanned air system. The results show how the approach can successfully help diagnose and adjust faulty configuration parameters in co-robotic systems.

Using Unmanned Aerial Vehicles (UAVs) to deploy sensor networks promises an autonomous and useful method of installation in remote or hard to access locations. Some sensors, such as soil moisture sensors, must be physically installed in soft soil, yet UAVs cannot easily determine soil softness with remote sensors. In this paper, we use data from an onboard accelerometer measured during UAV landings to determine the softness of the ground. We collect and analyze over 200 data sets gathered from 8 different materials: foam, carpet, wood, tile, grass, dirt, concrete, and woodchips. Based on this analysis, we examine a number of features from the accelerometer and four classification algorithms: LDA, QDA, SVM, and binary decision trees. The decision tree performs well and is simple to implement onboard the UAV. We implement this in our UAV control system and perform experiments to verify that the UAV can accurately classify the softness of the surface with 90% accuracy. This lays the groundwork for our future work on developing a UAV capable of installing sensors in soft soil.

Ocean monitoring using underwater sensor networks faces communication challenges in retrieving data, communicating large amounts of data between nodes, and covering increasing spatial regions while remaining connected. With underwater sensor networks that are capable of surfacing, unmanned aerial vehicles (UAVs) provide a solution to this by providing radio-based data muling services, but, as this area is still unexplored, the utility of this solution is unclear. In this paper, we examine the theoretical expectations, perform several field experiments, and analyze the communication success rates of 802.15.4 radios near the water surface both communicating between surface nodes as well as between a node and the UAV. These indicate that on the water surface internode radio communication is poor, but node to UAV communication can provide both reasonable ranges and success rates. We additionally measure and analyze the energy aspects of the systems, determining the impacts of parameters such as network size and distance between nodes on the UAV energy. Finally, we consolidate the information into an algorithm outlining how to configure and design hybrid UAV and underwater sensor network systems.

Remote sensing by Unmanned Aerial Vehicles (UAVs) is changing the way agriculture operates by increasing the spatial-temporal resolution of data collection. Micro-UAVs have the potential to further improve and enrich the data collected by operating close to the crops, enabling the collection of higher spatio-temporal resolution data. In this paper, we present a UAV-mounted measurement system that utilizes a laser scanner to compute crop heights, a critical indicator of crop health. The system filters, transforms, and analyzes the cluttered range data in real-time to determine the distance to the ground and to the top of the crops. We assess the system in an indoor testbed and in a corn field. Our findings indicate that despite the dense canopy and highly variable sensor readings, we can precisely fly over crops and measure its height to within 5cm of measurements gathered using current measurement technology.

Robots improve wireless sensor network (WSN) deployments by reducing deployment times, deploying nodes to improve coverage, and ferrying data. Utilizing Unmanned Aerial Vehicles (UAVs) to install sensor networks in environmentally sensitive areas is especially valuable, as the UAVs are able to quickly traverse rough and environmentally sensitive terrain. UAV based deployments are challenging, as the UAVs may need to install nodes in a specific orientation or location type, which is difficult to sense from a UAV. We present our work towards resolving these difficulties by first classifying the surface a UAV has landed on, and then conducting a post-deployment analysis of the installation.

System monitoring can help to detect abnormalities and avoid failures. Crafting monitors for today's robotic systems, however, can be very difficult due to the systems' inherent complexity. In this work we address this challenge through an approach that automatically infers system invariants and synthesizes those invariants into monitors. The approach is novel in that it derives invariants by observing the messages passed between system nodes and the invariants types are tailored to match the spatial, temporal, and operational attributes of robotic systems. Further, the generated monitor can be seamlessly integrated into systems built on top of publish-subscribe architectures. An application of the technique on a system consisting of a unmanned aerial vehicle (UAV) landing on a moving platform shows that it can significantly reduce the number of crashes in unexpected landing scenarios.

Wireless magnetic resonant power transfer is an emerging technology that has many advantages over other wireless power transfer methods due to its safety, lack of interference, and efficiency at medium ranges. In this paper, we develop a wireless magnetic resonant power transfer system that enables unmanned aerial vehicles (UAVs) to provide power to, and recharge batteries of wireless sensors and other electronics far removed from the electric grid. We address the difficulties of implementing and outfitting this system on a UAV with limited payload capabilities and develop a controller that maximizes the received power as the UAV moves into and out of range. We experimentally demonstrate our prototype wireless power transfer system by using a UAV to transfer nearly 5W of power to a ground sensor.

Quad-rotor Micro Aerial Vehicles (MAVs) are used widely in research and increasingly in commercial applications as the cost of these platforms has dropped. The cost of entry, however, is still high in large part due to the time and effort involved in repairing vehicles after crashes while learning about the system design and dynamics. In this paper, we present an omni-directional hovercraft, which has dynamics similar to MAVs and can be used as an educational platform to teach students about the behavior and control of MAV-like platforms with minimal cost and effort. Teaching students about the capabilities and challenges associated with MAVs is critical for educating future engineers and scientists that will develop and use the next generation of MAVs. In addition, the hovercraft provides a safe platform for researchers to test control and coordination algorithms before trying them on higher-cost MAVs.

We present a control algorithm for autonomous underwater robots with modular thruster configuration. The algorithm can handle arbitrary thruster configurations. It maintains the robot's desired attitude while solving for translational motion. The attitude can be arbitrarily chosen from the special orthogonal group SO3allowing the robot all possible orientations. The desired translational velocities can be chosen from R3allowing the robot to follow arbitrary trajectories underwater. If the robot is not fully holonomic then the controller chooses the closest possible solution using least squares and outputs the error vector. We verify the controller with experiments using our autonomous underwater robot AMOUR. We achieve roll errors of 1.0 degree (2.1 degrees standard deviation) and pitch errors of 1.5 degrees (1.8 degrees standard deviation). We also demonstrate experimentally that the controller can handle both nonholonomic and fully holonomic thruster configurations of the robot. In the later case we show how depth can be maintained while performing 360 degree rolls. Further, we demonstrate an input device that allows a user to control the robot's attitude while moving along a desired trajectory.

Underwater vehicles are typically operated using a tether or a slow acoustic link. We present an underwater optical communication system that enables a high-throughput and low-latency link to an underwater robot. The optical link allows the robot to operate in cluttered environments without the need for a tether. We demonstrate the performance of the system in a number of experiments which characterize the optical link and demonstrate remote control of the robot using a human input device.

Understanding the dynamics of bodies of water and their impact on the global environment requires sensing information over the full volume of water. We develop a gradient-based decentralized controller that dynamically adjusts the depth of a network of underwater sensors to optimize sensing for computing maximally detailed volumetric models. We prove that the controller converges to a local minimum. We implement the controller on an underwater sensor network capable of adjusting their depths. Through simulations and experiments, we verify the functionality and performance of the system and algorithm.

This paper describes a novel experiment in which two very different methods of underwater robot localization are compared. The first method is based on a geometric approach in which a mobile node moves within a field of static nodes, and all nodes are capable of estimating the range to their neighbours acoustically. The second method uses visual odometry, from stereo cameras, by integrating scaled optical flow. The fundamental algorithmic principles of each localization technique is described. We also present experimental results comparing acoustic localization with GPS for surface operation, and a comparison of acoustic and visual methods for underwater operation.

This paper explores the space of active modular trusses, ranging from a passive truss with one independent active climbing module to fully self-reconfiguring dynamically controllable trusses comprised of active modules and passive struts. We describe a hardware design for truss climbing and present hierarchical algorithms for controlling hyper-redundant modular trusses