Aerial robots are becoming more commonplace within society, but there is limited research about how people interact with these robots in common social situations. In this work, we seek to understand 1) people's natural perceptions and responses to aerial robot flight paths, 2) how people expect to interact with an aerial robot in assorted contexts, and 3) how that work can be scaled up to a multi-human approach. The first two components are prior work where we have conducted both virtual surveys and in-person design sessions with participants. For the next phase, we hope to combine the work exploring the flight paths and situational context, to then expand it from one robot-one human interaction to a multi-human interaction.

The primary objective of this paper is to introduce a new monocular depth estimation (MDE) model targeting under-represented environments using a novel dataset combining synthetic and real images. The proposed model is small and fast to allow use for UAS navigation and data collection in rainforest environments. Prior works on MDEs target outdoor environments while focusing on urban, ground-level viewpoints due to interest in self-driving or autonomous package delivery applications and data availability. However, under-represented environments, such as rainforests, can benefit from targeted, environment-specific MDEs because existing general MDEs cannot adapt to extreme environmental differences, leading to high error rates. Our model is trained using a distinct rainforest dataset that combines images generated using a synthetic dataset pipeline and depth images collected from aerial robot deployments in the Children’s Eternal Rainforest in Costa Rica. The proposed model will allow for improved rainforest navigation without using expensive LIDAR sensors and can improve the navigation of a UAS in rainforest environments by providing more accurate and useful measurements for object manipulation, such as leaf sampling. Our model outperforms MiDaS across the board and has over a 75% improvement, specifically in the relative error metrics, while maintaining a low runtime. The resulting model matches the performance of state-of-the-art monocular depth estimation models designed for common environments, i.e., urban and indoor environments, and outperforms them when used in a rainforest environment.

Human–robot proxemics behaviors can vary based on personal, robot, and environmental factors which, along with their deployment in public-facing interactions, calls for an in-depth exploration. This article explores the impact of altitude and safety modifications of small unmanned aerial vehicle (sUAV) on users’ comfortable interaction distance. By leveraging interaction techniques from literature like video, sound, and simulations, we explore personal space interactions in online studies (N = 376) with the sUAV and the Double telepresence robot. We then compare the findings with our in-person interaction data (N = 47). While in-person interactions are the ultimate goal, online methods can be used to reduce resources, allow larger sample sizes, and may lead to a more comprehensive sampling of population than would be expected from in-person studies. The lessons learned from this work are applicable broadly within the social robotics community, even outside those who are interested in proxemics interactions, to conduct future crowd-sourced experiments. The various modalities provided similar trends when compared with data from in-person studies. While the distances may not have been precise compared to those measured in the real world, these experiments are useful to detect patterns in human–robot interactions, and to conduct formative studies before committing resources to in-person testing.

The purpose of this project is to understand how people would expect to interact with an Unmanned Aerial Vehicle (UAV) in a social indoor environment under friendly, neutral, or adversarial contexts. The three environments will include one setting with the UAV serving as a tour guide, one as a security guard, and one as a food delivery mechanism. This work is novel in its inquiry into the affective nature of the interaction, comparison across situational contexts, and ability to compare preferences both within and between participants.Our findings will help researchers plan for appropriate safety and comfort measures, while being cognizant of the participants' preferences for and understanding of how drones operate. This study examines realistic indoor scenarios for which each participant designs their preferred interaction and presents exploratory results, including comparison to prior work with respect to motion gestures and comfortable approach distances. Initial findings suggest the importance of visibility of approaches, selecting approach heights relative to the person and based on the context of interaction, and criticality of the initial direction of motion when classifying the communicative content of UAV flight paths.

Assessing proficiency in small unmanned aerial vehicles (sUAVs) pilots is complex and not well understood, but increasingly important to employ these vehicles in serious jobs such as wildland firefighting and infrastructure inspection. The limited prior work with UAVs has focused on user training using modalities like simulators and VR and no performance assessments with line-of-sight UAVs. This paper presents a training methodology for novice pilots of sUAVs. We presented two studies: the Baseline study (21 participants) and the Training study (16 participants). Our work is of interest to sUAV operators, regulators, and companies developing this technologies to produce a more capable workforce capable of consistent, safe operations. We successfully utilized the method developed in [1] to assess user proficiency in flying UAVs. We presented a UAV pilot training schedule for novice users (in the Training study), and were able to determine the minimum training time necessary to observe performance gains and mitigate damage. Results indicate that task completions noticeably improved and crashes minimized by day 10 of training, with a training plateau observed by day 15.

The primary objective of this study was to utilize state-of-the-art deep learning-based monocular depth estimation models to assist UAS pilots in rainforest canopy data collection and navigation. Monocular depth estimation models provide a complementary technique to other depth measurement and estimation techniques to extend the range and improve measurements. Several state-of-the-art models were evaluated using a novel dataset composed of data from a simulated rainforest environment. In the evaluation, MiDaS outperformed the other models, and a segmentation pipeline was designed using this model to identify the highest areas of the canopies. The segmentation pipeline was evaluated using 1080p and 360p input videos from the simulated rainforest dataset. It was able to achieve an IoU of 0.848 and 0.826 and an F1 score of 0.915 and 0.902 at each resolution, respectively. We incorporated the proposed depth-estimation-based segmentation pipeline into an example application and deployed it on an edge system. Experimental results display the capabilities of a UAS using the segmentation pipeline for rainforest data collection.

Unmanned aerial vehicles (UAVs) are becoming more common, presenting the need for effective human-robot communication strategies that address the unique nature of unmanned aerial flight. Visual communication via drone flight paths, also called gestures, may prove to be an ideal method. However, the effectiveness of visual communication techniques is dependent on several factors including an observer's position relative to a UAV. Previous work has studied the maximum line-of-sight at which observers can identify a small UAV [1]. However, this work did not consider how changes in distance may affect an observer's ability to perceive the shape of a UAV's motion. In this study, we conduct a series of online surveys to evaluate how changes in line-of-sight distance and gesture size affect observers' ability to identify and distinguish between UAV gestures. We first examine observers' ability to accurately identify gestures when adjusting a gesture's size relative to the size of a UAV. We then measure how observers' ability to identify gestures changes with respect to varying line-of-sight distances. Lastly, we consider how altering the size of a UAV gesture may improve an observer's ability to identify drone gestures from varying distances. Our results show that increasing the gesture size across varying UAV to gesture ratios did not have a significant effect on participant response accuracy. We found that between 17 m and 75 m from the observer, their ability to accurately identify a drone gesture was inversely proportional to the distance between the observer and the drone. Finally, we found that maintaining a gesture's apparent size improves participant response accuracy over changing line-of-sight distances.

This body of work presents an iterative process of refinement to understand naive perception of communication using the motion of an unmanned aerial vehicle (UAV). This includes what people believe the UAV is trying to communicate, and how they expect to respond through physical action or emotional response. Previous work in this area sought to communicate without clear definitions of the states attempting to be conveyed. In an attempt to present more concrete states and better understand specific motion perception, this work goes through multiple iterations of state elicitation and label assignment. The lessons learned in this work will be applicable broadly to those interested in defining flight paths, and within the human-robot interaction community as a whole, as it provides a base for those seeking to communicate using non-anthropomorphic robots. We found that the Negative Attitudes towards Robots Scale (NARS) can be an indicator of how a person is likely to react to a UAV, the emotional content they are likely to perceive from a message being conveyed, and it is an indicator for the personality characteristics they are likely to project upon the UAV. We also see that people commonly associate motions from other non-verbal communication situations onto UAVs. Flight specific recommendations are to use a dynamic retreating motion from a person to encourage following, use a perpendicular motion to their field of view for blocking, simple descending motion for landing, and to use either no motion or large altitude changes to encourage watching. Overall, this research explores the communication from the UAV to the bystander through its motion, to see how people respond physically and emotionally. Advisor: Brittany A. Duncan

Unmanned Aerial Vehicle (UAV) flight paths have been shown to communicate meaning to human observers, similar to human gestural communication. This paper presents the results of a UAV gesture perception study designed to assess how observer viewpoint perspective may impact how humans perceive the shape of UAV gestural motion. Robot gesture designers have demonstrated that robots can indeed communicate meaning through gesture; however, many of these results are limited to an idealized range of viewer perspectives and do not consider how the perception of a robot gesture may suffer from obfuscation or self-occlusion from some viewpoints. This paper presents the results of three online user-studies that examine participants' ability to accurately perceive the intended shape of two-dimensional UAV gestures from varying viewer perspectives. We used a logistic regression model to characterize participant gesture classification accuracy, demonstrating that viewer perspective does impact how participants perceive the shape of UAV gestures. Our results yielded a viewpoint angle threshold from beyond which participants were able to assess the intended shape of a gesture's motion with 90% accuracy. We also introduce a perceptibility score to capture user confidence, time to decision, and accuracy in labeling and to understand how differences in flight paths impact perception across viewpoints. These findings will enable UAV gesture systems that, with a high degree of confidence, ensure gesture motions can be accurately perceived by human observers.

This paper presents a gesture set for communicating states to novice users from a small Unmanned Aerial System (sUAS) through an elicitation study comparing gestures created by participants recruited from the general public with varying levels of experience with an sUAS. Previous work in sUAS flight paths sought to communicate intent, destination, or emotion without focusing on concrete states such as Low Battery or Landing. This elicitation study uses a participatory design approach from human-computer interaction to understand how novice users would expect an sUAS to communicate states, and ultimately suggests flight paths and characteristics to indicate those states. We asked users from the general public (N=20) to create gestures for seven distinct sUAS states to provide insights for human-drone interactions and to present intuitive flight paths and characteristics with the expectation that the sUAS would have general commercial application for inexperienced users. The results indicate relatively strong agreement scores for three sUAS states: Landing (0.455), Area of Interest (0.265), and Low Battery (0.245). The other four states have lower agreement scores, however even they show some consensus for all seven states. The agreement scores and the associated gestures suggest guidance for engineers to develop a common set of flight paths and characteristics for an sUAS to communicate states to novice users.

Robotic rescuers digging through rubble, fire-fighting drones flying over populated areas, robotic servers pouring hot coffee for you, and a nursing robot checking your vitals are all examples of current or near-future situations where humans and robots are expected to interact in a dangerous situation. Dangerous HRI is an as-yet understudied area of the field. We define dangerous HRI as situations where humans experience some amount of risk of bodily harm while interacting with robots. This interaction could take many forms, such as a bystander (e.g. when an autonomous car waits at a crossing for a pedestrian), as a recipient of robotic assistance (rescue robots), or as a teammate (like an autonomous robot working with a SWAT team). To facilitate better study of this area, the Dangerous HRI workshop brings together researchers who perform experiments with some risk of bodily harm to participants and discuss strategies for mitigating this risk while still maintaining validity of the experiment. This workshop does not aim to tackle the general problem of human safety around robots, but instead focused on guidelines for and experience from experimenters.

Users play an integral role in the performance of many robotic systems, and robotic systems must account for differences in users to improve collaborative performance. Much of the work in adapting to users has focused on designing teleoperation controllers that adjust to extrinsic user indicators such as force, or intent, but do not adjust to intrinsic user qualities. In contrast, the Human-Robot Interaction community has extensively studied intrinsic user qualities, but results may not rapidly be fed back into autonomy design. Here we provide foundational evidence for a new strategy that augments current shared control, and provide a mechanism to directly feed back results from the HRI community into autonomy design. Our evidence is based on a study examining the impact of the user quality “locus of control” on telepresence robot performance. Our results support our hypothesis that key user qualities can be inferred from human-robot interactions (such as through path deviation or time to completion) and that switching or adaptive autonomies might improve shared control performance.

This project seeks to generate small Unmanned Aerial System (sUAS) flight paths that are broadly understood by the general population and can communicate states about both the sUAS and its understanding of the world. Previous work in sUAS flight paths has sought to communicate intent, destination, or emotion of the system without focusing on concrete states (e.g., low battery, landing, etc.). This work leverages biologically-based flight paths and experimental methodologies from human-human and human-humanoid robot interactions to assess the understanding of avian flight paths to communicate sUAS states to novice users. If successful, this work should inform: the human-robot interaction community about the perception of flight paths, sUAS manufacturers on how their systems could communicate with both operators and bystanders, and end users on ways to communicate with others when flying systems in public spaces. General design implications and future directions of work are suggested to build on the results here, which suggest that novice users gravitate towards labels they understand (draw attention and landing) while avoiding more technical labels (lost sensor).

Prescribed fires can lessen wildfire severity and control invasive species, but they can also be risky and costly. Unmanned aerial systems can reduce those drawbacks by, for example, dropping ignition spheres to ignite the most hazardous areas. Existing systems, however, lack awareness of the fire vectors to operate autonomously, safely, and efficiently. In this work we address that limitation, introducing an approach that integrates a lightweight fire simulator and a planner for trajectories and ignition sphere drop waypoints. Both components are unique in that they are amenable to input from the system's sensors and the fire crew to increase fire awareness. We conducted a preliminary study that confirms that such inputs improve the accuracy of the fire simulation to counter the unpredictability of the target environment. The field study of the system showed that the fire-aware planner generated safe trajectories with effective ignitions leveraging the fire simulator predictions.

This paper presents an investigation of human comfort with a small Unmanned Aerial Vehicle (sUAV) through a study offering a comparison of comfort with a sUAV versus a ground vehicle. Current research on human comfort with sUAVs has been limited to a single previous study, which did not include free flight, and while ground vehicle distancing has been studied, it has never been directly compared to a sUAV. The novelty in the approach is the use of a motion capture room to achieve smooth trajectories and precise measurements, while conducting the first free flight study to compare human comfort after interaction with aerial versus ground vehicles (within subjects, N=16). These results will contribute to understanding of social, collaborative, and assistive robots, with implications for general human-robot interactions as they evolve to include aerial vehicles. Based on the reduced stress and distance (36.5cm or 1.2ft) for ground vehicles and increased stress and distance (65.5cm or 2.15ft) for sUAVs, it is recommended that studies be conducted to understand the implications of design features on comfort in interactions with sUAVs and how they differ from those with ground robots.

This article will present a simulation-based approach to testing multiple variables in the behavior of a small Unmanned Aerial Vehicle (sUAV), inspired by insect and animal motions, to understand how these variables impact time of interaction, preference for interaction, and distancing in Human-Robot Interaction (HRI). Previous work has focused on communicating directionality of flight, intentionality of the robot, and perception of motion in sUAVs, while interactions involving direct distancing from these vehicles have been limited to a single study (likely due to safety concerns). This study takes place in a Cave Automatic Virtual Environment (CAVE) to maintain a sense of scale and immersion with the users, while also allowing for safe interaction. Additionally, the two-alternative forced-choice method is employed as a unique methodology to the study of collocated HRI in order to both study the impact of these variables on preference and allow participants to choose whether or not to interact with a specific robot. This article will be of interest to end-users of sUAV technologies to encourage appropriate distancing based on their application, practitioners in HRI to understand the use of this new methodology, and human-aerial vehicle researchers to understand the perception of these vehicles by 64 naive users. Results suggest that low speed (by 0.27m, p < 0.02) and high cyclicity (by 0.28m, p < 0.01) expressions can be used to increase distancing; that low speed (by 4.4s, p < 0.01) and three-dimensional (by 2.6s, p < 0.01) expressions can be used to decrease time of interaction; and low speed (by 10.4%, p < 0.01) expressions are less preferred for passability in human-aerial vehicle interactions.

This work provides a case study of both fixed-wing and rotorcraft small unmanned aerial systems (SUAS) used in a deployment at the SR530 mudslides in Washington state and compares the types of flight paths used by each vehicle type. Previously aerial imagery from SUAS have produced 2D and 3D reconstructions of simple terrain, but have not been used in complex terrain which encompasses both flat areas and drastic changes in the height of ground level, such as a mudslide. In this deployment, both types of SUAS platforms were used to collect imagery over terrain varied nearly 200m in elevation but different paths were used due to the complexity of the terrain, safety, privacy, and platform-specific limitations. The deployment found that paths with fixed-wing platforms can be thought of as stacked horizontal planes while rotorcraft can cover complex terrain with a set of vertical planes. The different paths contribute to autonomous path planning, particularly to accommodate vertical planes, and to general understanding of how different SUAS can be applied to challenging terrains. Future work in path planning should incorporate Geographic Information Systems (GIS) information to facilitate flight paths in vertical planes and to maintain altitude restrictions relative to radically changing elevations of a landscape.

Sky Writer is a collaborative communication medium that augments the traditional display of a UAV pilot and allows other stakeholders to communicate their needs and intentions to the pilot. UAV pilots engaging in time-critical missions, such as urban disaster responses, often must allocate most of their cognitive capacity towards flight tasks, making communication and collaboration with other stakeholders difficult or dangerous. Sky Writer addresses the needs of stakeholders while requiring minimal cognitive effort from the UAV pilot. The application presents stakeholders with an interface that provides contextual flight information and a live video stream of the flight. Stakeholders are able to sketch directly on the video stream or use a spotlight indicator that is mirrored across all displays in the system, including the pilot's display. The application can be used in any modern web browser and works with traditional and touch devices. Concept experimentation performed at Disaster City with two pilots indicated that the spotlight feature was particularly useful while the UAV was in motion, and the sketching features were most useful while the UAV was stationary. The system will be tested with professional responders soon to determine its efficacy in a simulated response, and to inform the ongoing design process.

This paper presents the first known human-subject study of comfortable approach distance and height for human interaction with a small unmanned aerial vehicle (sUAV), finding no conclusive difference in comfort with a sUAV approaching a human at above head height or below head height. Understanding the amount, if any, of discomfort introduced by a sUAV flying in close proximity to a human is critical for law enforcement, crowd control, entertainment, or flying personal assistants. Previous work has focused on how humans interact with each other or with unmanned ground vehicles, and the experimental methods typically rely on the human participant to consciously express distress. The approach taken was to duplicate the experimental set up in human proxemics studies, while adding psychophysiological sensing, under the hypothesis that human-robot interaction will mirror human-human interaction. The 16 participant, within-subjects experiment did not confirm this hypothesis. Instead a sUAV above height of a “tall” person in human experiments (2.13 m) did not produce statistically different heart rate variability nor cause the participant to stop the robot further away than for a sUAV at a “short” height (1.52 m). The lack of effect may be due to two possible confounds: i) duplicating prior human proxemics experiments did not capture how a sUAV would likely move or interact and ii) telling the participants that the robot could not hurt them. Despite possible confounding, the results raise the question of whether human-human psychological and physical distancing behavior transfers to human-aerial robot interactions.

The introduction of two types of unmanned aerial vehicles into a production of A Midsummer Night's Dream suggests that social proof informs untrained human groups. We describe the metaphors used in instructing actors, who were otherwise untrained and inexperienced with robots, in order to shape their expectations. Audience response to a robot crash depended on whether the audience had seen how the actors interacted with the robot "baby fairies." If they had not seen the actors treating a robot gently, an audience member would likely throw the robot expecting it to fly or handle it roughly. If they had seen the actors with the robots, the audience appeared to adopt the same gentle style and mechanisms for re-launching the micro-helicopter. The difference in audience behavior suggests that the principle of social proof will govern how untrained humans will react to robots.

This paper describes the Survivor Buddy human-robot interaction project and how it was used by four middle-school girls to illustrate the scientific process for an episode of "SciGirls", a Public Broadcast System science reality show. Survivor Buddy is a four degree of freedom robot head, with the face being a MIMO 740 multi-media touch screen monitor. It is being used to explore consistency and trust in the use of robots as social mediums, where robots serve as intermediaries between dependents (e.g., trapped survivors) and the outside world (doctors, rescuers, family members). While the SciGirl experimentation was neither statistically significant nor rigorously controlled, the experience makes three contributions. It introduces the Survivor Buddy project and social medium role, it illustrates that human-robot interaction is an appealing way to make robotics more accessible to the general public, and raises interesting questions about the existence of a minimum set of degrees of freedom for sufficient expressiveness, the relative importance of voice versus non-verbal affect, and the range and intensity of robot motions.