Drones are increasingly used in firefighting to support situational awareness. Yet, current designs and existing practices hinder information flow between humans and machines. Through interviews (N = 12), we explore firefighters’ perspectives on integrating drones into their main remote communication channel, namely, the radio. We examined current radio usage, the state of drone deployment, and gathered their feedback on a futuristic drone scenario. Our findings span from existing organizational and communication strategies to how drones are, and could further be, integrated into existing practices. We uncovered structured practices that could facilitate the integration of robotic agents. Firefighters further suggested specific requirements for drone communication over the radio, such as concise and timely messaging. We propose design recommendations for drones as radio-communicating agents, bridging established low-tech practices with emerging autonomy. This work demonstrates the feasibility of drones as radio-integrated teammates and establishes principles for designing them as reliable situationally-aware agents in safety-critical contexts.

Conveying emotions via touch (i.e., affective touch) is a crucial part of social interaction and bonding. Socially assistive robots have widely used affective touch. However, drones being aerial robots, touch has only been explored as input for control, navigation, or command. Thus, this work investigates the potential of affective touch using a perched socially assistive drone. We address two critical topics: (1) using affective touch as a modality with drones and (2) implementing effective touch gestures for emotional communication with drones. In an user study (N = 30), we explored participants’ preferences for interaction modalities and mapped touch gestures to emotions. Results show that touch was the most preferred modality. Afterward, participants selected touch gestures to convey 16 emotions to a drone. We determined which gesture best fits each emotion. Our work highlights the relevance of affective haptics and sheds light on the nature of touch gestures’ emotional conveyance in Human-Drone Interaction.

Figure 1: Current status and future vision of drone-assisted firefighting in Thailand: enhancing situational awareness, resource management, and decision-making in forest and industrial fire emergencies.

The growing presence of drones in human spaces has sparked curiosity regarding their role as social creatures. One approach to conveying the social aspects of robotic devices is to incorporate emotions. However, the social effects of emotions depend on their perceived appropriateness by humans. In this work, we investigate the psychological effects of appropriate vs. inappropriate emotions displayed on a drone in a delivery scenario. Through an online study (N =97), we observe significant differences in how people ascribe social attributes to a drone, understand both its functional acceptance (e.g., ease of use, usefulness), social acceptance (e.g., drone as an interaction partner), and assess its social competencies and human-like attributes. Overall, for a given situation of interaction, the drone is perceived more (resp. less) positively when displaying appropriate (resp. inappropriate) emotions. We conclude with a discussion on the use of emotions in drones and their psychological effects on users. This work contributes to a deeper understanding of emotion appropriateness and social interactions in robotics.

Small-size autonomous vehicles have reached a level of technological maturity that enables their use as efficient means for goods delivery. Yet, current designs of both ground and aerial vehicles fail to address potential information needs of bystanders: What is this robot doing? Can it harm me? Does it need help? Through a series of six co-design workshops ( \(N = 22\) ), we iteratively explore two types of autonomous healthcare delivery vehicles, namely aerial and ground robots, with a focus on bystanders’ perspectives and interaction needs between bystanders and AVs. The resulting design propositions were then synthesized into functional scale-model prototypes of a healthcare delivery robot and drone. We validate our co-design findings by conducting two laboratory Wizard-of-Oz experiments ( \(N = 24\) ). Our results demonstrate how familiar colors and iconography support goal sharing and bystanders in recognizing the delivery vehicle’s purpose and intentions. Mapping interaction needs of bystanders, we find contextual effects on bystander attitudes and the effectiveness of communication modalities. This work contributes to the design of future, socially acceptable small-size delivery vehicles fit for human spaces.

Research in affective robotics has been using emotions to improve human-robot interaction. One important aspect has been to design recognizable and believable emotions in robotics. Recent work argued that externally displayed emotions on robots may or may not be appropriate for a given situation. However, the selection of emotions as appropriate/inappropriate is not trivial. We here examine the practicality of an established model to craft for emotion appropriateness based on situations of interaction. To do so, we explored the use of the Ortony, Clore, and Collins (OCC) model, which provides a psychological framework of appraisal in which the characteristics of situations are defined and connected to emotions, to identify emotion categories and create contrasting perceptions of emotion appropriateness. We then mapped these categories to four recognizable emotions on aerial robots and designed two video clips (3min35s each) of respectively appropriate and inappropriate emotions. The clips were evaluated in an online study ( N =100) where significant differences were found in attitudes toward the robot's emotions. This paper contributes initial findings to designing for emotion appropriateness.

Assigning lifelike qualities to robotic agents (Anthropomorphism) is associated with complex affective interpretations of their behavior. These anthropomorphized perceptions are traditionally elicited through robots' designs. Yet, aerial robots (or drones) present a special case due to their – traditionally – non-anthropomorphic design, and prior research shows conflicting evidence on their perception as either person-like, animal-like, or machine-like. In this work, we explore how people perceive drones in a cross-dimensional space between these three dimensions by varying the affective state presented on the drone. To capture these perceptions, we developed a novel measurement instrumentAnZoMa. We describe the design, use, and deployment of the instrument in an online study (N=98). The study results suggest that different drone emotions triggered people to attribute various characteristics to the drone (e.g., interaction metaphors, traits, and features) and variations in acceptability of drone affective states. These results demonstrate the interdependencies between affective perceptions and anthropomorphism of drones. We conclude by discussing the necessity to integrate cross-dimensional perception of anthropomorphism in human-drone interaction and affective computing. This work contributes a novel tool to measure the dimensions and gravity of anthropomorphism and insights into interdependencies between different affective states displayed on drones and their anthropomorphized perception.

Drones are increasingly used in situations where they can assist a person. However, people are not familiar with drones approaching them. We propose an expressive light system embedded on a drone to convey its intention to initiate communication, which we present as a four-stage process: Drone-Initiated Engagement Model (DIEM). We then describe the design and development of an LED-based prototype divided in two configurations with a total of 26 light animations, and report their evaluation in an online survey (N = 156). We describe the suitability of different configurations, animations, and colors to convey each stage of DIEM. We finally validate our system in a user study (N = 45) that showed that participants can perceive all four stages of approach via the drone movement and that expressive lights provide a more nuanced user experience. We contribute insights into the design of expressive light systems, as a stepping stone towards machine-initiated communication.

Interacting with flying objects has fueled people’s imagination throughout history. Over the past decade, the Human-Drone Interaction (HDI) community has been working towards making this dream a reality. Despite notable findings, we lack a high-level perspective on the current and future use cases for interacting with drones. We present a holistic view of domains and applications of use that are described, studied, and envisioned in the HDI body of work. To map the extent and nature of the prior research, we performed a scoping review (N=217). We identified 16 domains and over 100 applications where drones and people interact together. We then describe in depth the main domains and applications reported in the literature and further present under-explored use cases with great potential. We conclude with fundamental challenges and opportunities for future research in the field. This work contributes a systematic step towards increased replicability and generalizability of HDI research.

Drones are rapidly populating human spaces, yet little is known about how these flying robots are perceived and understood by humans. Recent works suggested that their acceptance is predicated upon their sociability. This paper explores the use of facial expressions to represent emotions on social drones. We leveraged design practices from ground robotics and created a set of rendered robotic faces that convey basic emotions. We evaluated individuals’ response to these emotional facial expressions on drones in two empirical studies (N = 98, N = 98). Our results demonstrate that individuals accurately recognize five drone emotional expressions, as well as make sense of intensities within emotion categories. We describe how participants were emotionally affected by the drone, showed empathy towards it, and created narratives to interpret its emotions. As a consequence, we formulate design recommendations for social drones and discuss methodological insights on the use of static versus dynamic stimuli in affective robotics studies.

This work presents a user-centered perspective on the integration of semi-autonomous drones in firefighting. We first present the outcomes of an interview with the leading officers in Israel, which revealed opportunities and barriers in drone integration, as well as current usage and future needs. We envision a future where firefighters and drones can directly communicate with each other and propose the use of gestures in collocated environments. We then present the results of an elicitation study with operational firefighters (N=9), which resulted in a set of tasks for drones and matching user-defined gestures that can be performed in highly constrained environments. This research opens the space for future collocated human-drone interaction in emergency response and other complex professional settings.

We envision the human body as a platform for fast take-off and landing of drones in entertainment and professional uses such as medical emergencies, rescue missions, or supporting police units. This new interaction modality challenges our knowledge of human-drone experiences, in which interaction usually occurs at a distance from the body. This work explores important factors for understanding the interplay between drones and humans. We first investigated the suitability of various body locations for landing in an online study (N = 159). Our results, presented as body maps, show that the hand and upper back are particularly well-suited body locations. We further tested these findings in a follow-up study (N = 12), in which participants experienced drones landing on their bodies through carefully designed and pre-recorded 360°videos. This immersion into the landing scenarios helped us to identify common themes and research approaches for different body parts. Taken together, the findings provide first insights into location preferences and themes for drones landing on the human body.

Drone technologies represent a new category of mobile devices that are increasingly present in public spaces. They are becoming increasingly autonomous, featuring a wide range of capabilities from detecting objects to monitoring situations. Yet, little is known about the characteristics that influence their acceptability in public spaces. In this work, we investigate how people’s attitude towards drone capabilities is influenced by the context in which they are operating. We present three user studies: first, a participatory design study (N=5) in which we investigated people’s expectations towards drone capabilities and contexts of use; second, a pre-study (N=18) performed to select 6 contexts of use for public drones with three different severity levels; and third, a survey-based study (N=26) where we evaluated people’s attitude towards 10 drone capabilities in 6 contexts of varying severity levels. Our results demonstrate that people’s attitude towards drone capabilities is more positive for severe contexts. In addition, we found positive correlations for all capabilities between attitude and perceived severity of context. This work contributes to the design of context-sensitive human-drone interactions and to the future integration of public drones.

New technology is moving towards intuitive and natural interaction techniques that are increasingly embedded in human space (e.g., home and office environment) and aims to support multiple users, yet their interfaces do not cover it to the full. Imagine that you have a multi-user device, should it act differently in different situations, people, and group settings? Current Multi-User Interfaces address each of the users as an individual that works independently from others, and there is a lack of understanding of the mechanisms that impact shared usage of these products. Thus we have linked environmental (external) and user-centered (internal) factors to the way users interact with multi-user devices. We analyzed 124 papers that involve multi-user interfaces and created a classification model out of 8 factors. Both the model and factors were validated by a large-scale online study. Our model defines the factors affecting multi-user usage with a single device and leads to a decision on the most important ones in different situations. This paper is the first to identify these factors and to create a set of practical guidelines for designing Multi-User Interfaces.

Drones are an exciting technology that is quickly being adopted in the global consumer market. Africa has become a center of deployment with the first drone airport established in Rwanda and drones currently being used for applications such as medical deliveries, agriculture, and wildlife monitoring. Despite this increasing presence of drones, there is a lack of research on stakeholders' perspectives from this region. We ran a human-drone interaction user study (N=15) with experts from several sub-Saharan countries using a co-design methodology. Participants described novel applications and identified important design aspects for the integration of drones in this context. Our results highlight the potential of drones to address real world problems, the need for them to be culturally situated, and the importance of considering the social aspects of their interaction with humans. This research highlights the need for diverse perspectives in the human-drone interaction design process.

We put forth Physiologically Driven Storytelling, a new approach to interactive storytelling where narratives adaptively unfold based on the reader's physiological state. We first describe a taxonomy framing how physiological signals can be used to drive interactive systems both as input and output. We then propose applications to interactive storytelling and describe the implementation of a software tool to create Physiological Interactive Fiction (PIF). The results of an online study (N=140) provided guidelines towards augmenting the reading experience. PIF was then evaluated in a lab study (N=14) to determine how physiological signals can be used to infer a reader's state. Our results show that breathing, electrodermal activity, and eye tracking can help differentiate positive from negative tones, and monotonous from exciting events. This work demonstrates how PIF can support storytelling in creating engaging content and experience tailored to the reader. Moreover, it opens the space to future physiologically driven systems within broader application areas.

Drones are becoming ubiquitous and offer support to people in various tasks, such as photography, in increasingly interactive social contexts. We introduce drone.io, a projected body-centric graphical user interface for human-drone interaction. Using two simple gestures, users can interact with a drone in a natural manner. drone.io is the first human-drone graphical user interface embedded on a drone to provide both input and output capabilities. This paper describes the design process of drone.io. We present a proof of concept, drone-based implementation, as well as a fully functional prototype for a drone tour-guide scenario. We report drone.io's evaluation in three user studies (N=27) and show that people were able to use the interface with little prior training. We contribute to the field of human-robot interaction and the growing field of human-drone interaction.

The consumer drone market has shown a constant growth for the past few years. As drones become increasingly autonomous and used for a growing number of applications, it is crucial to establish parameters for collocated human-drone interaction. Prior research showed how ground robots should approach a person to initiate interaction. This paper builds upon prior work and investigates how a flying robot should approach a person. Because of the flight capability, drones present more approach parameters than ground robots and require further study to properly design future interactions. Since research methodologies in aerial robotics are not well established, we present a taxonomy of methodologies for human-drone interaction studies to guide future researchers in the field. This paper then contributes a user study (N=24) investigating proximity, speed, direction, and trajectory towards a comfortable drone approach. We present our study results and design guidelines for the safe approach of a drone in a collocated indoor environment.

Digitally presenting physiological signals as biofeedback to users raises awareness of both body and mind. This paper describes the effectiveness of conveying a physiological signal often overlooked for communication: breathing. We present the design and development of digital breathing patterns and their evaluation along three output modalities: visual, audio, and haptic. We also present Breeze, a wearable pendant placed around the neck that measures breathing and sends biofeedback in real-time. We evaluated how the breathing patterns were interpreted in a fixed environment and gathered qualitative data on the wearable device's design. We found that participants intentionally modified their own breathing to match the biofeedback, as a technique for understanding the underlying emotion. Our results describe how the features of the breathing patterns and the feedback modalities influenced participants' perception. We include guidelines and suggested use cases, such as Breeze being used by loved ones to increase connectedness and empathy.

Our environments are increasingly being populated with intelligent devices and robots. People use digital assistants with speech interfaces to play music, find out about the weather, or call a taxi. Such interfaces are designed for single user control, and often fail when multiple people interact simultaneously. For instance, if two users keep asking for a different song, the digital assistant will keep changing the music with no regards for users' comfort or any conflict that may arise. As these devices are shared, it is crucial that the interface is built to consider more than one user. This work investigates which control schemes are suitable for multiple users to simultaneously interact with a single domestic cleaning robot. The control schemes will be tested with small groups of people who live together. Our work will inform researchers on how to build control schemes that are acceptable and suitable in shared environments.

Interactive fiction is a literary genre that is rapidly gaining popularity. In this genre, readers are able to explicitly take actions in order to guide the course of the story. With the recent popularity of narrative focused games, we propose to design and develop an interactive narrative tool for content creators. In this extended abstract, we show how we leverage this interactive medium to present a tool for interactive storytelling in virtual reality. Using a simple markup language, content creators and researchers are now able to create interactive narratives in a virtual reality environment. We further discuss the potential future directions for a virtual reality storytelling engine.

As drones become ubiquitous, it is important to understand how cultural differences impact human-drone interaction. A previous elicitation study performed in the USA illustrated how users would intuitively interact with drones. We replicated this study in China to gain insight into how these user-defined interactions vary across the two cultures. We found that as per the US study, Chinese participants chose to interact primarily using gesture. However, Chinese participants used multi-modal interactions more than their US counterparts. Agreement for many proposed interactions was high within each culture. Across cultures, there were notable differences despite similarities in interaction modality preferences. For instance, culturally-specific gestures emerged in China, such as a T-shape gesture for stopping the drone. Participants from both cultures anthropomorphized the drone, and welcomed it into their personal space. We describe the implications of these findings on designing culturally-aware and intuitive human-drone interaction.

Haptic interfaces are ideal in situations where visual/auditory attention is impossible, unsafe, or socially unacceptable. However, conventional (vibrotactile) wearable interfaces often possess a limited bandwidth for expressing information. We explore a novel form of tactile stimulation through brushing, and demonstrate BrushTouch, a wearable prototype for brushing haptics. We also present schemes for conveying information such as time and direction through multi-tactor wrist-worn haptic interfaces. To evaluate BrushTouch, two user studies were run, comparing it to a conventional vibrotactile wristband across a number of tasks in both lab and mobile conditions. We show that for certain cues brushing can be more accurately recognized than vibration, enabling more effective spatial schemes for presenting information through haptic means. We then show that BrushTouch is capable of greater information transfer using such cues. We believe that brushing, as with other non-vibrotactile haptic techniques, merits further investigation as potential vehicles for richer haptic feedback.

Smartwatches and activity trackers are becoming prevalent, providing information about health and fitness, and offering personalized progress monitoring. These wearable devices often offer multimodal feedback with embedded visual, audio, and vibrotactile displays. Vibrations are particularly useful when providing discreet feedback, without users having to look at a display or anyone else noticing, thus preserving the flow of the primary activity. Yet, current use of vibrations is limited to basic patterns, since representing more complex information with a single actuator is challenging. Moreover, it is unclear how much the user--s current physical activity may interfere with their understanding of the vibrations. We address both issues through the design and evaluation of ActiVibe, a set of vibrotactile icons designed to represent progress through the values 1 to 10. We demonstrate a recognition rate of over 96% in a laboratory setting using a commercial smartwatch. ActiVibe was also evaluated in situ with 22 participants for a 28-day period. We show that the recognition rate is 88.7% in the wild and give a list of factors that affect the recognition, as well as provide design guidelines for communicating progress via vibrations.

Drones are becoming more popular and may soon be ubiquitous. As they enter our everyday environments, it becomes critical to ensure their usability through natural Human-Drone Interaction (HDI). Previous work in Human-Robot Interaction (HRI) shows that adding an emotional component is part of the key to success in robots' acceptability. We believe the adoption of personal drones would also benefit from adding an emotional component. This work defines a range of personality traits and emotional attributes that can be encoded in drones through their flight paths. We present a user study (N=20) and show how well three defined emotional states can be recognized. We draw conclusions on interaction techniques with drones and feedback strategies that use the drone's flight path and speed.

Personal drones are becoming popular. It is challenging to design how to interact with these flying robots. We present a Wizard-of-Oz (WoZ) elicitation study that informs how to naturally interact with drones. Results show strong agreement between participants for many interaction techniques, as when gesturing for the drone to stop. We discovered that people interact with drones as with a person or a pet, using interpersonal gestures, such as beckoning the drone closer. We detail the interaction metaphors observed and offer design insights for human-drone interactions.

We introduce spatially-aware virtual workspaces for the mobile environment. The notion of virtual workspaces was initially conceived to alleviate mental workload in desktop environments with limited display real-estate. Using spatial properties of mobile devices, we translate this approach and illustrate that mobile virtual workspaces greatly improve task performance for mobile devices. In a first study, we compare our spatially-aware prototype (mSpaces) to existing context switching methods for navigating amongst multiple tasks in the mobile environment. We show that users are faster, make more accurate decisions and require less mental and physical effort when using spatially-aware prototypes. We furthermore prototype pSpaces and m+pSpaces, two spatially-aware systems equipped with pico-projectors as auxiliary displays to provide dual-display capability to the handheld device. A final study reveals advantages of each of the different configurations and functionalities when comparing all three prototypes. Drawing on these findings, we identify design considerations to create, manipulate and manage spatially-aware virtual workspaces in the mobile environment.

Projector phones, handheld game consoles and many other mobile devices increasingly include more than one display, and therefore present a new breed of mobile Multi-Display Environments (MDEs) to users. Existing studies illustrate the effects of visual separation between displays in MDEs and suggest interaction techniques that mitigate these effects. Currently, mobile devices with heterogeneous displays such as projector phones are often designed without reference to visual separation issues; therefore it is critical to establish whether concerns and opportunities raised in the existing MDE literature apply to the emerging category of Mobile MDEs (MMDEs). This paper investigates the effects of visual separation in the context of MMDEs and contrasts these with fixed MDE results, and explores design factors for Mobile MDEs. Our study uses a novel eye-tracking methodology for measuring switches in visual context between displays and identifies that MMDEs offer increased design flexibility over traditional MDEs in terms of visual separation. We discuss these results and identify several design implications.