As space agencies prepare for long-term lunar missions (ie. Lunar Base Camp), addressing the challenges of complex surface tasks is crucial. Integrating advanced robotic systems can alleviate physical and cognitive burdens on astronauts, enhancing mission success and safety. This work focuses on the design, development, and evaluation REACH, a handheld device and its UI for interacting with the modular DLR Lightweight Rover Unit 2 (LRU), to improve human-robot collaboration during lunar EVAs. A Human-In-The-Loop pilot study compares the effectiveness of the REACH device to traditional verbal commands within a modular robotic system. Our results show REACH improves user control, task efficiency, and satisfaction with its intuitive interface and real-time feedback. These findings suggest REACH, and other like systems, can streamline tasks operations and management, aligning with Artemis mission goals. This work is a crucial step toward optimizing Human-Robot Interaction, setting the stage for more effective space missions.

This paper investigates how conversational agents that are embedded in smart environments should present themselves socially. In an online study, we simulate a future space habitat in which "astronauts" (participants) interact with one or more agents to complete several tasks related to science, maintenance, and inventory. We examine effects of agent affiliation (affiliation with a user, affiliation with a domain, or affiliation with all users and all domains) and narrative perspective (first-vs. third-person references to parts of the environment) on mental models of the smart environment as one or multiple entities, trust, performance, and social variables. Our findings suggest that in this type of setting, interacting with a single agent may increase mental demand, and that agents that speak about embodied interaction in third person are perceived as more trustworthy and competent than agents that speak in first person.

Virtual assistants, vacuum robots, security systems, and other smart home technologies are rapidly advancing, evolving, and gaining popularity. This raises questions of how people envision future interactions with smart home systems and how they imagine the future roles of such technologies in society. We deployed an online study that collected fictional short stories from 60 participants about smart home interactions. We identified themes regarding the roles of smart home technologies, social interactions with AI, and concerns about data privacy in the context of the home. We describe our method, discuss insights from the stories that explicitly reflect possible futures and implicitly reflect the present, and make design recommendations based on our findings.

The drive for increasing adoption of HRI technolo-gies is evident through research and development of manufac-turing, social, medical, and service robot solutions. However, novel methods and metrics are required to overcome the barrier between fundamental HRI research and its adoption in real-world environments. Hence, the fourth installment of the annual workshop, 'Test Methods and Metrics for Effective HRI,’ seeks to identify novel and emerging test methods and metrics for the holistic assessment and assurance of HRI performance. Specifically, the focus is on identifying innovative methods for the evaluation of HRI performance and to advance the growth of the HRI community based on the principles of collaboration, data sharing, and repeatability. The goal of this workshop is to break the boundaries between the development and adoption of HRI technologies through the promotion of robust experimental design, test methods, and metrics for assessing interaction and interface designs. This workshop will have participants from var-ious sectors in the HRI research community including academia, industry, and government in order to accomplish its aims.

Designers of smart-home systems must make decisions about the perceived identities and interconnectedness of their various devices. To inform these decisions, we performed an online study to examine whether people perceive multiple devices in a smart home as different interfaces for the same system, devices that talk to each other, or independent devices. We manipulated the types of devices in the system (hetero-geneous!homogeneous), how the devices relayed commands to each other (implicit/explicit), and the task requested. Participants were flexible in how they interpreted the devices, presenting an opportunity for designers to select a suitable model.

Remotely programming robots to execute tasks often relies on registering objects of interest in the robot's environment. Frequently, these tasks involve articulating objects such as opening or closing a valve. However, existing human-in-the-loop methods for registering objects do not consider articulations and the corresponding impact to the geometry of the object, which can cause the methods to fail. In this work, we present an approach where the registration system attempts to automatically determine the object model, pose, and articulation for user-selected points using nonlinear fitting and the iterative closest point algorithm. When the fitting is incorrect, the operator can iteratively intervene with corrections after which the system will refit the object. We present an implementation of our fitting procedure for one degree-of-freedom (DOF) objects with revolute joints and evaluate it with a user study that shows that it can improve user performance, in measures of time on task and task load, ease of use, and usefulness compared to a manual registration approach. We also present a situated example that integrates our method into an end-to-end system for articulating a remote valve.

With the infusion of computation into workplaces and homes, various service settings, and everyday objects, scholars in human–computer interaction (HCI) and related domains have begun to consider the research and design implications not only of smart “things,” but ofsmart environments. Much of the work on smart environments to date has focused on smart homes; related work in HCI explores user values for smart homes, means of interacting with computation in smart homes (e.g., interfaces and agents), how to balance the needs of multiple stakeholders, and how to preserve user trust and autonomy. However, the smart environments of the future will not always fit the smart home mold of a coalescence of products that exist to automate and ease everyday tasks for the end users. They will be both user-focused and goal-focused, public and private, large and small, and ephemeral and long-lasting. It will benefit the field to look atsmart environmentsas a unit of analysis—including what these different types of environments have in common and what they do not—from a systemic, user experience design-oriented view. In this survey article, we review prior research on smart environments and various related bodies of literature. Informed by our literature review, we articulate fivelensesthat distinguish different types of smart environments from one another. We then propose research directions for future work on this topic.

This special issue of the Transactions on Human-Robot Interaction highlights, documents, and explores the metrics, test methods, and artifacts used in human-robot interaction (HRI) research. This collection of articles brings to attention the commonalities between the application of measurement science for the assessment and assurance of human-centric robotics in a variety of application domains, including industry, education, and defense. This special issue draws specific attention to the use and impact of metrology toward the advancement of HRI technologies and algorithms, and it promotes the application of measurement science toward the benchmarking and replication of HRI research. Special attention is given to the use cases, data sets, test methodologies, measurement techniques, metrics, and statistical analyses used to evaluate system performance.

How are people using current smart home technologies, and how do they conceptualize future ones that are more interconnected and more capable than those available today? We deployed an online survey study to 150 participants to investigate use of and opinions about smart speakers, home robots, virtual assistants, and other smart home devices. We also gauged how impressions of connected smart home devices are shaped by the way the devices interact with one another. Through a mixed-methods qualitative and quantitative approach, we found that people mostly use single devices for single functions, and have simple and brief interactions with virtual assistants. However, they imagine their future devices to have more control over the physical environment (i.e., interact with each other) and envision them interacting with people in more socially complex ways. These findings motivate design considerations and research directions for connected smart home technologies.

We explored different ways in which a multi-robot system might recover after one robot experiences a failure. We compared four recovery conditions: Update (a robot fixes its error and continues the task), Re-embody (a robot transfers its intelligence to a different body), Call (the failed robot summons a second robot to take its place), and Sense (a second robot detects the failure and proactively takes the place of the first robot). We found that trust in the system and perceived competence of the system were higher when a single robot recovered from a failure on its own (by updating or re-embodying) than when a second robot took over the task. We also found evidence that two robots that used the same socially interactive intelligence were perceived more similarly than two robots with different intelligences. Finally, our study revealed a relationship between how people perceive the agency of a robot and how they perceive the performance of the system.

Verified and validated test methods, being necessary to measure the performance of complex systems, are important tools for driving innovation, benchmarking and improving performance, and establishing trust in collaborative human-robot teams. This full-day workshop aims to explore the metrology necessary for repeatably and independently assessing the collaborative performance of robotic systems in real-world human-robot interaction (HRI) scenarios. This workshop aims to bridge the gaps between the theory and applications of HRI in industry, accelerating the adoption of cutting edge technologies as the industry state-of-practice. The interest in collaborative HRI is evident in the current market as well as standards efforts toward manufacturing, social, medical, and service robot solutions. Though these domains have been considered separate for many years, recent technological and scientific advancements show that, while their applications may differ, the underlying principles of HRI performance impact each identically. As such, this workshop seeks to identify test methods and metrics for the holistic assessment and assurance of collaborative HRI performance. The focus is on identifying the key performance indicators of these seemingly disparate sectors, and additionally to establish a community based on the principles of transparency, repeatability, & establishing trust in the assessment of collaborative HRI. The goal is to aid in the advancement of HRI technologies through the development of experimental scenarios, protocols, test methods, & metrics for the verification and validation of interaction solutions and interface designs.

Since robots are increasingly expected to work in concert with humans in dynamic, unstructured environments, they will need to express information about their state and actions. We propose a formalism for planning robot communication that employs a probabilistic representation of the robots world. This representation takes on the form of a Markov Decision Process (MDP) and captures the uncertainty of interacting with humans. The key insight of this work is that humans preferences and time need to be carefully balanced against the robots in order to minimize human annoyance. The communication-MDP enables the robot to reason about the effects of its actions on a human interactor. We validated the model through a human subjects experiment (n=44) by learning communication policies for a loosely collaborative task. The results show that the communication-MDP improves participants' perceptions of the robot's thoughtfulness as an interactor.

Craters are among the most abundant features on the surface of many planets with great importance for planetary scientists. They reveal chronology information about planets and may be used for autonomous spacecraft navigation and landing. Although numerous research efforts have been carried out in the field of crater detection, existing crater detection algorithms (CDAs) are only helpful in a limited number of applications. A promising crater detection approach involves two main steps: 1) hypothesis generation (HG) and 2) hypothesis verification (HV). During HG, potential crater locations are detected. The validity of the hypothesized crater locations is then tested in a HV step. In this context, we discuss some commonly used algorithms for HG such as highlight-shadow region detection and Hough transform as well as our novel and enhanced algorithms based on interest point detection and convex grouping. A key objective of this paper is to analyze their performance while paying special attention to how they affect the accuracy of the verification step. To deal with different size craters, we focus on multiscale HG. For HV, we have chosen convolutional neural networks which have recently achieved state-of-the-art performance in many computer vision applications. Due to the variation of test sets in the literature, it is often challenging to compare the performance of different CDAs in a fair way. In this paper, we present a comprehensive performance evaluation and comparison of CDAs. Each algorithm has been trained/tested using common data sets generated by a systematic approach.

Auditory cues facilitate situational awareness by enabling humans to infer what is happening in the nearby environment. Unlike humans, many robots do not continuously produce perceivable state-expressive sounds. In this work, we propose the use of iconic auditory signals that mimic the sounds produced by a robot»s operations. In contrast to artificial sounds (e.g., beeps and whistles), these signals are primarily functional, providing information about the robot»s actions and state. We analyze the effects of two variations of robot sound, tonal and broadband, on auditory localization during a human-robot collaboration task. Results from 24 participants show that both signals significantly improve auditory localization, but the broadband variation is preferred by participants. We then present a computational formulation for auditory signaling and apply it to the problem of auditory localization using a human-subjects data collection with 18 participants to learn optimal signaling policies.

This paper identifies and characterizes factors that contribute to operator workload in unmanned vehicle systems. Our objective is to provide a basis for developing models of workload for use in design and operation of complex human-machine systems. In 1986, Hart developed a foundational conceptual model of workload, which formed the basis for arguably the most widely used workload measurement technique-the NASA Task Load Index. Since that time, however, there have been many advances in models and factor identification as well as workload control measures. Additionally, there is a need to further inventory and describe factors that contribute to human workload in light of technological advances, including automation and autonomy. Thus, we propose a conceptual framework for the workload construct and present a taxonomy of factors that can contribute to operator workload. These factors, referred to as workload drivers, are associated with a variety of system elements including the environment, task, equipment, and operator. In addition, we discuss how workload moderators, such as automation and interface design, can be manipulated in order to influence operator workload. We contend that workload drivers, workload moderators, and the interactions among drivers and moderators all need to be accounted for when building complex human-machine systems.

This paper describes a structured light-based sensor for hazard avoidance in planetary environments. The system presented here can also be used in terrestrial applications constrained by reduced onboard power and computational complexity and low illumination conditions. The sensor consists on a calibrated camera and laser dot projector system. The onboard hazard avoidance system determines the position of the projected dots in the image and through a triangulation process detects potential hazards. The paper presents the design parameters for this sensor and describes the image based solution for hazard avoidance. The system presented here was tested extensively in day and night conditions in Lunar analogue environments. The current system achieves over 97% detection rate with 1.7% false alarms over 2000 images.

Automatic crater detection in planetary images is an important task with many applications in planetary science, spacecraft navigation, landing, and control. Typically, crater detection algorithms consist of two main steps: candidate crater region extraction and crater verification. Various methods have been proposed for extracting candidate crater regions, ranging from detecting circular/elliptical regions to detecting highlight and shadow regions. For crater verification, powerful feature extraction and machine learning techniques have been employed. While this two-step approach can be efficient and robust, inaccuracies in the candidate crater region extraction step can result in mislocalized crater regions which could affect verification performance. In this paper, we investigate the robustness of various feature extraction methods to mislocalized crater regions. Using features which are robust to localization errors but also choosing a more representative training set has yielded significant performance improvements on an extensive dataset from the Lunar Reconnaissance Orbiter (LRO).

Non-humanoid robots are becoming increasingly utilized for collaborative tasks across many domains, including industrial and service settings. Collaborative tasks between the human and robot rely on each collaborator's ability to effectively convey their mental state while accurately estimating and interpreting their partner's knowledge, intent, and actions. My research focuses on nonverbal communication signals that a non-humanoid robot can utilize during human-robot collaboration. We focus on motion, light and sound as they are commonly used communication channels across many domains and are available on most robot platforms. As a first step towards this goal, I present a completed study exploring how to use a simple multimodal light and sound signal to request help during a collaborative task. We then discuss future work to generate and utilize more complex signals to convey a variety of statuses to improve collaboration.

Planetary rovers navigate in extreme environments for which a Global Positioning System (GPS) is unavailable, maps are restricted to relatively low resolution provided by orbital imagery, and compass information is often lacking due to weak or not existent magnetic fields. However, an accurate rover localization is particularly important to achieve the mission success by reaching the science targets, avoiding negative obstacles visible only in orbital maps, and maintaining good communication connections with ground. This paper describes a horizon solution for precise rover orientation estimation. The detected horizon in imagery provided by the on board navigation cameras is matched with the horizon rendered over the existing terrain model. The set of rotation parameters (roll, pitch yaw) that minimize the cost function between the two horizon curves corresponds to the rover estimated pose.

Small flying robots represent a rapidly emerging family of robotic technologies with aerial capabilities that enable unique forms of assistance in a variety of collaborative tasks. Such tasks will necessitate interaction with humans in close proximity, requiring that designers consider human perceptions regarding robots flying and acting within human environments. We explore the design space regarding explicit robot communication of flight intentions to nearby viewers. We apply design constraints to robot flight behaviors, using biological and airplane flight as inspiration, and develop a set of signaling mechanisms for visually communicating directionality while operating under such constraints. We implement our designs on two commercial flyers, requiring little modification to the base platforms, and evaluate each signaling mechanism, as well as a no-signaling baseline, in a user study in which participants were asked to predict robot intent. We found that three of our designs significantly improved viewer response time and accuracy over the baseline and that the form of the signal offered tradeoffs in precision, generalizability, and perceived robot usability.

Horizon line is a promising visual cue which can be exploited for robot localization or visual geo-localization. Prominent approaches to horizon line detection rely on edge detection as a pre-processing step which is inherently a non-stable approach due to parameter choices and underlying assumptions. We present a novel horizon line detection approach which uses machine learning and Dynamic Programming (DP) to extract the horizon line from a classification map instead of an edge map. The key idea is assigning a classification score to each pixel, which can be interpreted as the likelihood of the pixel belonging to the horizon line, and representing the classification map as a multi-stage graph. Using DP, the horizon line can be extracted by finding the path that maximizes the sum of classification scores. In contrast to edge maps which are typically binary (edge vs no-edge) and contain gaps, classification maps are continuous and contain no gaps, yielding significantly better solutions. Using classification maps instead of edge maps allows for removing certain assumptions such as the horizon is close to the top of the image or that the horizon forms a straight line. The purpose of these assumptions is to bias the DP solution but they fail to produce good results when they are not valid. We demonstrate our approach on three different data sets and provide comparisons with a traditional approach based on edge maps. Although our training set is comprised of a very small number of images from the same location, our results illustrate that our method generalizes well to images acquired under different conditions and geographical locations.

Assistive free-flyers (AFFs) are an emerging robotic platform with unparalleled flight capabilities that appear uniquely suited to exploration, surveillance, inspection, and telepresence tasks. However, unconstrained aerial movements may make it difficult for colocated operators, collaborators, and observers to understand AFF intentions, potentially leading to difficulties understanding whether operator instructions are being executed properly or to safety concerns if future AFF motions are unknown or difficult to predict. To increase AFF usability when working in close proximity to users, we explore the design of natural and intuitive flight motions that may improve AFF abilities to communicate intent while simultaneously accomplishing task goals. We propose a formalism for representing AFF flight paths as a series of motion primitives and present two studies examining the effects of modifying the trajectories and velocities of these flight primitives based on natural motion principles. Our first study found that modified flight motions might allow AFFs to more effectively communicate intent and, in our second study, participants preferred interacting with an AFF that used a manipulated flight path, rated modified flight motions as more natural, and felt safer around an AFF with modified motion. Our proposed formalism and findings highlight the importance of robot motion in achieving effective human-robot interactions.

This paper introduces an advanced rover localization system suitable for autonomous planetary exploration in the absence of Global Positioning System (GPS) infrastructure. Given an existing terrain map (image and elevation) obtained from satellite imagery and the images provided by the rover stereo camera system, the proposed method determines the best rover location through visual odometry, 3D terrain and horizon matching. The system is tested on data retrieved from a 3 km traverse of the Basalt Hills quarry in California where the GPS track is used as ground truth. Experimental results show the system presented here reduces by over 60% the localization error obtained by wheel odometry.

Vehicle detection from aerial images is becoming an increasingly important research topic in surveillance, traffic monitoring and military applications. The system described in this paper focuses on vehicle detection in rural environments and its applications to oil and gas pipeline threat detection. Automatic vehicle detection by unmanned aerial vehicles (UAV) will replace current pipeline patrol services that rely on pilot visual inspection of the pipeline from low altitude high risk flights that are often restricted by weather conditions. Our research compares a set of feature extraction methods applied for this specific task and four classification techniques. The best system achieves an average 85% vehicle detection rate and 1800 false alarms per flight hour over a large variety of areas including vegetation, rural roads and buildings, lakes and rivers collected during several day time illuminations and seasonal changes over one year.

In order for humans and robots to work effectively together, they need to be able to converse about abilities, goals and achievements. Thus, we are developing an interaction infrastructure called the "Human-Robot Interaction Operating System" (HRI/OS). The HRI/OS provides a structured software framework for building human-robot teams, supports a variety of user interfaces, enables humans and robots to engage in task-oriented dialogue, and facilitates integration of robots through an extensible API.

This paper describes an effort to identify common metrics for task-oriented human-robot interaction (HRI). We begin by discussing the need for a toolkit of HRI metrics. We then describe the framework of our work and identify important biasing factors that must be taken into consideration. Finally, we present suggested common metrics for standardization and a case study. Preparation of a larger, more detailed toolkit is in progress.

The Peer-To-Peer Human-Robot Interaction (P2P-HRI) project is developing techniques to improve task coordination and collaboration between human and robot partners. Our work is motivated by the need to develop effective human-robot teams for space mission operations. A central element of our approach is creating dialogue and interaction tools that enable humans and robots to flexibly support one another. In order to understand how this approach can influence task performance, we recently conducted a series of tests simulating a lunar construction task with a human-robot team. In this paper, we describe the tests performed, discuss our initial results, and analyze the effect of intervention on task performance.

We propose an architecture for a real-time multimodal system, which provides non-contact, adaptive user interfacing for Computer-Assisted Surgery (CAS). The system, called M/ORIS (for Medical/Operating Room Interaction System) combines gesture interpretation as an explicit interaction modality with continuous, real-time monitoring of the surgical activity in order to automatically address the surgeon's needs. Such a system will help reduce a surgeon's workload and operation time. This paper focuses on the proposed activity monitoring aspect of M/ORIS. We analyze the issues of Human-Computer Interaction in an OR based on real-world case studies. We then describe how we intend to address these issues by combining a surgical procedure description with parameters gathered from vision-based surgeon tracking and other OR sensors (e.g. tool trackers). We called this approach Scenario-based Activity Monitoring (SAM). We finally present preliminary results, including a non-contact mouse interface for surgical navigation systems.

This paper describes the development of efficient computer vision techniques for human-computer interaction. Our approach combines range and color information to achieve efficient, robust tracking using consumer-level computer hardware and cameras. In this paper, we present the design of the Human Oriented Tracking (HOT) library, present our initial results, and describe our current efforts to improve HOT's performance through model-based tracking.

Our goal is to make vehicle teleoperation accessible to all users. To do this, we develop easy-to-use yet capable Web tools which enable efficient, robust teleoperation in unknown and unstructured environments. Web-based teleoperation, however, raises many research issues, as well as prohibiting the use of traditional approaches. Thus, it is essential to develop new methods which minimize bandwidth usage, which provide sensor fusion displays, and which optimize human-computer interaction. We believe that existing systems do not adequately address these issues and have severely limited capability and performance as a result. In this paper we present a system design for safe and reliable Web-based vehicle teleoperation, describe an active and dynamic user interface, and explain how our approach differs from existing systems.