Cursor-based tele-operation interfaces for manipulators can enable widely available and accessible control of robots to make many near term applications possible. However, their efficiency is restricted by the challenge of controlling 6 Degrees-of-Freedom (DoF) with 2D input from the cursor. Existing interfaces make use of different strategies to tackle this challenge, including viewpoint constraints, mode switching, and visual overlays, but it is unclear how these strategies impact the efficiency and accessibility of the interface. In this paper we characterize the design space of cursor-based robot control interfaces and compare alternatives in two user studies. Study 1 (N=216) compares nine alternative interfaces focusing on control of 3 DoFs to understand the differences of the interfaces at the basic level and examine the impact of task parameters on efficiency. Study 2 (N=60) compares a subset of the interfaces integrated into a system that allows full control of a robot manipulator from three orthogonal views. We also present a framework for heuristically evaluating accessibility of these interfaces and discuss the efficiency and accessibility trade-off with recommendations.

New mobile manipulator platforms, like the Hello Robot Stretch, have made the idea of long-term in-home robotic assistance feasible. However, existing autonomous capabilities for such robots in unstructured, highly-varied environments are still not available. Instead, using robots with human tele-operation can have huge immediate impact. For these robots to serve populations that need them the most, their interfaces need to be accessible to people with mobility limitations. In this paper we explore the utility, usability, and accessibility of a tele-operated Stretch robot in the home. We first describe a browser-based interface for controlling the Stretch robot designed with accessibility in mind. We then present findings from a study (N=18) in which participants used the interface to remotely control the robot to perform realistic tasks in a kitchen, demonstrating the feasibility of tele-operated assistance and revealing challenges and opportunities. Next, we present a study with individuals with mobility limitations (N=3) identifying additional accessibility requirements for the interface. Participants in both studies agree to the utility of the robot despite current limitations.

A robot-assisted feeding system can potentially help a user with upper-body mobility impairments eat independently. However, autonomous assistance in the real world is challenging because of varying user preferences, impairment constraints, and possibility of errors in uncertain and unstructured environments. An autonomous robot-assisted feeding system needs to decide the appropriate strategy to acquire a bite of hard-to-model deformable food items, the right time to bring the bite close to the mouth, and the appropriate strategy to transfer the bite easily. Our key insight is that a system should be designed based on a user's preference about these various challenging aspects of the task. In this work, we explore user preferences for different modes of autonomy given perceived error risks and also analyze the effect of input modalities on technology acceptance. We found that more autonomy is not always better, as participants did not have a preference to use a robot with partial autonomy over a robot with low autonomy. In addition, participants' user interface preference changes from voice control during individual dining to web-based during social dining. Finally, we found differences on average ratings when grouping the participants based on their mobility limitations (lower vs. higher) that suggests that ratings from participants with lower mobility limitations are correlated with higher expectations of robot performance.

Augmented Reality (AR) benefits telementoring by enhancing the communication between the mentee and the remote mentor with mentor authored graphical annotations that are directly integrated into the mentee’s view of the workspace. An important problem is conveying the workspace to the mentor effectively, such that they can provide adequate guidance. AR headsets now incorporate a frontfacing video camera, which can be used to acquire the workspace. However, simply providing to the mentor this video acquired from the mentee’s first-person view is inadequate. As the mentee moves their head, the mentor’s visualization of the workspace changes frequently, unexpectedly, and substantially. This paper presents a method for robust high-level stabilization of a mentee first-person video to provide effective workspace visualization to a remote mentor. The visualization is stable, complete, up to date, continuous, distortion free, and rendered from the mentee’s typical viewpoint, as needed to best inform the mentor of the current state of the workspace. In one study, the stabilized visualization had significant advantages over unstabilized visualization, in the context of three number matching tasks. In a second study, stabilization showed good results, in the context of surgical telementoring, specifically for cricothyroidotomy training in austere settings.

Robot-assisted feeding (RAF) systems offer enormous potential benefits to community-centered care-giving environments. However, developers of RAF technologies often focus on evaluating their standard transactional functionality, omitting the impact of such technologies in contexts that extend past the interaction of the robot and food receiver. RAF technologies have complex social, cultural and self-identity implications, since a "meal" extends well beyond the simple provisioning of nourishment. To better understand these implications we conducted a contextual inquiry in an assisted-living community with five potential care recipients and five caregivers, as well as interviews with fifteen domain experts including occupational therapists and feeding specialists. Based on our findings from these studies, we developed a new framework for RAF technologies that formulates this vital task as a community-centered relational service. We then use this framework to qualitatively and quantitatively assess three existing feeding systems and identify areas of improvement. Our work reveals new insights about stakeholders of RAF technologies and provides a roadmap for technology developers to better serve the needs of these stakeholders.

Understanding people's attitudes towards robots and how those attitudes are affected by exposure to robots is essential to the effective design and development of social robots. Although researchers have been studying attitudes towards robots among adults and even children for more than a decade, little has been explored assessing attitudes among teens-a highly vulnerable population that presents unique opportunities and challenges for social robots. Our work aims to close this gap. In this paper we present findings from several participatory robot interaction and design sessions with 136 teenagers who completed a modified version of the Negative Attitudes Towards Robots Scale (NARS) before participation in a robot interaction. Our data reveal that most teens are 1) highly optimistic about the helpfulness of robots, 2) do not feel nervous talking with a robot, but also 3) do not trust a robot with their data. Ninety teens also completed a post-interaction survey and reported a significant change in the motional attitudes subscale of the NARS. We discuss the implications of our findings on the design of social robots for teens.

Maintaining independence and dignity is a primary goal of successful aging for older adults around the globe. Robots can support this goal in various ways by assisting everyday tasks that become challenging due to aging-related deterioration in physical and mental abilities. While a growing body of research tackles challenges in creating such robots, most work has focused on older adults with high socio-economic status in the developed world. In most cases, the price of these robots alone prohibits their potential use in the developing world. Further, socio-cultural differences in the developing world will limit the usability and chance of adoption of a robot designed based on users in the developed world. Our work aims to close this gap. In this paper we present findings from the user-centered design and development process of a low-cost assistive robot for older adults in the developing world named SHEBA, which is a Bengali term for care. We first interviewed 37 older adults and 21 caregivers in assisted and independent living settings in Dhaka, Bangladesh to gather requirements and understand priorities. We then developed a prototype focused on medication management and delivery and we brought it to an assisted living center to interact with potential older adult users. We interviewed 23 older adults and 5 caregivers who interacted with or observed our prototype to gather feedback. We present quantitative and qualitative data obtained in these interviews, identifying key requirements for robots designed for older adults in the developing world.

This poster presents the design, implementation, and evaluation of a method for robust high-level stabilization of mentees first-person video in augmented reality (AR) telementoring. This video is captured by the front-facing built-in camera of an AR headset and stabilized by rendering from a stationary view a planar proxy of the workspace projectively texture mapped with the video feed. The result is stable, complete, up to date, continuous, distortion free, and rendered from the mentee's default viewpoint. The stabilization method was evaluated in two user studies, in the context of number matching and for cricothyroidotomy training, respectively. Both showed a significant advantage of our method compared with unstabilized visualization.

Most common approaches to one-shot gesture recognition have leveraged mainly conventional machine learning solutions and image based data augmentation techniques, ignoring the mechanisms that are used by humans to perceive and execute gestures, a key contextual component in this process. The novelty of this work consists on modeling the process that leads to the creation of gestures, rather than observing the gesture alone. In this approach, the context considered involves the way in which humans produce the gestures - the kinematic and biomechanical characteristics associated with gesture production and execution. By understanding the main "modes" of variation we can replicate the single observation many times. Consequently, the main strategy proposed in this paper includes generating a data set of human-like examples based on "naturalistic" features extracted from a single gesture sample while preserving fundamentally human characteristics like visual saliency, smooth transitions and economy of motion. The availability of a large data set of realistic samples allows the use state-of-the-art classifiers for further recognition. Several classifiers were trained and their recognition accuracies were assessed and compared to previous one-shot learning approaches. An average recognition accuracy of 95% among all classifiers highlights the relevance of keeping the human "in the loop" to effectively achieve one-shot gesture recognition.

Previous work in the area of gesture production, has made the assumption that machines can replicate humanlike gestures by connecting a bounded set of salient points in the motion trajectory. Those inflection points were hypothesized to also display cognitive saliency. The purpose of this paper is to validate that claim using electroencephalography (EEG). That is, this paper attempts to find neural signatures of gestures (also referred as placeholders) in human cognition, which facilitate the understanding, learning and repetition of gestures. Further, it is discussed whether there is a direct mapping between the placeholders and kinematic salient points in the gesture trajectories. These are expressed as relationships between inflection points in the gestures trajectories with oscillatory mu rhythms (8-12 Hz) in the EEG. This is achieved by correlating fluctuations in mu power during gesture observation with salient motion points found for each gesture. Peaks in the EEG signal at central electrodes (motor cortex; C3/Cz/C4) and occipital electrodes (visual cortex; O3/Oz/O4) were used to isolate the salient events within each gesture. We found that a linear model predicting mu peaks from motion inflections fits the data well. Increases in EEG power were detected 380 and 500ms after inflection points at occipital and central electrodes, respectively. These results suggest that coordinated activity in visual and motor cortices is sensitive to motion trajectories during gesture observation, and it is consistent with the proposal that inflection points operate as placeholders in gesture recognition.

User's intentions may be expressed through spontaneous gesturing, which have been seen only a few times or never before. Recognizing such gestures involves one shot gesture learning. While most research has focused on the recognition of the gestures themselves, recently new approaches were proposed to deal with gesture perception and production as part of the recognition problem. The framework presented in this work focuses on learning the process that leads to gesture generation, rather than treating the gestures as the outcomes of a stochastic process only. This is achieved by leveraging kinematic and cognitive aspects of human interaction. These factors enable the artificial production of realistic gesture samples originated from a single observation, which in turn are used as training sets for state-of-the-art classifiers. Classification performance is evaluated in terms of recognition accuracy and coherency; the latter being a novel metric that determines the level of agreement between humans and machines. Specifically, the referred machines are robots which perform artificially generated examples. Coherency in recognition was determined at 93.8%, corresponding to a recognition accuracy of 89.2% for the classifiers and 92.5% for human participants. A proof of concept was performed towards the expansion of the proposed one shot learning approach to adaptive learning, and the results are presented and the implications discussed.

This paper discusses the problem of one shot gesture recognition. This is relevant to the field of human-robot interaction, where the user's intentions are indicated through spontaneous gesturing (one shot) to the robot. The novelty of this work consists of learning the process that leads to the creation of a gesture, rather on the gesture itself. In our case, the context involves the way in which humans produce the gestures - the kinematic and anthropometric characteristics and the users' proxemics (the use of the space around them). In the method presented, the strategy is to generate a dataset of realistic samples based on biomechanical features extracted from a single gesture sample. These features, called “the gist of a gesture”, are considered to represent what humans remember when seeing a gesture and the cognitive process involved when trying to replicate it. By adding meaningful variability to these features, a large training data set is created while preserving the fundamental structure of the original gesture. Having a large dataset of realistic samples enables training classifiers for future recognition. Three classifiers were trained and tested using a subset of ChaLearn dataset, resulting in all three classifiers showing rather similar performance around 80% recognition rate Our classification results show the feasibility and adaptability of the presented technique regardless of the classifier.

This research presents an exploratory study among touch-based and touchless interfaces selected to teleoperate a highly dexterous surgical robot. The possibility of incorporating touchless interfaces into the surgical arena may provide surgeons with the ability to engage in telerobotic surgery similarly as if they were operating with their bare hands. On the other hand, precision and sensibility may be lost. To explore the advantages and drawbacks of these modalities, five interfaces were selected to send navigational commands to the Taurus robot in the system: Omega, Hydra, and a keyboard. The first represented touch-based, while Leap Motion and Kinect were selected as touchless interfaces. Three experimental designs were selected to test the system, based on standardized surgically related tasks and clinically relevant performance metrics measured to evaluate the user's performance, learning rates, control stability, and interaction naturalness. The current work provides a benchmark and validation framework for the comparison of these two groups of interfaces and discusses their potential for current and future adoption in the surgical setting.

This paper presents a comprehensive evaluation among touchless, vision-based hand tracking interfaces (Kinect and Leap Motion) and the feasibility of their adoption into the surgical theater compared to traditional interfaces.