In this work, we present a method for personalizing human-robot interaction by using emotive facial expressions to generate affective robot movements. Movement is an important medium for robots to communicate affective states, but the expertise and time required to craft new robot movements promotes a reliance on fixed preprogrammed behaviors. Enabling robots to respond to multimodal user input with newly generated movements could stave off staleness of interaction and convey a deeper degree of affective understanding than current retrieval-based methods. We use autoencoder neural networks to compress robot movement data and facial expression images into a shared latent embedding space. Then, we use a reconstruction loss to generate movements from these embeddings and triplet loss to align the embeddings by emotion classes rather than data modality. To subjectively evaluate our method, we conducted a user survey and found that generated happy and sad movements could be matched to their source face images. However, angry movements were most often mismatched to sad images. This multimodal data-driven generative method can expand an interactive agent’s behavior library and could be adopted for other multimodal affective applications.

Movement and embodiment are communicative affordances central to social robotics, but designing embodied movements for robots often requires extensive knowledge of both robotics and movement theory. More accessible methods such as learning from demonstration often rely on physical access to the robot which is usually limited to research settings. Machine learning (ML) algorithms can complement hand-crafted or learned movements by generating new behaviors, but this requires large and diverse training datasets, which are hard to come by. In this work, we propose an embodied telepresence system for remotely crowdsourcing emotive robot movement samples that can serve as ML training data. Remote users control the robot through the internet using the motion sensors in their smartphones and view the movement either from a first-person or a third-person perspective. We evaluated the system in an online study where users created emotive movements for the robot and rated their experience. We then utilized the user-crafted movements as inputs to a neural network to generate new movements. We found that users strongly preferred the third-person perspective and that the ML-generated movements are largely comparable to the user-crafted movements. This work supports the usability of telepresence robots as a movement crowdsourcing platform.

The word “robot” frequently conjures unrealistic expectations of utilitarian perfection: tireless, efficient and flawless agents. However, real-world robots are far from perfect—they fail and make mistakes. Thus, roboticists should consider altering their current assumptions and cultivating new perspectives that account for a more complete range of robot roles, behaviors, and interactions. To encourage this, we explore the use of metaphors for generating novel ideas and reframing existing problems, eliciting new perspectives of human-robot interaction. Our work makes two contributions. We (1) surface current assumptions that accompany the term “robots,” and (2) present a collection of alternative perspectives of interaction with robots through metaphors. By identifying assumptions, we provide a comprehensible list of aspects to reconsider regarding robots’ physicality, roles, and behaviors. Through metaphors, we propose new ways of examining how we can use, relate to, and co-exist with the robots that will share our future.

We propose a method for modifying affective robot movements using neural networks. Social robots use gestures and other movements to express their internal states. However, a robot's interactive capabilities are hindered by the predominant use of a limited set of preprogrammed or hand-animated behaviors, which can be repetitive and predictable, making sustained human-robot interactions difficult to maintain. To address this, we developed a method for modifying existing emotive robot movements by using neural networks. We use hand-crafted movement samples and a classifying variational autoencoder trained on these samples. Our method then allows for adjustment of affective movement features by using simple arithmetic in the network's latent embedding space. We present the implementation and evaluation of this approach and show that editing in the latent space can modify the emotive quality of the movements while preserving recognizability and legibility in many cases. This supports neural networks as viable tools for creating and modifying expressive robot behaviors.

Social robots use gestures to express internal and affective states, but their interactive capabilities are hindered by relying on preprogrammed or hand-animated behaviors, which can be repetitive and predictable. We propose a method for automatically synthesizing affective robot movements given manually-generated examples. Our approach is based on techniques adapted from deep learning, specifically generative adversarial neural networks (GANs).

Blossom is an open-source social robotics platform responding to three challenges in human-robot interaction (HRI) research: (1) Designing, manufacturing, and programming social robots requires a high level of technical knowledge; (2) social robot designs are fixed in appearance and movement capabilities, making them hard to adapt to a specific application; and (3) the use of rigid mechanisms and hard outer shells limits the robots’ expressive capabilities. Addressing these challenges, Blossom aims at three design objectives: accessibility, flexibility, and expressiveness. The robot’s mechanism can be quickly assembled and partially extended by end-users. Blossom’s appearance is open-ended through handcrafted fabric exteriors created and customized by users. Smooth organic movements are achieved with tensile mechanisms, elastic components, and a soft exterior cover attached loosely to the body. Blossom’s smartphone-based gesture generation requires neither programming nor character animation experience, allowing lay users to create their own behaviors. All elements in the design were conceived with a low barrier-of-entry in mind. The result is an accessible and customizable social robot for researchers. This article details the implementation of Blossom’s design and demonstrates the platform’s potential through four field deployment case studies.