Expressive robot motion positively impacts human-robot interaction by improving user engagement, likability, or task performance. We present a novel approach to automatically generate and modulate complex, expressive full-body gesture sequences from a multimodal (text, audio, video) context description. Based on a unified mathematical implementation of the Principles of Animation using Dynamic Movement Primitives, we use multimodal foundation models to generate such sequences along with parametric motion variations that are highly context-aware. This method extends the state of the art in terms of flexibility and generality by being interpretable, composable, and working across different robot morphologies. Moreover, we integrate the system into a continuous control framework and leverage knowledge distillation to learn a much smaller model, significantly improving token efficiency and system latency. Results from a user study with a human-like platform indicate that participants judged our system’s motions to be better aligned with the interaction context than motions produced under a non-modulated or API-level condition. We also demonstrate how the system generates expressive motion for robots with different kinematics, showcasing its versatility. Paper webpage: https://gen-mod-expressive-motion-behavior.github.io/.

Our goal is to enable social robots to interact autonomously with humans in a realistic, engaging, and expressive manner. The 12 Principles of Animation are a well-established framework animators use to create movements that make characters appear convincing, dynamic, and emotionally expressive. This paper proposes a novel approach that leverages Dynamic Movement Primitives (DMPs) to implement key animation principles, providing a learnable, explainable, modulable, online adaptable and composable model for automatic expressive motion generation. DMPs, originally developed for general imitation learning in robotics and grounded in a spring-damper system design, offer mathematical properties that make them particularly suitable for this task. Specifically, they enable modulation of the intensities of individual principles and facilitate the decomposition of complex, expressive motion sequences into learnable and parametrizable primitives. We present the mathematical formulation of the parameterized animation principles and demonstrate the effectiveness of our framework through experiments and application on three robotic platforms with different kinematic configurations, in simulation, on actual robots and in a user study. Our results show that the approach allows for creating diverse and nuanced expressions using a single base model.

A robot’s physical design impacts user acceptance, engagement, and trust while also influencing social and functional expectations about the robot’s capabilities. Robot design, and industrial design in general, can also be a driver of differentiation in a competitive market. In this paper, we leverage generative AI to incrementally explore design spaces of robot appearances. With the goal of overcoming training data bias of text-to-image models that favor stereotypical morphologies and designs, we propose a set of generation methods that enable a designer to guide the exploration process through text-, style-, and structure-based specifications from user or client feedback. Further, using Low-Rank Adaptation for model fine-tuning, the method allows to define the aesthetic direction by an image collection that conveys a particular style or theme ("mood boards"). The experiments demonstrate that our extensions retain image quality in terms of statistical and structural features and allow for both diversity and specificity in the design process. In a case study, we apply this method in the user-centered design process and discuss its opportunities and limitations.