Natural gesturing is very important for the credibility of social robots. It is even more crucial for storytelling robots since the expression, emotion and emphasis must be highlighted. In this paper we propose a hybrid gesture generation approach for a storytelling robot that combines beats automatically generated by a GAN with a probabilistic semantic related gesture insertion system. Beats are executed according to a probability based on the duration of the sentences and semantic gesture insertions are dependent of the previous occurrences of the gestures associated to the words. The polarity of the text is extracted and affects several features of the motion to arouse emotion. A qualitative evaluation of robot behavior is conducted and confirms the approach as a promising one as storytelling system.

Abstract Natural gestures are a desirable feature for a humanoid robot, as they are presumed to elicit a more comfortable interaction in people. With this aim in mind, we present in this paper a system to develop a natural talking gesture generation behavior. A Generative Adversarial Network (GAN) produces novel beat gestures from the data captured from recordings of human talking. The data is obtained without the need for any kind of wearable, as a motion capture system properly estimates the position of the limbs/joints involved in human expressive talking behavior. After testing in a Pepper robot, it is shown that the system is able to generate natural gestures during large talking periods without becoming repetitive. This approach is computationally more demanding than previous work, therefore a comparison is made in order to evaluate the improvements. This comparison is made by calculating some common measures about the end effectors’ trajectories (jerk and path lengths) and complemented by the Fréchet Gesture Distance (FGD) that aims to measure the fidelity of the generated gestures with respect to the provided ones. Results show that the described system is able to learn natural gestures just by observation and improves the one developed with a simpler motion capture system. The quantitative results are sustained by questionnaire based human evaluation .