In this paper, we address a challenging task, synchronous motion captioning, that aim to generate a language description synchronized with human motion sequences. This task pertains to numerous applications, such as aligned sign language transcription and unsupervised action segmentation and temporal grounding. Our method introduces mechanisms to control self- and cross-attention distributions of the Transformer, allowing interpretability and aligned text generation. We achieve this through masking strategies and structuring losses that push the model to maximize attention only on the most important frames contributing to the generation of a motion word. These constraints aim to prevent undesired mixing of information in attention maps and to provide a monotonic attention distribution across tokens. Thus, the cross attentions of tokens are used for progressive text generation in synchronization with human motion sequences. We demonstrate the superior performance of our approach through evaluation on the two available benchmark datasets, KIT-ML and HumanML3D. As visual evaluation is essential for this task, we provide a comprehensive set of animated visual illustrations of the output of synchronous text generation in the code repository.

Schizophrenia, autism, or social phobia are typically accompanied by social interaction deficits. The objective of the AlterEgo European project is the creation of an interactive cognitive architecture, implementable in various artificial agents, allowing a continuous interaction with patients suffering from social disorders by virtue of changes in behavioral (robot-based) as well as morphological (avatar-based) properties of that agent. Here we present the scientific foundations of the project and its four main experimental steps. The results of the first similarity experiments are reported, which show that healthy participants functionally adapt their social motor interaction when they interact with agents (real or artificial) morphologically and behaviorally similar to, or different from, them. These results obtained with healthy participants have consequences for the rehabilitation of socially deficient patients.

This paper models recent data in the field of postural coordination showing the existence of self-organized postural states, and transition between them, underlying suprapostural tracking movements. The proposed closed-loop controller captures the complex postural behaviors observed in humans and can be used to implement efficient and simple balance control principles in humanoids.