Synchrony refers to individuals' temporal coordination during social interactions. The analysis of this phenomenon is complex, requiring the perception and integration of multimodal communicative signals. The evaluation of synchrony has received multidisciplinary attention because of its role in early development, language learning, and social connection. Originally studied by developmental psychologists, synchrony has now captured the interest of researchers in such fields as social signal processing, robotics, and machine learning. This paper emphasizes the current questions asked by synchrony evaluation and the state-of-the-art related methods. First, we present definitions and functions of synchrony in youth and adulthood. Next, we review the noncomputational and computational approaches of annotating, evaluating, and modeling interactional synchrony. Finally, the current limitations and future research directions in the fields of developmental robotics, social robotics, and clinical studies are discussed.

Emotional speech classification is a key problem in social interaction analysis. Traditional emotional speech classification methods are completely supervised and require large amounts of labeled data. In addition, various feature sets are usually used to characterize the emotional speech signals. Therefore, we propose a new co-training algorithm based on multi-view features. More specifically, we adopt different features for the characterization of speech signals to form different views for classification, so as to extract as much discriminative information as possible. We then use the co-training algorithm to classify emotional speech with only few annotations. In this article, a dynamic weighted co-training algorithm is developed to combine different features (views) to predict the common class variable. Experiments prove the validity and effectiveness of this method compared to self-training algorithm.

This work presents the development of a real-time framework for the research of multimodal feedback of robots/talking agents in the context of Human Robot Interaction (HRI) and Human Computer Interaction (HCI). For evaluating the framework, a Multimodal corpus is built (ENTERFACE_STEAD), and a study on the important multimodal features was done for building an active Robot/Agent listener of a storytelling experience with Humans. The experiments show that even when building the same reactive behavior models for Robot and Talking Agents, the interpretation and the realization of the behavior communicated is different due to the different communicative channels Robots/Agents offer be it physical but less human-like in Robots, and virtual but more expressive and human-like in Talking agents.

In this paper, we present an automatic motherese detection system for the study of parent-infant interaction analysis. Motherese is a speech register directed towards infants and it is characterized by higher pitch, slower tempo, and exaggerated intonation. The goal of this paper is to propose and evaluate different approaches for the detection of motherese from home movies. We investigated the characterization by supra-segmental features (prosody) but also by segmental ones namely the MFCC (Mel Frequency Cepstral Coefficients). Concerning the classification stage, we investigated two different methods: the k-nn (k-nearest neighbors) and the GMM (Gaussian Mixture Models). Experimental results show that segmental features play a major role on the detection.