The GDPR was enacted to reign in the mighty corporations of the internet. Then, it was unleashed on all organizations, large and small alike. We report the results of a multi-site field study on Italian schools, and the challenges they face to implement the GDPR while running activities full of sensitive issues without an army of legal and compliance officers. The sample study consisted of one kindergarten, ten primary schools, two junior secondary schools, and two secondary schools. We did not find evidence of the privacy paradox (spotless on paper but careless on the field). In contrast, school staff mostly crumble when by-the-book procedures cannot be implemented with the resources that they actually have. We discuss what happen on the field, from critical privacy incidents with potential impact on pupils security and safety, to ‘formal’ privacy incidents for which life is too short to bother-and how a risk-based approach could address them.

Several attempts have been made to enhance mindfulness through Virtual and Mixed Reality. To date, they only offer users an alternative way of presenting guided imagery (e.g., mentally visualizing a beach vs. rendering a beach). We propose a preliminary study investigating whether allowing users to actively explore guided imagery through their actions (e.g., grasping virtual objects with hands) affects the mindfulness experience. To this aim, we present a preliminary study on a VR scenario for mindfulness practice that encourages the user’s interactive behavior in two conditions: interactive multimodal VR vs. audio-only. No significant difference was observed in self-reported mindfulness between the two conditions.

Industry 5.0 rethinks the role of human operators in production processes with the final goal to promote societal well-being. To achieve such a goal, novel computational approaches reshaping human-machine collaboration are needed. This paper presents a computational framework, stemmed from Cognitive Sciences, to enable human operators and machines to share a cognitive common ground in co-working hybrid processes.

Emergent states are temporal group phenomena that arise from collective affective, behavioral, and cognitive processes shared among the group's members during their interactions. Cohesion is one such state, mainly conceptualized by scholars as affective in nature, and frequently distinguished into the two dimensions social and task cohesion. Whereas social cohesion is related to the need of belonging to a group, task cohesion is related to the group's goals and tasks. In this article, we emphasize the importance of behavioral interaction dynamics to predict cohesion's dynamics. Drawing from Social Science insights, we investigate the interplay between social and task cohesion to predict their dynamics across group tasks from nonverbal behavioral features. Three computational architectures exploiting transfer learning are presented. Transfer learning capitalizes on information learnt by a model for a specific dimension to predict the dynamics of the other dimension. Results show that integrating the influence of social cohesion to predict dynamics of task cohesion outperforms state-of-the-art. To predict dynamics of social cohesion, a model integrating the reciprocal impact of social and task cohesion significantly improves performance with respect to the state-of-the-art and a model only integrating the impact of task cohesion on dynamics of social cohesion.

By leveraging Interactional Sociology theories, multimodal behavioral features and recurrent neural architectures, we incrementally build computational models for trust analysis in multiparty human-robot interactions (HRI). We show that the model’s performance improves when i) modeling group dynamics with different granularities (i.e. group member, dyadic, and group as a whole), and ii) modeling users-robot interactions as a question-answer sequence.

Trust is an important aspect of a human-robot interaction (HRI) as it mitigates the performance of many activities. Users’ trust may be impacted when robots make mistakes. To be able to properly time trust-reparation actions, robots should detect trust variations during the interaction. There are very few computational models of trust for such a task. The existing ones relied on either Psychological or Sociological theories that gave place to different definitions and analysis tools. We can distinguish two main approaches in the trust literature: the mentalist and the interactionist one. In this paper, we compare both approaches for trust detection, and explore how the adoption of two different assessment tools on an HRI dataset may lead to different results. We identify criteria that set them apart, and provide guidelines on the possibilities that each approach offers depending on the target computational model of trust.

In a joint action, a group of individuals coordinate their movements to reach a shared goal. When a change–i.e., an event that affects group functioning–occurs, the group adopts strategies to face it. This article investigates how a change involving a strategic core role in a group affects interpersonal coordination, and ultimately group effectiveness in performing a joint action. Following the entrainment theory, interpersonal coordination is addressed in terms of the rhythmic cycles of the individuals and of the group and their adjustment. Music is used as an ideal ecological scenario for investigation. More specifically, this article focuses on orchestra playing. By adopting a computational approach, research is devoted to measure how a change of conductor (i.e., the leader) influences entrainment between players and its variation over time as well as the relationship between entrainment and external ratings of the orchestra performance. Results show that, whereas the change of conductor had a limited significant effect on entrainment, a significant effect was found when entrainment is used as a predictor of the external ratings. Both the obtained results and the techniques developed for measuring entrainment may open novel research directions in the area of automated analysis of group behavior, and particularly of emotion in groups.

Transactive Memory System (TMS) is a mental representation of the distribution of knowledge between the members of a team. Can an Embodied Conversational Agent perform as well as a Human when intervening as a leader to support the development of the team’s TMS? And, if yes, are there differences in the way the team perceives their respective interventions? In this paper, a perceptive online study is conducted on how Human leader interventions affect the perception of a team’s TMS. The results are compared to the ones from a previous study evaluating an Embodied Conversational agent leader rather than a human one. Both the agent and the human adopt nonverbal behaviors characterizing 2 leadership styles: Transformational (TFL) and Transactional (TAL). TFL is expected to stimulate team members curiosity and creativity in problem-solving; instead, TAL emphasizes the role of the leader in supervising the team, providing it with feedback when needed. The results show that the intervention from both the agent and the human are perceived to potentially improve the perceived TMS of a team. Another interesting insight is that the TFL style works better when performed by the Human, where both the TAL and TFL style perform well when realized by the agent.

A natural human-robot interaction (HRI) relies on the robot’s capacity to understand the users’ behaviors through psychological and sociological concepts. Users expect the robot to act in a realistic manner to create a more human-like relationship. In this context, trust is an essential concept as it determines the effectiveness of the system and its acceptance by users. The understanding of trust dynamics in HRI is still low and systematic studies of multimodal trust-related behaviors in HRI are relatively rare given the rising popularity of the topic. To bridge this gap, in this paper we present a novel coding system TURIN (Trust in hUman Robot INteraction) to study trust in HRI. A preliminary assessment of the coding system was carried out on the Vernissage dataset. Results show a significant agreement between expert annotators.

Cohesion is an affective group phenomenon. It has received a lot of attention from scholars both in Social Sciences and in Affective Computing that showed that cohesion and emotion influence each other, highlighting the need to jointly analyze them. This study presents 2 deep neural network architectures grounded on multitask learning to jointly predict cohesion and emotion. Inspired by 2 major Social Sciences approaches on group emotion (i.e., Top-down and Bottom-up), these architectures exploit cohesion and emotion interdependencies intending to improve the prediction of the dynamics (i.e. changes over time) of the Social and Task dimensions of cohesion. Emotion, here, is addressed in terms of its valence. Both architectures are evaluated against the performances of a similar model that only predicts the dynamics of both the Social and Task dimensions of cohesion, without integrating valence. Statistical analysis shows that only the deep model implementing the Bottom-up approach significantly improved the predictions of the Task cohesion’s dynamics. This result confirms the theoretical and practical benefits of multitasking as it takes full advantage of the inherent relationships between group emotion and cohesion to improve Task cohesion’s predictions.

We are organizing again the workshop on Interdisciplinary Insights into Group and Team Dynamics which is a joint effort between researchers in the the ICMI and INGRoup (Interdisciplinary Network for Group Research) communities. This workshop aims to provide a common destination for researchers to exchange ideas and collaborate. We have found in previous years that instigating interdisciplinary collaborations can be hard. The aim of this workshop is to sustain a joint community to foster continued cross-disciplinary exchange and mutual understanding.

Emergent states are behavioral, cognitive and affective processes appearing among the members of a group when they interact together. In the last decade, the development of computational approaches received a growing interest in building Human-Centered systems. Such a development is particularly difficult because some of these states have several dimensions interplaying somehow and somewhere over time. In this paper, we focus on cohesion, its dimensions and their interplay. Several definitions of cohesion exist, it can be simply defined as the tendency of a group to stick together to pursue goals and/or affective needs. This plethora of definitions resulted in many different cohesion dimensions. Social and Task dimensions are the most investigated both in Social Sciences and Computer Science since they both play an important role in a wide range of contexts and groups. To the best of our knowledge, however, no previous work on the prediction of cohesion dynamics focused on how these 2 dimensions interplay. We leverage Social Sciences to address this issue. In particular, we take advantage of the importance of Social cohesion for creating flexible and constructive relationships to reinforce Task cohesion. We describe a Deep Neural Network architecture (DNN) for predicting the dynamics of Task cohesion by applying transfer learning from a pre-trained model dedicated to the prediction of Social cohesion dynamics. Our architecture is evaluated against several baselines. Results show that it significantly improves the predictions of the Task cohesion dynamics, confirming the benefits of integrating Social Sciences insights into models architectures.

The task of predicting dialog acts (DA) based on conversational dialog is a key component in the development of conversational agents. Accurately predicting DAs requires a precise modeling of both the conversation and the global tag dependencies. We leverage seq2seq approaches widely adopted in Neural Machine Translation (NMT) to improve the modelling of tag sequentiality. Seq2seq models are known to learn complex global dependencies while currently proposed approaches using linear conditional random fields (CRF) only model local tag dependencies. In this work, we introduce a seq2seq model tailored for DA classification using: a hierarchical encoder, a novel guided attention mechanism and beam search applied to both training and inference. Compared to the state of the art our model does not require handcrafted features and is trained end-to-end. Furthermore, the proposed approach achieves an unmatched accuracy score of 85% on SwDA, and state-of-the-art accuracy score of 91.6% on MRDA.

We present WoNoWa, a novel multi-modal dataset of small group interactions in collaborative tasks. The dataset is explicitly designed to elicit and to study over time a Transactive Memory System (TMS), a group's emergent state characterizing the group's meta-knowledge about "who knows what". A rich set of automatic features and manual annotations, extracted from the collected audio-visual data, is available on request for research purposes. Features include individual descriptors (e.g., position, Quantity of Motion, speech activity) and group descriptors (e.g., F-formations). Additionally, participants' self-assessments are available. Preliminary results from exploratory analyses show that the WoNoWa design allowed groups to develop a TMS that increased across the tasks. These results encourage the use of the WoNoWa dataset for a better understanding of the relationship between behavioural patterns and TMS, that in turn could help to improve group performance.

There has been gathering momentum over the last 10 years in the study of group behavior in multimodal multiparty interactions. While many works in the computer science community focus on the analysis of individual or dyadic interactions, we believe that the study of groups adds an additional layer of complexity with respect to how humans cooperate and what outcomes can be achieved in these settings. Moreover, the development of technologies that can help to interpret and enhance group behaviours dynamically is still an emerging field. Social theories that accompany the study of groups dynamics are in their infancy and there is a need for more interdisciplinary dialogue between computer scientists and social scientists on this topic. This workshop has been organised to facilitate those discussions and strengthen the bonds between these overlapping research communities

The dominant approaches to text representation in natural language rely on learning embeddings on massive corpora which have convenient properties such as compositionality and distance preservation. In this paper, we develop a novel method to learn a heavy-tailed embedding with desirable regularity properties regarding the distributional tails, which allows to analyze the points far away from the distribution bulk using the framework of multivariate extreme value theory. In particular, a classifier dedicated to the tails of the proposed embedding is obtained which exhibits a scale invariance property exploited in a novel text generation method for label preserving dataset augmentation. Experiments on synthetic and real text data show the relevance of the proposed framework and confirm that this method generates meaningful sentences with controllable attribute, e.g. positive or negative sentiments.

Interpersonal human-human interaction is a dynamical exchange and coordination of social signals, feelings and emotions usually performed through and across multiple modalities such as facial expressions, gestures, and language. Developing machines able to engage humans in rich and natural interpersonal interactions requires capturing such dynamics. This paper addresses primitive emotional contagion during dyadic interactions in which roles are prefixed. Primitive emotional contagion was defined as the tendency people have to automatically mimic and synchronize their multimodal behavior during interactions and, consequently, to emotionally converge. To capture emotional contagion, a cross-recurrence based methodology that explicitly integrates short and long-term temporal dynamics through the analysis of both facial expressions and sentiment was developed. This approach is employed to assess emotional contagion at unimodal, multimodal and cross-modal levels and is evaluated on the Solid SAL-SEMAINE corpus. Interestingly, the approach is able to show the importance of the adoption of cross-modal strategies for addressing emotional contagion.

This paper investigates how unitizing affects external observers' annotation of group cohesion. We compared unitizing techniques belonging to these categories: interval coding, continuous coding, and a technique inspired by a cognitive theory on event perception. We applied such techniques for sampling coding units from a set of recordings of social interactions rich in behaviors related to cohesion. Then, we compared the cohesion scores the observers assigned to each coding unit. Results show that the three techniques can lead to suitable ratings and that the technique inspired to cognitive theories leads to scores reflecting variability in cohesion better than the other ones.

Creative joint activity is a form of real-time dynamic problem solving in which people collaborate to reach a common creative goal (e.g., to solve a mathematical problem, to improvise a piece of music, to write a novel, to sketch a story, and so on). While there exist interfaces able to produce social, emotional, communicative signals while collaborating with single human users to go through the creative process, the design of embodied interfaces able to observe and simultaneously effectively support creative joint activity with multiple human users is still an emerging research field. We define Creative Embodied Interfaces (CEIs) such interfaces, having either anthropomorphic or non-anthropomorphic aspect, and being either physically or virtually present in the real world. We argue that CEIs will enable a novel interaction paradigm that could be exploited in several fields such as science, education, health-care, arts, entertainment, social inclusion, companionship. This paper is aimed at providing definition and a first framework of CEIs combining psychological theories of creativity and computational models of social signal analysis/synthesis in avatars.

Understanding emotional states is a challenging task which frequently leads to misinterpretation even in human observers. While the perception of emotions has been studied extensively in human psychology, little is known about what factors influence the human perception of emotions in robots and virtual characters. In this paper, we build on the Brunswik lens model to investigate the influence of (a) the agent's embodiment using a 2D virtual character, a 3D blended embodiment, a recording of the 3D platform and a recording of a human, as well as (b) the level of human-likeness on people's ability to interpret emotional facial expressions in an agent. In addition, we measure social traits of the human observers and analyze how they correlate to the success in recognizing emotional expressions. We find that interpersonal differences play a minor role in the perception of emotional states. However, both embodiment and human-likeness as well as related perceptual dimensions such as perceived social presence and uncanniness have an effect on the attribution of emotional states.

Mimicry plays an important role in social interaction. In human communication, it is used to establish rapport and bonding both with other humans, as well as robots and virtual characters. However, little is known about the underlying factors that elicit mimicry in humans when interacting with a robot. In this work, we study the influence of embodiment on participants' ability to mimic a social character. Participants were asked to intentionally mimic the laughing behavior of the Furhat mixed embodied robotic head and a 2D virtual version of the same character. To explore the effect of embodiment, we present two novel approaches to automatically assess people's ability to mimic based solely on videos of their facial expressions. In contrast to participants' self-assessment, the analysis of video recordings suggests a better ability to mimic when people interact with the 2D embodiment.

Laughter is a social signal capable of facilitating interaction in groups of people: it communicates interest, helps to improve creativity, and facilitates sociability. This article focuses on: endowing virtual characters with computational models of laughter synthesis, based on an expressivity-copying paradigm; evaluating how the physically co-presence of the laughing character impacts on the user’s perception of an audio stimulus and mood. We adopt music as a means to stimulate laughter. Results show that the character presence influences the user’s perception of music and mood. Expressivity-copying has an influence on the user’s perception of music, but does not have any significant impact on mood.

The “social learning and multimodal interaction for designing artificial agents” workshop aims at presenting scientific and philosophical advances related to social learning and multimodal interaction for enhancing the design of artificial agents. Papers presented in the workshop include studies on human behavior modeling, on social robotics and on virtual agents. Our two invited speakers, Prof. Catherine Pelachaud and Prof. Louis-Philippe Morency will enrich and open the door to further discussion by bringing their widely acknowledged expertise in the field.

This paper introduces two novel algorithms for detecting groups of people standing or freely moving in a crowded environment. The proposed algorithms exploit low-level features extracted from videos. The first algorithm, the Link Method, uses a learning and forgetting strategy for modeling dynamics of proxemics between individuals. Two versions of this algorithm are proposed: they differ in the analysis of proxemics. The second one, called Interpersonal Synchrony Method, explicitly adopts interpersonal synchrony to refine clusters of persons detected by combining together proxemics and 2D field of view of individuals. The algorithms are evaluated on both simulated and real-world video sequences from state-of-the-art databases. Clustering metrics such as the Adjusted Mutual Information shows that our models outperform the approach based on F-formations. This work developed algorithms that can be readily applied in robotics, to allow robots to automatically detect groups in crowded environments.

Chronic (persistent) pain (CP) affects 1 in 10 adults; clinical resources are insufficient, and anxiety about activity restricts lives. Technological aids monitor activity but lack necessary psychological support. This article proposes a new sonification framework, Go-with-the-Flow, informed by physiotherapists and people with CP. The framework proposes articulation of user-defined sonified exercise spaces (SESs) tailored to psychological needs and physical capabilities that enhance body and movement awareness to rebuild confidence in physical activity. A smartphone-based wearable device and a Kinect-based device were designed based on the framework to track movement and breathing and sonify them during physical activity. In control studies conducted to evaluate the sonification strategies, people with CP reported increased performance, motivation, awareness of movement, and relaxation with sound feedback. Home studies, a focus group, and a survey of CP patients conducted at the end of a hospital pain management session provided an in-depth understanding of how different aspects of the SESs and their calibration can facilitate self-directed rehabilitation and how the wearable version of the device can facilitate transfer of gains from exercise to feared or demanding activities in real life. We conclude by discussing the implications of our findings on the design of technology for physical rehabilitation.

In this paper, we investigate the detection of laughter from the user's nonverbal full-body movement in social and ecological contexts. Eight hundred and one laughter and nonlaughter segments of full-body movement were examined from a corpus of motion capture data of subjects participating in social activities that stimulated laughter. A set of 13 full-body movement features was identified, and corresponding automated extraction algorithms were developed. These features were extracted from the laughter and nonlaughter segments, and the resulting dataset was provided as input to supervised machine learning techniques. Both discriminative (radial basis function-support vector machines, k-nearest neighbor, and random forest) and probabilistic (naive Bayes and logistic regression) classifiers were trained and evaluated. A comparison of automated classification with the ratings of human observers for the same laughter and nonlaughter segments showed that the performance of our approach for automated laughter detection is comparable with that of humans. The highest F-score (0.74) was obtained by the random forest classifier, whereas the F-score obtained by human observers was 0.70. Based on the analysis techniques introduced in the paper, a vision-based system prototype for automated laughter detection was designed and evaluated. Support vector machines (SVMs) and Kohonen's self-organizing maps were used for training, and the highest F-score was obtained with SVM (0.73).

In our demo, LoL, a user interacts with a virtual agentable to copy and to adapt its laughing and expressive behaviorson-the-fly. Our aim is to study copying capabilitiesparticipate in enhancing user’s experience in the interaction.User listens to funny audio stimuli in the presenceof a laughing agent: when funniness of audio increases, theagent laughs and the quality of its body movement (directionand amplitude of laughter movements) is modulated on-theflyby user’s body features.

Mimicry and laughter are two social signals displaying affiliation among people. To date, however, their relationship remains uninvestigated and relatively unexploited in designing the behaviour of robots and virtual characters. This paper presents an experiment aimed at examining how laughter and mimicry are related. The hypothesis is that hand movements a person produces during a laughter episode are mimicked through equivalent or other hand movements other participants in the interaction produce when they laugh. To investigate this, we analysed mimicry at two levels of specificity during laughter and non-laughter periods in a playful triadic social interaction. Changes in mimicry rates over the whole interaction were analysed as well as possible leader-follower relationships. Results show that hand movement rates were varied and strongly dependent on group. Even though hand movement are more frequent during laughter, mimicry does not increase. Mimicry levels, however, increase over the course of a session indicating that familiarity and comfort may increase emotional contagion.

Within the EU ILHAIRE Project, researchers of several disciplines (e.g., computer sciences, psychology) collaborate to investigate the psychological foundations of laughter, and to bring this knowledge into shape for the use in new technologies (i.e., affective computing). Within this framework, in order to endow machines with laughter capabilities (encoding as well as decoding), one crucial task is an adequate description of laughter in terms of morphology. In this paper we present a work methodology towards automated full body laughter detection: starting from expert annotations of laughter videos we aim to identify the body features that characterize laughter.

The embodied and reflexive interaction paradigms separately proved to be effective for learning music in childhood. However, nowadays, there is a scarcity of research addressing the joined adoption of these paradigms, both from a theoretical and a technological point of view. BeSound supports children to explore - by means of their own body - rhythm, melody, and harmony and to creatively combine them together. Firstly, the child is engaged in a game in which she has to mimic the movement of the characters of stories BeSound tells her; then she can ad-lib a music dialogue with the characters. Each character was previously associated with a component of Laban's Effort and it was described through a set of whole-body movement features. These features are automatically detected, analysed, and used to control the music response of BeSound.

Embodiment and reflexive interaction proved to be effective approaches to music education in childhood. A research challenge consists of merging them. This paper presents BeSound, an application intended to support children in learning the basic elements of composition. Children explore rhythm, melody, and harmony by playing at mimicking objects or characters; the qualities of their whole-body movements are analysed in real-time according to Rudolf Laban's Theory of Effort and used to control sound. The paper focuses on the design of BeSound and describes the analysis performed to distinguish between direct and flexible movements - the Laban's Space component - and between light and heavy movements - the Laban's Weight component.

Embodied cooperation "arises when two co-present, individuals in motion coordinate their goal-directed actions". The adoption of the embodied cooperation paradigm for the development of embodied and social multimedia systems opens new perspectives for future User Centric Media. Systems for embodied music listening, which enable users to influence music in real-time by movement and gesture, can greatly benefit from the embodied cooperation paradigm. This paper presents the design and the evaluation of an application, Sync4All, based on such a paradigm, allowing users to experience social embodied music listening. Each user rhythmically and freely moves a mobile phone trying to synchronise her movements with those of the other ones. The level of such a synchronisation influences the music experience. The evaluation of Sync4All was aimed at finding out which is the overall attitude of the users towards the application, and how the participants perceived embodied cooperation and music embodiment.

Since its first edition in 2009, the International Workshop on Social Behaviour in Music (SBM) has been an occasion for researchers and practitioners for discussing recent advances in automated analysis of social behaviour, being music the selected test-bed and application scenario. The first edition of SBM was held in Vancouver, Canada, in the framework of the 2009 IEEE International Conference on Social Computing (SocialCom 2009). The second one was held in Genova, Italy, in the framework of the 4th International ICST Conference on Intelligent Technologies for Interactive Entertainment (Intetain 2011). SBM is now at its third edition, which takes place in the framework of the 14th International Conference on Multimodal Interaction (ICMI 2012), Santa Monica, California, USA. Again, SBM aims at providing a picture of current research breakthrough and issues, giving at the same time directions for future works and collaborations.

This paper presents a multimodal system for real-time analysis of nonverbal affective social interaction in small groups of users. The focus is on two major aspects of affective social interaction: the synchronization of the affective behavior within a small group and the emergence of functional roles, such as leadership. A small group of users is modeled as a complex system consisting of single interacting components that can auto-organize and show global properties. Techniques are developed for computing quantitative measures of both synchronization and leadership. Music is selected as experimental test-bed since it is a clear example of interactive and social activity, where affective nonverbal communication plays a fundamental role. The system has been implemented as software modules for the EyesWeb XMI platform (http://www.eyesweb.org). It has been used in experimental frameworks (a violin duo and a string quartet) and in real-world applications (in user-centric applications for active music listening). Further application scenarios include entertainment, edutainment, therapy and rehabilitation, cultural heritage, and museum applications. Research has been carried out in the framework of the EU-ICT FP7 Project SAME (http://www.sameproject.eu).

This work aims at defining a computational model of human emotional entrainment. Music, as a non-verbal language to express emotions, is chosen as an ideal test bed for these aims. We start from multimodal gesture and motion signals, recorded in a real world collaborative condition in an ecological setting. Four violin players were asked to play, alone or in duo, a music fragment in two different perceptual feedback modalities and in four different emotional states. We focused our attention on phase synchronisation of the head motions of the players. From observation by subjects (musicians and observers), an evidence of entrainment emerges between players. The preliminary results, based on a reduced data set, however do not grasp fully this phenomenon. A more extended analysis is current subject of investigation.