The wide adoption of mobile devices and social media platforms have dramatically increased the collection and sharing of personal information. More and more frequently, users are called to make decisions concerning the disclosure of their personal information. In this study, we investigate the factors affecting users’ choices toward the disclosure of their personal data, including not only their demographic and self-reported individual characteristics, but also their social interactions and their mobility patterns inferred from months of mobile phone data activity. We report the findings of a field study conducted with a community of 63 subjects provided with (i) a smart-phone and (ii) a Personal Data Store (PDS) enabling them to control the disclosure of their data. We monitor the sharing behavior of our participants through the PDS and evaluate the contribution of different factors affecting their disclosing choices of location and social interaction data. Our analysis shows that social interaction inferred by mobile phones is an important factor revealing willingness to share, regardless of the data type. In addition, we provide further insights on the individual traits relevant to the prediction of sharing behavior.

A key acceptability criterion for artificial agents will be the possible moral implications of their actions. In particular, intelligent persuasive systems (systems designed to influence humans via communication) constitute a highly sensitive topic because of their intrinsically social nature. Still, ethical studies in this area are rare and tend to focus on the output of the required action; instead, this work focuses on the acceptability of persuasive acts themselves.Building systems able to persuade while being ethically acceptable requires that they be capable of intervening flexibly and of taking decisions about which specific persuasive strategy to use. We show how, exploiting a behavioral approach, based on human assessment of moral dilemmas, we obtain results that will lead to more ethically appropriate systems. Experiments we have conducted address the type of persuader, the strategies adopted and the circumstances. Dimensions surfaced that can characterize the interpersonal differences concerning moral acceptability of machine performed persuasion, usable for strategy adaptation. We also show that the prevailing preconceived negative attitude toward persuasion by a machine is not predictive of actual moral acceptability judgement when subjects are confronted with specific cases.

Stress can impact many aspects of our lives, such as the way we interact and work with others, or the first impressions that we make. In the past, stress has been most commonly assessed through self-reported questionnaires; however, advancements in wearable technology have enabled the measurement of physiological symptoms of stress in an unobtrusive manner. Using a dataset of job interviews, we investigate whether first impressions of stress (from annotations) are equivalent to physiological measurements of the electrodermal activity (EDA). We examine the use of automatically extracted nonverbal cues stemming from both the visual and audio modalities, as well EDA stress measurements for the inference of stress impressions obtained from manual annotations. Stress impressions were found to be significantly negatively correlated with hireability ratings i.e individuals who were perceived to be more stressed were more likely to obtained lower hireability scores. The analysis revealed a significant relationship between audio and visual features but low predictability and no significant effects were found for the EDA features. While some nonverbal cues were more clearly related to stress, the physiological cues were less reliable and warrant further investigation into the use of wearable sensors for stress detection.

Incorporating research on personality recognition into computers, both from a cognitive as well as an engineering perspective, would facilitate the interactions between humans and machines. Previous attempts on personality recognition have focused on a variety of different corpora (ranging from text to audiovisual data), scenarios (interviews, meetings), channels of communication (audio, video, text), and different subsets of personality traits (out of the five ones from the Big Five Model). Our study uses simple acoustic and visual nonverbal features extracted from multimodal data, which have been recorded in previously uninvestigated scenarios, and consider all five personality traits and not just a subset. First, we look at the human-machine interaction scenario, where we introduce the display of different “collaboration levels.” Second, we look at the contribution of the human-human interaction (HHI) scenario on the emergence of personality traits. Investigating the HHI scenario creates a stronger basis for future human-agents interactions. Our goal is to study, from a computational approach, the emergence degree of the five personality traits in these two scenarios. The results demonstrate the relevance of each of the two scenarios when it comes to the degree of emergence of certain traits and the feasibility to automatically recognize personality under different conditions.

In this paper, we present a novel approach to predict crime in a geographic space from multiple data sources, in particular mobile phone and demographic data. The main contribution of the proposed approach lies in using aggregated and anonymized human behavioral data derived from mobile network activity to tackle the crime prediction problem. While previous research efforts have used either background historical knowledge or offenders' profiling, our findings support the hypothesis that aggregated human behavioral data captured from the mobile network infrastructure, in combination with basic demographic information, can be used to predict crime. In our experimental results with real crime data from London we obtain an accuracy of almost 70% when predicting whether a specific area in the city will be a crime hotspot or not. Moreover, we provide a discussion of the implications of our findings for data-driven crime analysis.

Research has proven that stress reduces quality of life and causes many diseases. For this reason, several researchers devised stress detection systems based on physiological parameters. However, these systems require that obtrusive sensors are continuously carried by the user. In our paper, we propose an alternative approach providing evidence that daily stress can be reliably recognized based on behavioral metrics, derived from the user's mobile phone activity and from additional indicators, such as the weather conditions (data pertaining to transitory properties of the environment) and the personality traits (data concerning permanent dispositions of individuals). Our multifactorial statistical model, which is person-independent, obtains the accuracy score of 72.28% for a 2-class daily stress recognition problem. The model is efficient to implement for most of multimedia applications due to highly reduced low-dimensional feature space (32d). Moreover, we identify and discuss the indicators which have strong predictive power.

The Workshop on Computational Personality Recognition aims to define the state-of-the-art in the field and to provide tools for future standard evaluations in personality recognition tasks. In the WCPR14 we released two different datasets: one of Youtube Vlogs and one of Mobile Phone interactions. We structured the workshop in two tracks: an open shared task, where participants can do any kind of experiment, and a competition. We also distinguished two tasks: A) personality recognition from multimedia data, and B) personality recognition from text only. In this paper we discuss the results of the workshop.

While our daily activities usually involve interactions with others, the current methods on activity recognition do not often exploit the relationship between social interactions and human activity. This paper addresses the problem of interpreting social activity from human interactions captured by mobile sensing networks. Our first goal is to discover different social activities such as chatting with friends from interaction logs and then characterize them by the set of people involved, and the time and location of the occurring event. Our second goal is to perform automatic labeling of the discovered activities using predefined semantic labels such as coffee breaks, weekly meetings, or random discussions. Our analysis was conducted on a real-life interaction network sensed with Bluetooth and infrared sensors of about fifty subjects who carried sociometric badges over 6 weeks. We show that the proposed system reliably recognized coffee breaks with 99% accuracy, while weekly meetings were recognized with 88% accuracy.

Recent studies in social and personality psychology introduced the notion of personality states conceived as concrete behaviors that can be described as having the same contents as traits. Our paper is a first step towards addressing automatically this new perspective. In particular, we will focus on the classification of excerpts of social behavior into personality states corresponding to the Big Five traits, rather than focusing on the more traditional goal of using those behaviors to directly infer about the personality traits of the person producing them. The multimodal behavioral cues we exploit were obtained by means of the Sociometric Badges worn by people working at a research institution for a period of six weeks. We investigate the effectiveness of cues concerning acted social behaviors as well as of other situational characteristics for the sake of personality state classification. The encouraging results show that our classifiers always, and sometimes greatly, improve the performances of a random baseline classifier (from 1.5 to 1.8 better than chance). At a general level, we believe that these results support the proposed shift from the classification of personality traits to the classification of personality states.

The paper addresses the automatic recognition of social and task-oriented functional roles in small-group meetings, focusing on several properties: a) the importance of non-linguistic behaviors, b) the relative time-consistency of the social roles played by a given person during the course of a meeting, and c) the interplays and mutual constraints among the roles enacted by the different participants in a social encounter. In particular, this paper proposes that the Influence Model framework can address these properties of functional roles, and compares the performance obtained by this framework to the performances of models that consider only property (a) (SVM), and to those that address both (a) and (b) (HMM). The results obtained confirm our expectations: the classification of social functional roles improves if models account for temporal dependencies among the roles played by the same subject, for the time properties of the roles played by each individual, and for the mutual constraints among the roles of different group members. The two versions of the Influence Model (IM and newIM), which encode all three properties together, outperform both the SVM and the HMM on most of the figures of merit used. Of particular interest is the capability of the Influence Model to obtain good or very good results on the less-populated classes-Orienteer and Seeker for the task area, and Attacker and Supporter for the socio-emotional area.

In this work, we investigate the relationships between social network structure and personality; we assess the performances of different subsets of structural network features, and in particular those concerned with ego-networks, in predicting the Big-5 personality traits. In addition to traditional survey-based data, this work focuses on social networks derived from real-life data gathered through smartphones. Besides showing that the latter are superior to the former for the task at hand, our results provide a fine-grained analysis of the contribution the various feature sets are able to provide to personality classification, along with an assessment of the relative merits of the various networks exploited.

The user's personality in Human-Computer Interaction (HCI) plays an important role for the overall success of the interaction. The present study focuses on automatically recognizing the Big Five personality traits from 2-5 min long videos, in which the computer interacts using different levels of collaboration, in order to elicit the manifestation of these personality traits. Emotional Stability and Extraversion are the easiest traits to automatically detect under the different collaborative settings: all the settings for Emotional Stability and intermediate and fully-non collaborative settings for Extraversion. Interestingly, Agreeableness and Conscientiousness can be detected only under a moderately non-collaborative setting. Finally, our task does not seem to activate the full range of dispositions for Creativity.

In this paper we modeled the effects that dominant people might induce on the nonverbal behavior (speech energy and body motion) of the other meeting participants using Granger causality technique. Our initial hypothesis that more dominant people have generalized higher influence was not validated when using the DOME-AMI corpus as data source. However, from the correlational analysis some interesting patterns emerged: contradicting our initial hypothesis dominant individuals are not accounting for the majority of the causal flow in a social interaction. Moreover, they seem to have more intense causal effects as their causal density was significantly higher. Finally dominant individuals tend to respond to the causal effects more often with complementarity than with mimicry.

This work investigates the suitability of medium-grained meeting behaviors, namely, speaking time and social attention, for automatic classification of the Extraversion personality trait. Experimental results confirm that these behaviors are indeed effective for the automatic detection of Extraversion. The main findings of our study are that: 1) Speaking time and (some forms of) social gaze are effective indicators of Extraversion, 2) classification accuracy is affected by the amount of time for which meeting behavior is observed, 3) independently considering only the attention received by the target from peers is insufficient, and 4) distribution of social attention of peers plays a crucial role.

Personality plays an important role in the way people manage the images they convey in self-presentations and employment interviews, trying to affect the other"s first impressions and increase effectiveness. This paper addresses the automatically detection of the Big Five personality traits from short (30-120 seconds) self-presentations, by investigating the effectiveness of 29 simple acoustic and visual non-verbal features. Our results show that Conscientiousness and Emotional Stability/Neuroticism are the best recognizable traits. The lower accuracy levels for Extraversion and Agreeableness are explained through the interaction between situational characteristics and the differential activation of the behavioral dispositions underlying those traits.

In this paper we propose a new type of system explicitly aimed at influencing immediate behavior in an informal, non goal-oriented co-located small group. The state of the group dynamics is automatically assessed in order for the system to continuously plan and deploy minimalist strategies using evocative means to influence behavior, rather than explicit recommendations. A key aspect of our approach is that the main interaction channel is left for direct human-to-human interaction, while no large conscious elaboration effort nor actions are meant by the user toward the interface. We present here the concept and a working implementation of a prototype system targeted to a museum scenario. The system has the form of a table in the museum cafe, and is aimed at inducing a group of friends to talk about the content of their visit to the museum.

In this paper, we target the automatic recognition of personality states in a meeting scenario employing visual and acoustic features. The social psychology literature has coined the name personality state to refer to a specific behavioral episode wherein a person behaves as more or less introvert/extrovert, neurotic or open to experience, etc. Personality traits can then be reconstructed as density distributions over personality states. Different machine learning approaches were used to test the effectiveness of the selected features in modeling the dynamics of personality states.

This paper presents a multimodal framework employing eye-gaze, head-pose and speech cues to explain observed social attention patterns in meeting scenes. We first investigate a few hypotheses concerning social attention and characterize meetings and individuals based on ground-truth data. This is followed by replication of ground-truth results through automated estimation of eye-gaze, head-pose and speech activity for each participant. Experimental results show that combining eye-gaze and head-pose estimates decreases error in social attention estimation by over 26%.

Smart spaces not only are surrounded by networked computers, mobile devices, and ubiquitous sensors, but also encapsulate a large amount of social and communication information. Humans, the center ...

Smart spaces are not only surrounded by networked computers, mobile devices, and ubiquitous sensors, they encapsulate a large amount of social and communication information. Humans, the center of p...

The domestic environment is more than a place where to live. It is a territory of meaning, a place where pleasure, affect and aesthetics are deeply interwoven with the functional and utilitarian dimensions. With the aging process, the home is progressively invested with new meanings and functions, and becomes the emotional center of older people's life.

This paper targets the automatic detection of individual performances in group tasks by means of short sequences, "thin slices", of nonverbal behavior. We designed our task as a classification one. We also investigated the relevance of social context in our task and the effectiveness of our feature selection.

Acceptance of adaptive museum guides raises important issues stemming from both the nature of the scenario (museum visit) and the very kind of technological approach adopted (adaptivity). As to the former, museum guides play a utilitarian role in a hedonic scenario; at present, however, it is not clear how this reflects on the balance between intrinsic and extrinsic motivations for guide acceptance. The adaptive nature of the guide, in turn, raises questions about the impact of the opaqueness of the system behavior, of the alleged loss of perceived control over the interaction, and the role of presentation personalization. All these issues are explored in this paper by means of a model derived from TAM and comprising both extrinsic and intrinsic motivational constructs. The results of a analysis of data from 115 subjects show that the motivational structure of the guide usage is mainly utilitarian, with intrinsic motivations playing a role insofar as they acquire an instrumental value. The impact of the control issues on acceptability is low and indirect, while the importance of the feedback provided by the system is confirmed. Finally, personalization positively impacts on user engagement, this way strengthening the empirical and theoretical groundings for work in adaptive systems.

This paper targets the automatic detection of personality traits in a meeting environment by means of audio and visual features; information about the relational context is captured by means of acoustic features designed to that purpose. Two personality traits are considered: Extraversion (from the Big Five) and the Locus of Control. The classification task is applied to thin slices of behaviour, in the form of 1-minute sequences. SVM were used to test the performances of several training and testing instance setups, including a restricted set of audio features obtained through feature selection. The outcomes improve considerably over existing results, provide evidence about the feasibility of the multimodal analysis of personality, the role of social context, and pave the way to further studies addressing different features setups and/or targeting different personality traits.

In this paper, an influence model is used to recognize functional roles played during meetings. Previous works on the same corpus demonstrated a high recognition accuracy using SVMs with RBF kernels. In this paper, we discuss the problems of that approach, mainly over-fitting, the curse of dimensionality and the inability to generalize to different group configurations. We present results obtained with an influence modeling method that avoid these problems and ensures both greater robustness and generalization capability.

Xface is a set of open source tools for creation of embodied conversational agents (ECAs) using MPEG4 and keyframe based rendering driven by SMIL-Agent scripting language. Xface Toolkit, coupled with SMIL-Agent scripting serves as a full 3D facial animation authoring package. Xface project is initiated by Cognitive and Communication Technologies (TCC) division of FBK-irst (formerly ITC-irst). The toolkit is written in ANSI C++, and is open source and platform independent.

In this paper we describe a pilot study for an intervention aimed at enhancing social skills in high functioning children with autism. We found initial evidences that the use of a, social interaction and may lessen the repetitive behaviors typical of autism. These positive effects also appear to be transferred to other tasks following the intervention. We hypothesize that the effect is due to some unique characteristics of the interfaces used, in particular enforcing some tasks to be done together through the use of multiple-user GUI actions.

One of the research goals in the human-computer interaction community is to build believable Embodied Conversational Agents, that is, agents able to communicate complex information with human-like expressiveness and naturalness. Since emotions play a crucial role in human communication and most of them are expressed through the face, having more believable ECAs implies to give them the ability of displaying emotional facial expressions.This paper presents a system based on Hidden Markov Models (HMMs) for the synthesis of emotional facial expressions during speech. The HMMs were trained on a set of emotion examples in which a professional actor uttered Italian non-sense words, acting various emotional facial expressions with different intensities.The evaluation of the experimental results, performed comparing the "synthetic examples" (generated by the system) with a reference "natural example" (one of the actor's examples) in three different ways, shows that HMMs for emotional facial expressions synthesis have some limitations but are suitable to make a synthetic Talking Head more expressive and realistic.

In this paper, we present a study concerning psychological ownership for digital entities in the context of collaborative working environments. In the first part of the paper we present a conceptual framework of ownership: various issues such as definition, effects, target factors and behavioral manifestation are explicated. We then focus on ownership marking, a behavioral manifestation that is closely tied to psychological ownership. We designed an experiment using DiamondTouch Table to investigate the effect of two of the most widely used ownership markers on users' attitudes and performance. Both performance and attitudinal differences were found, suggesting the significant role of ownership and ownership markers in the groupware and interactive workspaces design.

In this paper, we discuss a machine learning approach to automatically detect functional roles played by participants in a face to face interaction. We shortly introduce the coding scheme we used to classify the roles of the group members and the corpus we collected to assess the coding scheme reliability as well as to train statistical systems for automatic recognition of roles. We then discuss a machine learning approach based on multi-class SVM to automatically detect such roles by employing simple features of the visual and acoustical scene. The effectiveness of the classification is better than the chosen baselines and although the results are not yet good enough for a real application, they demonstrate the feasibility of the task of detecting group functional roles in face to face interactions.

The NESPOLE! System is a speech communication system designed to support multilingual interaction between common users and providers of e-commerce services over the Internet. The core of the system is a distributed interlingua-based speech-to-speech translation system, which is supported by multimodal capabilities that allow the two parties participating in the communication to share Web pages and graphical content which can be annotated using gestures. We describe the unique features and considerations behind the design and implementation of this system, and evaluate these within the context of a constructed full prototype of the system that was developed for the domain of travel planning

In this paper we present DaFEx (Database of Facial Expressions), a database created with the purpose of providing a benchmark for the evaluation of the facial expressivity of Embodied Conversational Agents (ECAs). DaFEx consists of 1008 short videos containing emotional facial expressions of the 6 Ekman's emotions plus the neutral expression. The facial expressions were recorded by 8 professional actors (male and female) in two acting conditions ("utterance" and "no- utterance") and at 3 intensity levels (high, medium, low). The properties of DaFEx were studied by having 80 subjects classify the emotion expressed in the videos. High rates of accuracy were obtained for most of the emotions displayed. We also tested the effect of the intensity level, of the articulatory movements due to speech, and of the actors' and subjects' gender, on classification accuracy. The results showed that decoding accuracy decreases with the intensity of emotions; that the presence of articulatory movements negatively affects the recognition of fear, surprise and of the neutral expression, while it improves the recognition of anger; and that facial expressions seem to be recognized (slightly) better when acted by actresses than by actors.

Synthetic characters are an effective modality to convey messages to the user, provide visual feedback about the system internal understanding of the communication, and engage the user in the dialogue through emotional involvement. In this paper we argue for a fine-grain distinction of the expressive capabilities of synthetic agents: avatars should not be considered as an indivisible modality but as the synergic contribution of different communication channels that, properly synchronized, generate an overall communication performance. In this view, we propose SMIL-AGENT as a representation and scripting language for synthetic characters, which abstracts away from the specific implementation and context of use of the character. SMIL-AGENT has been defined starting from SMIL 0.1 standard specification and aims at providing a high-level standardized language for presentations by different synthetic agents within diverse communication and application contexts.

Embodied Conversational Agents that can express emotions are a popular topic. Yet, despite recent attempts, reliable methods are still lacking to assess the quality of facial displays. This paper extends and refines the work in [6], focusing on the role of the upper and the lower portions of the face. We analysed the recognition rates and errors from the responses of 74 subjects to the presentations of dynamic (human and synthetic) faces. The results points to the possibility of: a) addressing the issue of the naturalness of synthetic faces, and b) a greater importance of the upper part.

In this work, we present a study about adaptation on a mobile museum guide aiming at investigating the relationships between personality traits and the attitudes toward some basic dimensions of adaptivity. Each participant was exposed to two simulated systems that realized an adaptive and a non-adaptive version, respectively, on each of the dimensions investigated. The study showed interesting effects of Big Five personality traits on acceptance of the adaptivity dimensions; in particular conscientiousness, creativity and stability. Locus of control seemed to have a limited yet quite selective effect on delegating to the system the choice of follow-ups.

In this paper, we describe the first prototype of a system called StoryTable, aimed at supporting a group of children in the activity of storytelling. The system is based on a special multi-user touchable device (the MERL DiamondTouch) and it was designed with the purpose of enforcing collaboration between children. The paper discusses how the main design choices were influenced by the paradigm of cooperative learning and presents two observational studies to assess the effects of the different design choices on the storytelling activity.

When multilingual communication through a speech-to-speech translation system is supported by multimodal features, e.g. pen-based gestures, the following issues arise concerning the nature of the supported communication: a) to what extend does multilingual communication differ from ‘ordinary’ monolingual communication with respect to the dialogue structure and the communicative strategies used by participants; b) the patterns of integration between speech and gestures. Building on the outcomes of a previous work, we present results from a study aimed at addressing those issues. The initial findings confirm that multilingual communication, and the way in which it is realized by actual systems (e.g., with or without the push-to-talk mode) affects the form and structure of the conversation.

Multimodal interfaces, which combine two or more input modes (speech, pen, touch...), are expected to be more efficient, natural and usable than single-input interfaces. However, the advantage of multimodal input has only been ascertained in highly controlled experimental conditions (S.L. Oviatt, 1997; 1999); in particular, we lack data about what happens with "real" human-human, multilingual communication systems. We discuss the results of an experiment aiming to evaluate the added value of multimodality in a "true" speech-to-speech translation system, the NESPOLE! system, which provides for multilingual and multimodal communication in the tourism domain, allowing users to interact through the Internet sharing maps, Web-pages and pen-based gestures. We compared two experimental conditions differing as to whether multimodal resources were available: a speech-only condition (SO), and a multimodal condition (MM). Most of the data show tendencies for MM to be better than SO.