Stress can have long term adverse effects on individuals' physical and mental well-being. Changes in the speech production process is one of many physiological changes that happen during stress. Microphones, embedded in mobile phones and carried ubiquitously by people, provide the opportunity to continuously and non-invasively monitor stress in real-life situations. We propose StressSense for unobtrusively recognizing stress from human voice using smartphones. We investigate methods for adapting a one-size-fits-all stress model to individual speakers and scenarios. We demonstrate that the StressSense classifier can robustly identify stress across multiple individuals in diverse acoustic environments: using model adaptation StressSense achieves 81% and 76% accuracy for indoor and outdoor environments, respectively. We show that StressSense can be implemented on commodity Android phones and run in real-time. To the best of our knowledge, StressSense represents the first system to consider voice based stress detection and model adaptation in diverse real-life conversational situations using smartphones.

Incorporating annotators' knowledge into a machine-learning framework for detecting psychological traits using multimodal data is an open issue in human communication and social computing. We present a model that is designed to exploit the subjective judgements of multiple annotators on a social trait labeling task. Our two-stage model first estimates a ground truth by modeling the annotators using both the annotations and annotators' self-reported confidences. In the second stage, we train a classifier using the estimated ground truth as labels. We also define ways to verify the consistency of our model and validate it using annotations and nonverbal cues for a dominance estimation task in a group interaction scenario on the publicly available DOME corpus, in addition to synthetically generated data. Our models give satisfactory results, outperforming the commonly used majority voting as well as other approaches in the literature.

This paper addresses the task of automatically detecting outcomes of social interaction patterns, using non-verbal audio cues in competitive role-playing games (RPGs). For our experiments, we introduce a new data set which features 3 hours of audio-visual recordings of the popular “Are you a Werewolf?” RPG. Two problems are approached in this paper: Detecting lying or suspicious behavior using non-verbal audio cues in a social context and predicting participants' decisions in a game-day by analyzing speaker turns. Our best classifier exhibits a performance improvement of 87% over the baseline for detecting deceptive roles. Also, we show that speaker turn based features can be used to determine the outcomes in the initial stages of the game, when the group is large.

In this paper we present the Idiap Wolf Database. This is a audio-visual corpus containing natural conversational data of volunteers who took part in a competitive role-playing game. Four groups of 8-12 people were recorded. In total, just over 7 hours of interactive conversational data was collected. The data has been annotated in terms of the roles and outcomes of the game. There are 371 examples of different roles played over 50 games. Recordings were made with headset microphones, an 8-microphone array, and 3 video cameras and are fully synchronised. The novelty of this data is that some players have deceptive roles and the participants do not know what roles other people play.

Text Dependent (TD) Speaker Recognition systems assume that the password to be uttered by the speaker is known to the system. As the password is known, the system can apply a password-specific model capturing the speaker dynamics well. This enables TD systems to perform better than textindependent systems. We present a variation of the TD systems, called text-conditioning, in which the password is uniquely chosen by each user. This delivers a higher level of discrimination since the linguistic and phonetic differences of the passwords themselves are exploited in separating the speakers. As the database for such a study was not publicly available, we built an extensive database for speaker recognition having such text-conditioning property. The database is tested with various speaker recognition trials. The results indicate that for the design of a practical TD speakerrecognition system, “text-conditioning ” does offer a significant edge.