Voice assistants (VAs) are predicted to develop from merely reactive to increasingly proactive agents in the future. While proactivity allows a leap towards more intelligent conversations with VAs, designing proactive agents is not straightforward, as benefit and acceptance of proactive behavior is dependent on a plethora of factors. For instance, proactive agents run the risk of disrupting users who are already engaged in ongoing primary tasks. While a large body of research is therefore concerned with when to proactively interrupt users, how to interrupt them has received less attention. To close this gap, a driving simulator study was conducted to find linguistic best practices for designing proactive prompts in German. Low linguistic complexity as well as suggestive rather than imposing language were found to influence study participants’ preferences for proactive prompts. These findings underline that the existing framework for designing proactive interactions needs to be enhanced by nuanced linguistic considerations.

Technological advancements over the past years have led to a leap in Conversational User Interface (CUI) performance. However, interactions with CUIs are not error-free and error handling thus plays a crucial role in CUI design. So far, CUI repair strategies oftentimes neglect what is known from Human-Human Interaction (HHI). The present work borrows from human-centered error handling and applies the principle of Least Collaborative Effort to Human-Computer Interaction (HCI). A within-subjects WoZ driving simulator study (n = 48) is conducted to examine user preferences qualitatively and quantitatively for seven error handling strategies in an automotive setting. Our findings show that human-centered repair strategies comply with the principle of Least Collaborative Effort and lead to significantly lower repair costs than a classic HCI error handling strategy. These results introduce costs as a measurement tool for examining strategy preferences when repairing erroneous dialogs and offer assistance for designing user-centric error handling strategies.

In this contribution to INTERSPEECH 2010 Paralinguistic Challenge we explore the capabilities of decision level fusion and ensemble based techniques for classification tasks on the provided AGENDER corpus.Ensemble members are generated by providing multiple feature sets generated by feature selection, and novel fusion methods (developed in order to give special support to under-represented classes) are applied for decision making.Results are compared to standard classification approaches and possible benefits are discussed.

The Entertainment potential of Virtual Reality is yet to be fully realised.In recent years, this potential has been described through the Holodeck™ metaphor, without however addressing the issue of content creation and gameplay.Recent progress in Interactive Narrative technology makes it possible to envision immersive systems.Yet, little is known about the usability of such systems or which paradigms should be adopted for gameplay and interaction.We report user experiments carried out with a fully immersive Interactive Narrative system based on a CAVE-like system, which explore two interactivity paradigms for user involvement (Actor and Ghost).Our results confirm the potential of immersive Interactive Narratives in terms of performance but also of user acceptance.

We present a contribution to the Open Performance subchallenge of the INTERSPEECH 2009 Emotion Challenge.We evaluate the feature extraction and classifier of EmoVoice, our framework for real-time emotion recognition from voice on the challenge database and achieve competitive results.Furthermore, we explore the benefits of discretizing numeric acoustic features and find it beneficial in a multi-class task.

In this paper, we report on classification results for emotional user states (4 classes, German database of children interacting with a pet robot).Starting with 5 emotion labels per word, we obtained chunks with different degrees of prototypicality.Six sites computed acoustic and linguistic features independently from each other.A total of 4232 features were pooled together and grouped into 10 low level descriptor types.For each of these groups separately and for all taken together, classification results using Support Vector Machines are reported for 150 features each with the highest individual Information Gain Ratio, for a scale of prototypicality.With both acoustic and linguistic features, we obtained a relative improvement of up to 27.6%, going from low to higher prototypicality.

In this paper, we describe an Augmented Reality Art installation, which reacts to user behaviour using Multimodal analysis of affective signals. The installation features a virtual tree, whose growth is influenced by the perceived emotional response from spectators. The system implements a 'magic mirror' paradigm (using a large-screen display or projection system) and is based on the ARToolkit with extended representations for scene graphs. The system relies on a PAD dimensional model of affect to support the fusion of different affective modalities, while also supporting the representation of affective responses that relate to aesthetic impressions. The influence of affective input on the visual component is achieved by mapping affective data to an L-System governing virtual tree behaviour. We have performed an early evaluation of the system, both from the technical perspective and in terms of user experience. Post-hoc questionnaires were generally consistent with data from multimodal affective processing, and users rated the overall experience as positive and enjoyable, regardless of how proactive they were in their interaction with the installation.

In this paper, we report on classification results for emotional user states (4 classes, German database of children interacting with a pet robot).Six sites computed acoustic and linguistic features independently from each other, following in part different strategies.A total of 4244 features were pooled together and grouped into 12 low level descriptor types and 6 functional types.For each of these groups, classification results using Support Vector Machines and Random Forests are reported for the full set of features, and for 150 features each with the highest individual Information Gain Ratio.The performance for the different groups varies mostly between ≈ 50% and ≈ 60%.

We present a data-mining experiment on feature selection for automatic emotion recognition. Starting from more than 1000 features derived from pitch, energy and MFCC time series, the most relevant features in respect to the data are selected from this set by removing correlated features. The features selected for acted and realistic emotions are analyzed and show significant differences. All features are computed automatically and we also contrast automatically with manually units of analysis. A higher degree of automation did not prove to be a disadvantage in terms of recognition accuracy

Recently, there has been a significant amount of work on the recognition of emotions from speech and biosignals.Most approaches to emotion recognition so far concentrate on a single modality and do not take advantage of the fact that an integrated multimodal analysis may help to resolve ambiguities and compensate for errors.In this paper, we describe various methods for fusing physiological and voice data at the feature-level and the decision-level as well as a hybrid integration scheme.The results of the integrated recognition approach are then compared with the individual recognition results from each modality.