This research involved 88 young adults aged between 20 years and 35 years from two different countries, Spain and Italy. This work aims to explore preferences of the two groups toward synthetic voices, created for the experiment with variations in gender and quality for each language. The Spanish group was asked to evaluate the two high-quality voices of Elena and Pablo and the two low-quality voices of Maria and Juan while the Italian group was asked to assess the high-quality voices of Giulia and Antonio and the low-quality voices of Clara and Edoardo. The shortened and digitized version of the Virtual Agent Voice Acceptance Questionnaire (VAVAQ) was administered, respectively, in the Spanish or Italian version on the basis of the referring group to collect participants’ preferences. Due to the pandemic situation, participants were mainly contacted via email. Each participant was provided with a specific link. Outcomes revealed that Spanish and Italian young adults showed a greater appreciation toward the high-quality female voice compared to the other proposed voices. Regarding participants’ cross-cultural differences, Italian participants seem to judge the voices as more emotionally engaging than the Spanish participants whereas Spanish participants consider the audited voices as more natural and expressive than the Italian participants.

The evaluation of systems that claim to recognise emotions expressed by human beings is a contested and complex task: The early pioneers in this field gave the impression that these system will eventually recognise a flash of anger, suppressed glee/happiness, momentary disgust or contempt, lurking fear, or sadness in someone’s face or voice (Picard and Klein, 2002; Schuller et al., 2011). Emotion recognition systems are trained on ‘labelled’ databases – collection of video/audio recording comprising images and voices of humans enacting one emotional state. Machine learning programs then regress the pixel distributions or wave forms against the labels. The system is then said to have learnt how to recognise and interpret human emotions and rated using information science metrics. These systems are adopted by the world at large for applications ranging from autistic spectrum communications to teaching and learning, and onwards to covert surveillance. The training databases depend upon human emotions recorded in ideal conditions - faces looking at the camera and centrally located, voices articulated through noise-cancelling microphones. Yet there are reports that the posed training data set, that is racially-skewed and gender unbalanced, does not prepare these systems to cope with data-in-the-wild and that expression-unrelated variations (like illumination, head pose, and identity bias (Li and Deng, 2020)) can impact their performance as well. Deployments of these systems tend to adopt one or other and apply it to data collected outside laboratory conditions and use the resulting classifications in subsequent processing. We have devised a testing method that helps to quantify the similarities and differences of facial emotion recognition systems (FER) and speech emotion recognition systems (SER). We report on the development of a data base comprising videos and sound track of 64 politicians and 7 government spokespersons (25 F, 46 M; 34 White Europeans, 19 East Asians, and 18 South Asians), ranging in age from 32-85 years, and each of the 71 has on average three 180 s videos; a total of 16.66 h of data. We have compared the performance of two FERs (Emotient and Affectiva) and two SERs (OpenSmile and Vokaturi) on our data by analysing emotions reported by these systems on a frame-by-frame basis. We have analysed the directly observable head movements, and the indirectly observable muscle movement parts of the face and for the muscle movements in the vocal tract. There was little agreement in emotions recognised, and the differences were exacerbated more women than for men, and more for South and East Asians than for White Europeans. Levels of agreement and disagreement on both high-level (i.e. emotion labels) and lower-level features (e.g. Euler angles of head movement) are shown. We show that inter-system disagreement may also be used as an effective response variable in reasoning about data features that influence disagreement. We argue that reliability of subsequent processing in approaches that adopt these systems may be enhanced by restricting action to cases where systems agree within a given tolerance level. This paper may be considered as a foray into the greater debate about the so-called algorithmic (un)fairness and data bias in the development and deployment of machine learning systems of which FERs and SERs are a good exemplar.

This paper investigates the correlation between mutual gaze and linguistic repetition, a form of alignment, which we take as evidence of mutual understanding. We focus on a multimodal corpus made of three-party conversations and explore the question of whether mutual gaze events correspond to moments of repetition or non-repetition. Our results, although mainly significant on word unigrams and bigrams, suggest positive correlations between the presence of mutual gaze and the repetitions of tokens, lemmas, or parts-of-speech, but negative correlations when it comes to paired levels of representation (tokens or lemmas associated with their part-of-speech). No compelling correlation is found with duration of mutual gaze. Results are strongest when ignoring punctuation as representations of pauses, intonation, etc. in counting aligned tokens.

This paper surveys 60 English Machine Reading Comprehension datasets, with a view to providing a convenient resource for other researchers interested in this problem.We categorize the datasets according to their question and answer form and compare them across various dimensions including size, vocabulary, data source, method of creation, human performance level, and first question word.Our analysis reveals that Wikipedia is by far the most common data source and that there is a relative lack of why, when, and where questions across datasets.

Abstract Assistive Ambient Living (AAL) in ageing refers to any device used to support ageing related psychological and physical changes aimed at improving seniors’ quality of life and reducing caregivers’ burdens. The diffusion of these devices opens the ethical issues related to their use in the human personal space. This is particularly relevant when AAL technologies are devoted to the ageing population that exhibits special bio-psycho-social aspects and needs. In spite of this, relatively little research has focused on ethical issues that emerge from AAL technologies. The present article addresses ethical issues emerging when AAL technologies are implemented for assisting the elderly population and is aimed at raising awareness of these aspects among healthcare providers. The overall conclusion encourages a person-oriented approach when designing healthcare facilities. This process must be fulfilled in compliance with the general principles of ethics and individual nature of the person devoted to. This perspective will develop new research paradigms, paving the way for fulfilling essential ethical principles in the development of future generations of personalized AAL devices to support ageing people living independently at their home.

The probability that intervals are related by a particular Allen relation is calculated relative to sample spaces\nΩn given by the number n of, in one case, points, and, in another, interval names. In both cases, worlds in\nthe sample space are assumed equiprobable, and Allen relations are classified as short, medium and long,\naccording to the number of shared borders.

Spoken interaction with a machine results in a behaviour that is not very common in face-to-face human communication:Off-Talk, which is defined as speech utterances that are not directed to an immediate interlocutor, the machine, but to another person or even oneself. It is our contention that a system which is able to detect theOff-Talkutterances can interact with a human in a more efficient manner by acknowledging that the utterances are not directed to the system and hence, not replying toOff-Talkutterances. In this paper, we demonstrate the discrimination power of a wide range of Electroencephalogram (EEG) frequency bands using wavelet transform analysis and propose models forOn-TalkandOff-Talkdetection using audio and EEG signals, and their fusion. Our study shows that the EEG signal can identify the occurrence ofOff-Talkutterances with promising accuracy and its fusion with audio features adds a slight improvement in these results.

Casual conversation has become a focus for dialogue applications.Such talk is ubiquitous and its structure differs from that found in the task-based interactions that have been the focus of dialogue system design for many years.It is unlikely that such conversations can be modelled as an extension of task-based talk.We review theories of casual conversation, report on our studies of the structure of casual dialogue, and outline challenges we see for the development of spoken dialog systems capable of carrying on casual friendly conversation in addition to performing welldefined tasks.

This paper describes a recently created multimodal corpus that has been designed to address multiparty interaction modelling, specifically collaborative aspects in task-based group interactions. A set of human-human interactions was collected with HD cameras, microphones and a Kinect sensor. The scenario involves 2 participants playing a game instructed and guided by a facilitator. Additionally to the recordings, survey material was collected, including personality tests of the participants and experience assessment questionnaires. The corpus will be exploited for modelling behavioral aspects in collaborative group interaction by taking into account the speakers' multimodal signals and psychological variables.

This work reports on a series of experiments involving 960 participants (aged between 20-30 years and equally balanced by gender), asked to play the receiver role in a modified version of the Ultimatum Game, where together with information on the offer's fairness (e.g. 40 (fair) vs 10 (unfair) of 100 euros), a photo depicted the trader's appearance (trustworthy vs. untrustworthy) and a text provided his moral description (honest vs. dishonest). Receivers were asked to motivate their decision in connection with the appearance, moral judgment, and fairness of the offer, and report on how these variables affected their emotional feelings. Data analysis shows that, in all conditions containing a fair offer, the trader's appearance plays a significant role in the receivers' decisions in terms of acceptance rate. Moral descriptions play a significant role only in conditions containing an unfair offer. However, when asked to motivate their choices, subjects do not feel the interference of the social appearance, rather they provide more or less equal number of motivations with reference to the amount of offers and moral judgments. As for the emotions driving their decisions, non-converging feelings are observed both at intra and inter group level.

In many contexts from casual everyday conversations to formal discussions, people tend to repeat their interlocutors, and themselves. This phenomenon not only yields random repetitions one might expect from a natural Zipfian distribution of linguistic forms, but also projects underlying discourse mechanisms and rhythms that researchers have suggested establishes conversational involvement and may support communicative progress towards mutual understanding. In this paper, advances in an automated method for assessing interlocutor synchrony in task-based Human-to-Human interactions are reported. The method focuses on dialogue structure, rather than temporal distance, measuring repetition between speakers and their interlocutors last n-turns (n = 1, however far back in the conversation that might have been) rather than utterances during a prior window fixed by duration. The significance of distinct linguistic levels of repetition are assessed by observing contrasts between actual and randomized dialogues, in order to provide a quantifying measure of communicative success. Definite patterns of repetitions where identified, notably in contrasting the role of participants (as information giver or follower). The extent to which those interacted sometime surprisingly with gender, eye-contact and familiarity is the principal contribution of this work.

The dynamics of social interactions during human-human conversation have attracted the attention of many researchers aiming at improving the naturalness of automatic dialogue systems and creating socially aware machines.While many studies have investigate social dynamics in relation to the emotional sphere of a conversation, in this work we investigate whether these dynamics are correlated with the discourse structure, particularly topic changes.We conduct an observational study in two corpora of differing nature.In both corpora we find that at topic terminations, the interactional entropy, defined as the amount of social signals exchanged by the participants, decreases with the introduction of a new topic.We conclude that a relation exists between topic changes and amount of social interaction and, hence, that social signals have a discourse value in addition to a social function.This value, we suggest, is group acknowledgement that a new topic has gained the floor.Understanding these social dynamics within conversations can be exploited in the development of socially aware dialogue systems.

Conversational interaction is a dynamic activity in which participants engage in the construction of meaning and in establishing and maintaining social relationships. Lexical and prosodic accommodation have been observed in many studies as contributing importantly to these dimensions of social interaction. However, while previous works have considered accommodation mechanisms at global levels (for whole conversations, halves and thirds of conversations), this work investigates their evolution through repeated analysis at time intervals of increasing granularity to analyze the dynamics of alignment in a spoken language corpus. Results show that the levels of both prosodic and lexical accommodation fluctuate several times over the course of a conversation.

In the past few years there has been a lot of interest and excitement generated by solid modeling. Numerous papers have been presented on the value and benefits of this technology. Many talented individuals have contributed to the state of the art. Solid modeling has played a larger role at SIGGRAPH in the past three years, with tutorials, panel discussions and technical papers. All this material has been presented for the most part by academics, researchers and developers of solid modeling systems.