This demo presents CO-PARLEY, a mobile socially interactive agent designed to support individuals experiencing difficulties with emotion regulation.The system engages users through reciprocal coregulation, a dynamic and two-way process in which the agent and user mutually influence each other's emotional and physiological states.By combining verbal and nonverbal interaction with real-time physiological synchrony, CO-PARLEY fosters therapeutic alliance, trust, and emotional awareness.Built on a modular framework that integrates multimodal sensing, dialogue management, and adaptive behavior generation, the agent supports users in emotionally challenging moments.In general, this alliance can improve the effectiveness of psychoeducational and awareness exercises.

This paper introduces SIA-Lab, a versatile and broadly applicable platform to advance socially interactive agents (SIAs).Unlike existing single-use case systems, SIA-Lab is a modular and scalable framework built on Android and mobile technologies, enabling rapid explorative and comparative studies between human-human and human-agent interactions, offering valuable insights into behavioral dynamics and user engagement.Effective in health-related applications, including screening, therapeutic assistance, and posttreatment care, it is equally adaptable to education and other settings.By integrating dialog management, affective modeling, and multimodal interaction analysis, SIA-Lab represents a comprehensive toolkit for researchers and practitioners to evaluate and design next-generation supportive technologies.

In the field of affective computing, where research continually advances at a rapid pace, the demand for user-friendly tools has become increasingly apparent. In this paper, we present the AffectToolbox, a novel software system that aims to support researchers in developing affect-sensitive studies and prototypes. The proposed system addresses the challenges posed by existing frameworks, which often require profound programming knowledge and cater primarily to power-users or skilled developers. Aiming to facilitate ease of use, the AffectToolbox requires no programming knowledge and offers its functionality to reliably analyze the affective state of users through an accessible graphical user interface. The architecture encompasses a variety of models for emotion recognition on multiple affective channels and modalities, as well as an elaborate fusion system to merge multi-modal assessments into a unified result. The entire system is open-sourced and will be publicly available to ensure easy integration into more complex applications through a well-structured, Python-based code base - therefore marking a substantial contribution toward advancing affective computing research and fostering a more collaborative and inclusive environment within this interdisciplinary field.

Estimating the momentary level of participant's engagement is an important prerequisite for assistive systems that support human interactions. Previous work has addressed this task in within-domain evaluation scenarios, i.e. training and testing on the same dataset. This is in contrast to real-life scenarios where domain shifts between training and testing data frequently occur. With MultiMediate'24, we present the first challenge addressing multi-domain engagement estimation. As training data, we utilise the NOXI database of dyadic novice-expert interactions. In addition to within-domain test data, we add two new test domains. First, we introduce recordings following the NOXI protocol but covering languages that are not present in the NOXI training data. Second, we collected novel engagement annotations on the MPIIGroupInteraction dataset which consists of group discussions between three to four people. In this way, MultiMediate'24 evaluates the ability of approaches to generalise across factors such as language and cultural background, group size, task, and screen-mediated vs. face-to-face interaction. This paper describes the MultiMediate'24 challenge and presents baseline results. In addition, we discuss selected challenge solutions.

Automatic analysis of human behaviour is a fundamental prerequisite for the creation of machines that can effectively interact with- and support humans in social interactions. In MultiMediate'23, we address two key human social behaviour analysis tasks for the first time in a controlled challenge: engagement estimation and bodily behaviour recognition in social interactions. This paper describes the MultiMediate'23 challenge and presents novel sets of annotations for both tasks. For engagement estimation we collected novel annotations on the NOvice eXpert Interaction (NOXI) database. For bodily behaviour recognition, we annotated test recordings of the MPIIGroupInteraction corpus with the BBSI annotation scheme. In addition, we present baseline results for both challenge tasks.

Backchannels, i.e. short interjections of the listener, serve important meta-conversational purposes like signifying attention or indicating agreement. Despite their key role, automatic analysis of backchannels in group interactions has been largely neglected so far. The MultiMediate challenge addresses, for the first time, the tasks of backchannel detection and agreement estimation from backchannels in group conversations. This paper describes the MultiMediate challenge and presents a novel set of annotations consisting of 7234 backchannel instances for the MPIIGroup Interaction dataset. Each backchannel was additionally annotated with the extent by which it expresses agreement towards the current speaker. In addition to a an analysis of the collected annotations, we present baseline results for both challenge tasks.

Artificial mediators are promising to support human group conversations but at present their abilities are limited by insufficient progress in group behaviour analysis. The MultiMediate challenge addresses, for the first time, two fundamental group behaviour analysis tasks in well-defined conditions: eye contact detection and next speaker prediction. For training and evaluation, MultiMediate makes use of the MPIIGroup Interaction dataset consisting of 22 three- to four-person discussions as well as of an unpublished test set of six additional discussions. This paper describes the MultiMediate challenge and presents the challenge dataset including novel fine-grained speaking annotations that were collected for the purpose of MultiMediate. Furthermore, we present baseline approaches and ablation studies for both challenge tasks

A shorter product life cycle and therefore, reduced project duration is the major success factor for machinery and plant engineering today. One use case for Virtual Commissioning (VC) is the early software test without an existing plant. The primary effort of VC lies in the manual creation of the plant model. There are various simulation tools with different advantages available. Selecting one tool has consequences for the whole engineering process. The exchange of models across different tools is not possible or discouraged by the respective vendors due to various reasons. This paper describes a concept to exchange behavior models via the table-based import interfaces for an automated model generation within common simulation tools. The concept utilizes an approach to exchange the behavior of components using the vendor-neutral and standardized programming language PLCopenXML containing all IEC 61131-3 programming languages. This removes the main obstacle for a broader industrial application of VC, as it minimizes the manual effort for creating and exchanging behavior models through the use of standardized library elements and existing interfaces.

Underwater sounds provide essential information for marine researchers to study sea mammals.During long-term studies large amounts of sound signals are being recorded using hydrophones.To facilitate the time consuming process of manually evaluating the recorded data, computational systems are often employed.Recent approaches utilize Convolutional Neural Networks (CNNs) to analyze spectrograms extracted from the audio signal.In this paper we explore the potential of relevance analysis to enhance the performance of existing CNN approaches.For this purpose, we present a fusion system that utilizes intermediate outputs of three state of the art CNNs, which are fine tuned to recognize whale sounds in spectrograms.Hereby we use Explainable Artificial Intelligence (XAI) to asses the relevance of each feature within the obtained representations.Based on those relevance values, we create novel masking algorithms to extract significant subsets of respective representations.These subsets are used to train an ensemble of classification systems that are serving as input for the final fusion step.We observe that a classification system can benefit from the inclusion of Relevance-based Feature Masking in terms of improved performance and reduced input dimensionality.The presented work is part of the INTERSPEECH 2019 Computational Paralinguistics Challenge.

Using Social Agents as health-care assistants or trainers is one focus area of IVA research. This paper presents a concept of our mobile Social Agent EmmA in the role of a vocational reintegration assistant for burn-out outpatient treatment. We follow a typical par- ticipatory design approach including experts and patients in order to address requirements from both sides. Since the success of such treatments is related to a patients emotion regulation capabilities, we employ a real-time social signal interpretation together with a computational simulation of emotion regulation that influences the agent's social behavior as well as the situational selection of verbal treatment strategies. Overall, our interdisciplinary approach sketches a novel integrative concept for Social Agents as assistants for burn-out patients.

The user experience (UX) of graphical user interfaces (GUIs) often depends on how clearly visual designs communicate/signify "affordances", such as if an element on the screen can be pushed, dragged, or rotated. Especially for novice users figuring out the complexity of a new interface can be cumbersome. In the "past" era of mouse-based interaction mouseover effects were successfully utilized to trigger a variety of assistance, and help users in exploring interface elements without causing unintended interactions and associated negative experiences. Today's GUIs are increasingly designed for touch and lack a method similiar to mouseover to help (novice) users to get acquainted with interface elements. In order to address this issue, we have studied gazeover, as a technique for triggering "help or guidance" when a user's gaze is over an interactive element, which we believe is suitable for today's touch interfaces. We report on a user study comparing pragmatic and hedonic qualities of gazeover and mouseover, which showed significant higher ratings in hedonic quality for the gazeover technique. We conclude by discussing limitations and implications of our findings.

We introduce SSJ Creator, a modern Android GUI enabling users to design and execute social signal processing pipelines using nothing but their smartphones and without writing a single line of code. It is based on a modular Java-based social signal processing framework (SSJ), which is able to perform realtime multimodal behaviour analysis on Android devices using both device internal and external sensors.

Visual search plays an important role in our daily lives and can be very frustrating whenever we cannot remember where we left objects, such as keys or wallets. This is especially true for elderly people, since they forget things more often and face this challenge very frequently. While there are several studies which analyze eye movements during visual search, none of them actually tries to detect whether a user is searching for something or not. However, this information is necessary to recognize when the user needs assistance. Therefore, we propose an eye-tracking-based multimodal approach in order to detect visual search and to support the user in that situation. Furthermore, we explore multiple strategies to inform the user of the desired object's location using a head mounted display. With the help of a prototypical implementation and evaluation of the acquired sensor data, we show that our method is feasible and capable of dealing with this challenge.

User authentication systems are at an impasse. The most ubiquitous method -- the password -- has numerous problems, including susceptibility to unintentional exposure via phishing and cross-site password reuse. Second-factor authentication schemes have the potential to increase security but face usability and deployability challenges. For example, conventional second-factor schemes change the user authentication experience. Furthermore, while more secure than passwords, second-factor schemes still fail to provide sufficient protection against (single-use) phishing attacks.