Learning computer science (CS) often involves cognitively demanding tasks that can induce stress, particularly for novice learners. While prior research has primarily relied on self-report measures to study stress in learning, less is known about how acute socially evaluative stress manifests physiologically during CS learning activities, or how these responses may differ across genders. This study investigates physiological responses to acute stress embedded within an introductory HTML learning module.

In automated UI design generation, a key challenge is the lack of support for iterative processes, as most systems focus solely on end-to-end output. This stems from limited capabilities in interpreting design intent and a lack of transparency for refining intermediate results. To better understand these challenges, we conducted a formative study that identified concrete and actionable requirements for supporting iterative design with Generative Tools. Guided by these findings, we propose PrototypeFlow, a human-centered system for automated UI generation that leverages multi-modal inputs and models. PrototypeFlow takes natural language descriptions and layout preferences as input to generate the high-fidelity UI design. At its core is a theme design module that clarifies implicit design intent through prompt enhancement and orchestrates sub-modules for component-level generation. Designers retain full control over inputs, intermediate results, and final prototypes, enabling flexible and targeted refinement by steering generation and directly editing outputs. Our experiments and user studies confirmed the effectiveness and usefulness of our proposed PrototypeFlow.

The integration of artificial intelligence in education, particularly in computerised adaptive testing (CAT), has the potential to enhance assessment accuracy and efficiency. However, many existing models rely on large datasets, making them impractical for resource-constrained educational settings. This study explores machine learning (ML) approaches for CAT using small datasets, focusing on the TIMSS 2019 dataset. We evaluate ML models for predicting student performance and introduce a recommender system for estimating unattempted test item responses. Our findings show that linear regression with L1 regularisation shows decent results for predicting students' performance levels in TIMSS, especially with limited training data. Furthermore, we demonstrate that truncated singular value decomposition and neighbourhood-based collaborative filtering effectively estimate unattempted item re-sponses, enabling more precise performance predictions. Our results suggest that emphasising test item relationships over individual student characteristics improves prediction accuracy, reducing the need for extensive data calibration. This research contributes to adaptive learning by providing a scalable and computationally efficient approach to CAT, making personalised assessments more accessible in diverse educational environments.

The rise of Large Language Models (LLMs) has streamlined frontend interface creation through tools like Vercel's v0, yet surfaced challenges in design quality (e.g., accessibility, and usability). Current solutions, often limited by their focus, generalisability, or data dependency, fall short in addressing these complexities. Moreover, none of them examine the quality of LLM-generated UI design. In this work, we introduce DesignRepair, a novel dual-stream design guideline-aware system to examine and repair the UI design quality issues from both code aspect and rendered page aspect. We utilised the mature and popular Material Design as our knowledge base to guide this process. Specifically, we first constructed a comprehensive knowledge base encoding Google's Material Design principles into low-level component knowledge base and high-level system design knowledge base. After that, DesignRepair employs a LLM for the extraction of key components and utilizes the Playwright tool for precise page analysis, aligning these with the established knowledge bases. Finally, we integrate Retrieval-Augmented Generation with state-of-the-art LLMs like GPT-4 to holistically refine and repair frontend code through a strategic divide and conquer approach. Our extensive evaluations validated the efficacy and utility of our approach, demonstrating significant enhancements in adherence to design guidelines, accessibility, and user experience metrics.

Mental stress can have a detrimental impact on the lives of individuals who experience it on a regular basis, hence making the development of effective stress detection tools essential. Most current approaches use general models to detect stress, failing to account for individual variability in stress responses that require personalised methods. In this paper, we demonstrate that temporal multimodal models aid personalisation since they have the ability to draw upon diverse sources of information, which together could provide greater insight into the emotional state of an individual. Our methodology integrates multiple modalities including audio, video, physiology and text to provide realtime stress monitoring. We focus on lightweight architecture and compare it with several other methods, and conduct experiments across two datasets. We show that a temporal late-fusion strategy outperforms traditional late-fusion methods, and significantly improves the CCC of the baseline models in both datasets. This highlights how temporal methods best capture the nuanced ways stress manifests in different individuals.

BACKGROUND: Patient data contain a wealth of information that could aid in understanding the onset and progression of disease. However, the task of modelling clinical data, which consist of multiple heterogeneous time series of different lengths, measured at different time intervals, is a complex one. A growing body of research has applied temporal pattern mining to this problem to identify common patterns in clinical attributes over time. However, the vast majority of these algorithms use techniques that are not ideally suited to clinical data. We present an efficient and scalable framework designed specifically for temporal pattern mining of real-world clinical data. Our framework combines temporal abstraction, an extended version of the efficient pattern-growth algorithm, TPMiner, the concepts of relative risk and the odds ratio to identify interesting and high-risk patterns and multiprocessing to improve computational efficiency. A complete set of cut-off values for discretisation and interpretation of the data is provided and is applicable to studies on ageing populations in general. We name this framework Clinical Temporal Pattern Mining or C-TPM. RESULTS: The framework is applied to data from two real-world studies of Alzheimer's disease (AD). The patterns discovered were predictive of AD in survival analysis models with a Concordance index of up to 0.87 and contain clinically relevant variables. A visualisation module provides a clear picture of the discovered patterns for ease of interpretability. CONCLUSIONS: The framework provides an effective and scalable method of modelling multivariate, longitudinal clinical data and can identify patterns in uncommon diseases and those that progress slowly over time. It is generalisable to clinical data from other medical domains as well as non-clinical data.

Emotion elicitation using emotional film clips is one of the most common and ecologically valid methods in Affective Computing. However, selecting and validating appropriate materials that evoke a range of emotions is challenging. Here, we present EmoStim: A Database of Emotional Film Clips as a film library with rich and varied content. EmoStim is designed for researchers interested in studying emotions in relation to either discrete or componential models of emotion. To create the database, 139 film clips were selected from literature and then annotated by 638 participants through the CrowdFlower platform. We selected 99 film clips based on the distribution of subjective ratings that effectively distinguished between emotions defined by the discrete model. We show that the selected film clips reliably induce a range of specific emotions according to the discrete model. Further, we describe relationships between emotions, emotion organization in the componential space, and underlying dimensions representing emotional experience. The EmoStim database and participant annotations are freely available for research purposes. The database can be used to enrich our understanding of emotions further and serve as a guide to select or creating additional materials.

It is our great pleasure to welcome you to the 22nd IEEE International Symposium on Mixed and Augmented Reality (ISMAR), held from 16 to 20 October 2023 in Sydney, Australia. ISMAR stands as the foremost international academic conference in the fields of Augmented Reality (AR), Mixed Reality (MR) and Virtual Reality (VR). Over the years, it has continually expanded its horizon, delving into the latest developments in AR, MR, and VR within both commercial and research domains. The conference is organized and supported by IEEE, IEEE Computer Society, and IEEE VGTC.

Embodied agents, in the form of virtual agents or social robots, are rapidly becoming more widespread. In human-human interactions, humans use nonverbal behaviours to convey their attitudes, feelings, and intentions. Therefore, this capability is also required for embodied agents in order to enhance the quality and effectiveness of their interactions with humans. In this paper, we propose a novel framework that can generate sequences of joint angles from the speech text and speech audio utterances. Based on a conditional Generative Adversarial Network (GAN), our proposed neural network model learns the relationships between the co-speech gestures and both semantic and acoustic features from the speech input. In order to train our neural network model, we employ a public dataset containing co-speech gestures with corresponding speech audio utterances, which were captured from a single male native English speaker. The results from both objective and subjective evaluations demonstrate the efficacy of our gesture-generation framework for Robots and Embodied Agents.

User engagement in Virtual Reality (VR) games is crucial for creating immersive and captivating gaming experiences that meet the expectations of players. However, understanding and measuring these levels in VR games presents a challenge for game designers, as current methods, such as self-reports, may be limited in capturing the full extent of user engagement. Additionally, approaches based on biological signals to measure engagement in VR games present complications and challenges, including signal complexity, interpretation difficulties, and ethical concerns. This study explores body movements, as a novel approach to measure user engagement in VR gaming. We employ E4, emteqPRO, and off-the-shelf IMUs to measure the body movements from diverse participants engaged in multiple VR games. Further, we examine the simultaneous occurrence of player motivation and physiological responses to explore potential associations with body movements. Our findings suggest that body movements hold promise as a reliable and objective indicator of user engagement, offering game designers valuable insights on generating more engaging and immersive experiences.

BACKGROUND: Feature selection is often used to identify the important features in a dataset but can produce unstable results when applied to high-dimensional data. The stability of feature selection can be improved with the use of feature selection ensembles, which aggregate the results of multiple base feature selectors. However, a threshold must be applied to the final aggregated feature set to separate the relevant features from the redundant ones. A fixed threshold, which is typically used, offers no guarantee that the final set of selected features contains only relevant features. This work examines a selection of data-driven thresholds to automatically identify the relevant features in an ensemble feature selector and evaluates their predictive accuracy and stability. Ensemble feature selection with data-driven thresholding is applied to two real-world studies of Alzheimer's disease. Alzheimer's disease is a progressive neurodegenerative disease with no known cure, that begins at least 2-3 decades before overt symptoms appear, presenting an opportunity for researchers to identify early biomarkers that might identify patients at risk of developing Alzheimer's disease. RESULTS: The ensemble feature selectors, combined with data-driven thresholds, produced more stable results, on the whole, than the equivalent individual feature selectors, showing an improvement in stability of up to 34%. The most successful data-driven thresholds were the robust rank aggregation threshold and the threshold algorithm threshold from the field of information retrieval. The features identified by applying these methods to datasets from Alzheimer's disease studies reflect current findings in the AD literature. CONCLUSIONS: Data-driven thresholds applied to ensemble feature selectors provide more stable, and therefore more reproducible, selections of features than individual feature selectors, without loss of performance. The use of a data-driven threshold eliminates the need to choose a fixed threshold a-priori and can select a more meaningful set of features. A reliable and compact set of features can produce more interpretable models by identifying the factors that are important in understanding a disease.

Emotions are multifaceted phenomena that affect our behaviour, perception, and cognition. Increasing evidence indicates that induction mechanisms play a crucial role in triggering emotions by simulating the sensations required for an experimental design. Over the years, many reviews have evaluated a passive elicitation mechanism where the user is an observer, ignoring the importance of self-relevance in emotional experience. So, in response to the gap in the literature, this study intends to explore the possibility of using Virtual Reality (VR) as an active mechanism for emotion induction. VR can simulate controlled environments with high immersion, presence, and interaction to induce intense emotions. Therefore, researchers can evaluate emotional experiences in a realistic context. For the success and quality of research settings, VR must select the appropriate material to effectively evoke emotions. Therefore, in the present review, we evaluated to what extent VR virtual environments, videos, games, tasks, avatar, images, and 360-degree panoramas can elicit emotions. Further, we present public datasets, discuss challenges and recommendations, and review emotion-sensing interfaces related to VR research. The conclusions reveal the VR's potential to evoke emotions effectively and naturally by generating motivational and empathy mechanisms, which makes it an ecologically valid paradigm to study emotions.

Emotions are an inseparable part of human nature affecting our behavior in response to the outside world. Although most empirical studies have been dominated by two theoretical models including discrete categories of emotion and dichotomous dimensions, results from neuroscience approaches suggest a multi-processes mechanism underpinning emotional experience with a large overlap across different emotions. While these findings are consistent with the influential theories of emotion in psychology that emphasize a role for multiple component processes to generate emotion episodes, few studies have systematically investigated the relationship between discrete emotions and a full componential view. This article applies a componential framework with a data-driven approach to characterize emotional experiences evoked during movie watching. The results suggest that differences between various emotions can be captured by a few (at least 6) latent dimensions, each defined by features associated with component processes, including appraisal, expression, physiology, motivation, and feeling. In addition, the link between discrete emotions and component model is explored and results show that a componential model with a limited number of descriptors is still able to predict the level of experienced discrete emotion(s) to a satisfactory level. Finally, as appraisals may vary according to individual dispositions and biases, we also study the relationship between personality traits and emotions in our computational framework and show that the role of personality on discrete emotion differences can be better justified using the component model.

Humans use gestures as a means of non-verbal communication. Often accompanying speech, these gestures have several purposes but in general, aim to convey an intended message to the receiver. Researchers have tried to develop systems to allow embodied agents to be better communicators when interacting with humans via using gestures. In this article, we present a scoping literature review of the methods and the metrics used to generate and evaluate co-speech gestures. After collecting a set of papers using a term search on the Scopus database, we analysed the content of these papers based on methodology (i.e., model, the dataset used), evaluation measures (i.e., objective and subjective) and limitations. The results indicate that data-driven approaches are used more frequently. In terms of evaluation measures, we found a trend of combining objective and subjective metrics, while no standards exist for either. This literature review provides an overview of the research in the area and, more specifically insights the trends and the challenges to be met in building a system to automatically generate gestures for embodied agents.

In this study, we propose our data-driven approach to investigate the emotional experience triggered using Virtual Reality (VR) games. We considered a full Component Process Model (CPM) which theorise emotional experience as a multi-process phenomenon. We validated the possibility of the proposed approach through a pilot experiment and confirmed that VR games can be used to trigger a diverse range of emotions. Using hierarchical clustering, we showed a clear distinction between positive and negative emotion in the CPM space.

Emotions are inseparable part of human nature affecting our behavior in response to the outside world. Although most empirical studies have been dominated by two theoretical models including discrete categories of emotion and dichotomous dimensions, results from neuroscience approaches suggest a multi-processes mechanism underpinning emotional experience with a large overlap across different emotions. While these findings are consistent with the influential theories of emotion in psychology that emphasise a role for multiple component processes to generate emotion episodes, few studies have systematically investigated the relationship between discrete emotions and a full componential view. This paper applies a componential framework with a data-driven approach to characterise emotional experiences evoked during movie watching. Results suggest that differences between various emotions can be captured by a few (at least 6) latent dimensions, each defined by features associated with component processes, including appraisal, expression, physiology, motivation, and feeling. In addition, the link between discrete emotions and component model is explored and results show that a componential model with limited number of descriptors is still able to predict the level of experienced discrete emotion(s) to a satisfactory level.

This paper proposes a prosody based approach for Automatic Personality Perception. Social psychology has shown that whenever we listen to a voice for the first time, we spontaneously and unconsciously attribute personality traits to the speaker. The attribution process is not necessarily accurate, but it is important because it shapes our behavior towards others. The experiments of this work are performed over a corpus of 640 speech samples (322 individuals in total) assessed in terms of speaker's personality traits by 11 judges. The results show that it is possible to predict some of the personality traits with accuracy higher than 70%. The effect of different prosodic features has also been analyzed and compared with findings in the psychological literature.

Personality is a psychological construct aimed at explaining the wide variety of human behaviors in terms of a few, stable and measurable individual characteristics. In this respect, any technology involving understanding, prediction and synthesis of human behavior is likely to benefit from Personality Computing approaches, i.e. from technologies capable of dealing with human personality. This paper is a survey of such technologies and it aims at providing not only a solid knowledge base about the state-of-the-art, but also a conceptual model underlying the three main problems addressed in the literature, namely Automatic Personality Recognition (inference of the true personality of an individual from behavioral evidence), Automatic Personality Perception (inference of personality others attribute to an individual based on her observable behavior) and Automatic Personality Synthesis (generation of artificial personalities via embodied agents). Furthermore, the article highlights the issues still open in the field and identifies potential application areas.

By explicitly describing what has been done in the past, surveys implicitly outline what can (and sometimes should) be done in the future. The insightful commentary by Wright contributes significantly to this latter aspect, especially when it comes to aligning Personality Computing with the latest developments in Personality Science. This response article tries to progress in such a direction by discussing on Wright's suggestions from a computing science point of view.

Persuasive communication is part of everyone's daily life. With the emergence of social websites like YouTube, Facebook and Twitter, persuasive communication is now seen online on a daily basis. This paper explores the effect of multi-modality and perceived personality on persuasiveness of social multimedia content. The experiments are performed over a large corpus of movie review clips from Youtube which is presented to online annotators in three different modalities: only text, only audio and video. The annotators evaluated the persuasiveness of each review across different modalities and judged the personality of the speaker. Our detailed analysis confirmed several research hypotheses designed to study the relationships between persuasion, perceived personality and communicative channel, namely modality. Three hypotheses are designed: the first hypothesis studies the effect of communication modality on persuasion, the second hypothesis examines the correlation between persuasion and personality perception and finally the third hypothesis, derived from the first two hypotheses explores how communication modality influence the personality perception.

In this paper, we present ongoing experiments and insights regarding automatic assessment of perceived personality. While within the INTERSPEECH Speaker Trait Challenge participants will train systems in order to recognize binary targets along the Big 5 personality trait, we will analyze and discuss properties of the data, the labeling scheme and the predictive quality. Conducting factor analyses, estimating reliability, and building regression models capturing dimensions of personality we compare all results to our former and current work and introduce a new extension of our personality database. Eventually, this paper contributes in methodology and understanding on how to asses the perceived personality from an unknown speaker by humans and machines.

LIDIAP

From a cognitive point of view, personality perception corresponds to capturing individual differences and can be thought of as positioning the people around us in an ideal personality space. The more similar the personality of two individuals, the closer their position in the space. This work shows that the mutual position of two individuals in the personality space can be inferred from prosodic features. The experiments, based on ordinal regression techniques, have been performed over a corpus of 640 speech samples comprising 322 individuals assessed in terms of personality traits by 11 human judges, which is the largest database of this type in the literature. The results show that the mutual position of two individuals can be predicted with up to 80% accuracy.

Whenever we listen to a voice for the first time, we attribute personality traits to the speaker. The process takes place in a few seconds and it is spontaneous and unaware. While the process is not necessarily accurate (attributed traits do not necessarily correspond to the actual traits of the speaker), still it significantly influences our behavior toward others, especially when it comes to social interaction. This paper proposes an approach for the automatic prediction of the traits the listeners attribute to a speaker they never heard before. The experiments are performed over a corpus of 640 speech clips (322 identities in total) annotated in terms of personality traits by 11 assessors. The results show that it is possible to predict with high accuracy (more than 70 percent depending on the particular trait) whether a person is perceived to be in the upper or lower part of the scales corresponding to each of the Big -Five, the personality dimensions known to capture most of the individual differences.

This paper presents experiments where natural and spontaneous cognitive processes, in particular those who lead to the attribution of personality traits to unacquainted people, are used as a natural form of feature extraction. In particular, personality assessments provided by human judges are used as features to distinguish between professional and non-professional speakers. The same task is performed with prosodic features extracted with a fully automatic process for comparison purposes. Furthermore both prosodic features and personality assessments are combined. The results show that the discrimination between professional and non-professional speaking styles can be performed with an accuracy of 87.2% when using prosodic features, of 75.5% when using personality assessments, and of 90.0% when using the combination of the two.

This paper proposes an approach for the automatic recognition of roles in settings like news and talk-shows, where roles correspond to specific functions like Anchorman, Guest or Interview Participant. The approach is based on purely nonverbal vocal behavioral cues, including who talks when and how much (turn-taking behavior), and statistical properties of pitch, formants, energy and speaking rate (prosodic behavior). The experiments have been performed over a corpus of around 50 hours of broadcast material and the accuracy, percentage of time correctly labeled in terms of role, is up to 89%. Both turn-taking and prosodic behavior lead to satisfactory results. Furthermore, on one database, their combination leads to a statistically significant improvement.