Depression is a significant global health issue with increasing prevalence. Current diagnostic methods rely on subjective observations and questionnaires, often resulting in underestimation of the condition and insufficient treatment. This study investigates voice-based markers for detecting depressive symptoms through a novel system of virtual humans (VHs) capable of engaging in open-ended talks, unlike previous research which relied primarily on structured clinical interview formats. A total of 101 participants (42 with depressive symptoms) engaged in six casual social interactions with VHs simulating basic emotions, forming the DEPTALK dataset. Speech recordings and their automatic transcriptions were processed using state-of-the-art pre-trained transformer-based models to generate embeddings. We first employed a conversation-level aggregation strategy, combining embeddings across each dialogue and classifying them with Extreme Gradient Boosting. A single model trained on all six conversations per participant outperformed emotion-specific models, achieving F1 scores of 0.566 for speech, 0.329 for text, and 0.648 for the multimodal fusion, indicating that aggregating emotionally diverse interactions exposes stronger depression cues. To capture temporal dynamics, we further implemented a turn-level aggregation strategy using Gated Recurrent Units and training on all conversations. This approach improved performance for text (F1 = 0.505) and maintained competitive results for speech (F1 = 0.541), although the multimodal GRU model (F1 = 0.556) did not surpass the best conversation-level model. Overall, findings suggest that in casual conversations, depressive symptoms are primarily conveyed through prosody, with the addition of semantic context further enhancing detection. This study advances the understanding of speech-based depression patterns in simulated social interactions and highlights the potential of using VHs for more objective depressive symptoms detection.

Open-ended questionnaires allow respondents to express freely, capturing richer information than close-ended formats, but they are harder to analyze. Recent natural language processing advancements enable automatic assessment of open-ended responses, yet its use in psychological classification is underexplored. This study proposes a methodology using pre-trained large language models (LLMs) for automatic classification of open-ended questionnaires, applied to autism spectrum disorder (ASD) classification via parental reports. We compare multiple training strategies using transcribed responses from 51 parents (26 with typically developing children, 25 with ASD), exploring variations in model fine-tuning, input representation, and specificity. Subject-level predictions are derived by aggregating 12 individual question responses. Our best approach achieved 84% subject-wise accuracy and 1.0 ROC-AUC using an OpenAI embedding model, per-question training, including questions in the input, and combining the predictions with a voting system. In addition, a zero-shot evaluation using GPT-4o was conducted, yielding comparable results, underscoring the potential of both compact, local models and large out-of-the-box LLMs. To enhance transparency, we explored interpretability methods. Proprietary LLMs like GPT-4o offered no direct explanation, and OpenAI embedding models showed limited interpretability. However, locally deployable LLMs provided the highest interpretability. This highlights a trade-off between proprietary models' performance and local models' explainability. Our findings validate LLMs for automatically classifying open-ended questionnaires, offering a scalable, cost-effective complement for ASD assessment. These results suggest broader applicability for psychological analysis of other conditions, advancing LLM use in mental health research.

Emotions play a critical role in numerous processes, including, but not limited to, social interactions. Consequently, the ability to evoke and recognize emotions is a challenging task with widespread implications, notably in the field of mental health assessment systems. However, up until now, emotional elicitation methods have not utilized simulated open social conversations. Our study introduces a comprehensive Virtual Human (VH), equipped with a realistic avatar and conversational abilities based on a Large Language Model. This architecture integrates psychological constructs—such as personality, mood, and attitudes—with emotional facial expressions, lip synchronization, and voice synthesis. All these features are embedded into a modular, cognitively-inspired framework, specifically designed for voice-based semi-guided emotional conversations in real time. The validation process involved an experiment with 64 participants interacting with six distinct VHs, each designed to provoke a different basic emotion. The system took an average of 4.44 s to generate the VH’s response. Participants assessed the naturalness and realism of the conversation, scoring averages of 4.61 and 4.44 out of 7, respectively. The VHs successfully generated the intended emotional valence in the users, while arousal was not evoked, though it could be recognized in the VHs. Our findings underscore the feasibility of employing VHs within affective computing to elicit emotions in socially and ecologically valid contexts. This development holds significant potential for application in sectors such as health, education, and marketing, among others.

Depressive symptoms involve dysfunctional social attitudes and heightened negative emotional states. Identifying biomarkers requires data collection in realistic environments that activate depression-specific phenomena. However, no previous research analysed biomarkers in combination with AI-powered conversational virtual humans (VH) for mental health assessment. This study aims to explore gaze and head movements patterns related to depressive symptoms during conversations with emotional VH. A total of 105 participants were evenly divided into a control group and a group of subjects with depressive symptoms (SDS). They completed six semi-guided conversations designed to evoke basic emotions. The VHs were developed using a cognitive-inspired framework, enabling real-time voice-based conversational interactions powered by a Large Language Model, and including emotional facial expressions and lip synchronization. They have embedded life-history, context, attitudes, emotions and motivations. Signal processing techniques were applied to obtain gaze and head movements features, and heatmaps were generated. Then, parametric and non-parametric statistical tests were applied to evaluate differences between groups. Additionally, a two-dimensional t-SNE embedding was created and combined with k-means clustering. Results indicate that SDS exhibited shorter blinks and longer saccades. The control group showed affiliative lateral head gyros and accelerations, while the SDS demonstrated stress-related back-and-forth movements. SDS also displayed the avoidance of eye contact. The exploratory multivariate statistical unsupervised learning achieved 72.3% accuracy. The present study analyse biomarkers in affective processes with multiple social contextual factors and information modalities in ecological environments, and enhances our understanding of gaze and head movements patterns in individuals with depressive symptoms, ultimately contributing to the development of more effective assessments and intervention strategies.

Contains fulltext : 295424.pdf (Publisher’s version ) (Open Access)

Researchers increasingly use electrodermal activity (EDA) to assess emotional states, developing novel applications that include disorder recognition, adaptive therapy, and mental health monitoring systems. However, movement can produce major artifacts that affect EDA signals, especially in uncontrolled environments where users can freely walk and move their hands. This work develops a fully automatic pipeline for recognizing and correcting motion EDA artifacts, exploring the suitability of long short-term memory (LSTM) and convolutional neural networks (CNN). First, we constructed the EDABE dataset, collecting 74h EDA signals from 43 subjects collected during an immersive virtual reality task and manually corrected by two experts to provide a ground truth. The LSTM-1D CNN model produces the best performance recognizing 72% of artifacts with 88% accuracy, outperforming two state-of-the-art methods in sensitivity, AUC and kappa, in the test set. Subsequently, we developed a polynomial regression model to correct the detected artifacts automatically. Evaluation of the complete pipeline demonstrates that the automatically and manually corrected signals do not present differences in the phasic components, supporting their use in place of expert manual correction. In addition, the EDABE dataset represents the first public benchmark to compare the performance of EDA correction models. This work provides a pipeline to automatically correct EDA artifacts that can be used in uncontrolled conditions. This tool will allow to development of intelligent devices that recognize human emotional states without human intervention.

Attachment styles are known to have significant associations with mental and physical health. Specifically, insecure attachment leads individuals to higher risk of suffering from mental disorders and chronic diseases. The aim of this study is to develop an attachment recognition model that can distinguish between secure and insecure attachment styles from voice recordings, exploring the importance of acoustic features while also evaluating gender differences. A total of 199 participants recorded their responses to four open questions intended to trigger their attachment system using a web-based interrogation system. The recordings were processed to obtain the standard acoustic feature set eGeMAPS, and recursive feature elimination was applied to select the relevant features. Different supervised machine learning models were trained to recognize attachment styles using both gender-dependent and gender-independent approaches. The gender-independent model achieved a test accuracy of 58.88%, whereas the gender-dependent models obtained 63.88% and 83.63% test accuracy for women and men respectively, indicating a strong influence of gender on attachment style recognition and the need to consider them separately in further studies. These results also demonstrate the potential of acoustic properties for remote assessment of attachment style, enabling fast and objective identification of this health risk factor, and thus supporting the implementation of large-scale mobile screening systems.

Attention, control inhibition, and visual-spatial working memory represent the three basic sets of cognitive processes involving on executive functions (EF). Basic EF are relevant abilities in daily life that allow to control and monitor adapted behaviors in order to achieve specific goals. In the educational field, EF are related to academic achievement, social functioning, as well as the inhibition of maladaptive behaviors. Their impairment often leads to an incapacity to perform multiple and simultaneous mental activities, as well as to plan and monitor learning. The main aim of cognitive neuropsychology intervention is to identify effective methods that allow transferring trained strategies and abilities to daily life. Accordingly, virtual reality games (VRG) are showing ecological validity effectiveness in EF training. In this framework, the aim of this study was to examine the effectiveness of a VRG cooking-based for improving basic EF processing. 31 healthy subjects (M=24.3; SD=2.51) participated to 3 training sessions of 25 minutes each. Each session involved 6 VRG characterized by different levels of difficulties. Three traditional measures were administered to participants pre- and post-VRG: The Corsi test for assessing visual-spatial working memory, the Dual-task, and the Flanker task for attention and inhibition control respectively. The results reported a significant improvement of the three EF abilities after training, showing the potential effectiveness of a VRG along with the traditional measures. Future studies on students with learning disabilities are needed to compare performance and effectiveness.

Neuropsychological assessment has been traditionally made through paper-and-pencil batteries which usually are time-consuming, decontextualized, and non-ecological. These abilities play a critical role in education since they are very related to learning capacity, academic achievement, social functioning, as well as the inhibition of maladaptive behaviors. Meanwhile, serious games are being used in education and psychology to achieve assessments without these limitations, including neuropsychological assessments. While traditional tests can be analyzed with classical statistics, a large number of variables can be extracted from serious games, the analysis of which can be more complex. Machine learning can handle this large amount of information and find patterns that allow us to recognize behaviors. This study aimed to investigate whether machine learning could be used to improve predictive validity in applying a serious game for neuropsychological assessment. Results were based on 60 subjects, including 42 cognitive activities. The validation process showed best results on attention, memory, planning, and cognitive flexibility, achieving accuracies higher or equal to 0.8 and Cohen’s Kappas higher than 0.55, which implies that the Virtual Serious Game could be a valid tool to perform a neuropsychological evaluation along with traditional tests.

Design education and practice are continuously evolving. Educational institutions must include intellectual complexities and new curriculum to support good design education. The design education future emerges multidisciplinary knowledge, teaching innovation and employment necessities. This paper describes a methodology centered in product validation with industrial design students. Focusing on discovering the student experience during the project execution, in addition to observing closely the female design student's perception on the methodology and process developed. The academic project was the design of a novel tool board. The students developed the proposed project in a period of eight weeks. Sixteen students participated as a sample of this research. The methodology consisted of eight phases that spanned from project brief to project conclusion, introducing two phases focused on validation exercises for the elements created to reach the solution of the tool board. During the end of the two evaluation phases, two surveys were applied asking for information on his previous experience during his design education and three elements that assessment the design methodology implementation: utility, novelty, and relevance. Using multiple choice and Likert scale answers the students answered the surveys. The survey's findings revealed relevant information on the project implementation focused on evaluation phases during the product design. The results revealed how students reflected on their previous experience developing projects, and how the design tool board integrate important phases like validation. Also, the students evaluated with a positive value the utility, novelty, and relevance of the developed project. However, the most important finding was the female perception comparing male students. The female assessment of novelty and relevance increased during project implementation, highlighting novelty as a perceived element to a greater range than men. This research results allowed us to discover more information about female students experience with creative and validation processes.

Abstract The validity of environmental simulations depends on their capacity to replicate responses produced in physical environments. However, very few studies validate navigation differences in immersive virtual environments, even though these can radically condition space perception and therefore alter the various evoked responses. The objective of this paper is to validate environmental simulations using 3D environments and head-mounted display devices, at behavioural level through navigation. A comparison is undertaken between the free exploration of an art exhibition in a physical museum and a simulation of the same experience. As a first perception validation, the virtual museum shows a high degree of presence. Movement patterns in both ‘museums’ show close similarities, and present significant differences at the beginning of the exploration in terms of the percentage of area explored and the time taken to undertake the tours. Therefore, the results show there are significant time-dependent differences in navigation patterns during the first 2 minutes of the tours. Subsequently, there are no significant differences in navigation in physical and virtual museums. These findings support the use of immersive virtual environments as empirical tools in human behavioural research at navigation level. Research highlights The latest generation HMDs show a high degree of presence. There are significant differences in navigation patterns during the first 2 minutes of a tour. Adaptation time need to be considered in future research. Training rooms need to be realistic, to avoid the ‘wow’ effect in the main experiment. Results support the use of Virtual Reality and the latest HMDs as empirical tools in human behavioural research at navigation level.