Depression is a significant global health issue with increasing prevalence. Current diagnostic methods rely on subjective observations and questionnaires, often resulting in underestimation of the condition and insufficient treatment. This study investigates voice-based markers for detecting depressive symptoms through a novel system of virtual humans (VHs) capable of engaging in open-ended talks, unlike previous research which relied primarily on structured clinical interview formats. A total of 101 participants (42 with depressive symptoms) engaged in six casual social interactions with VHs simulating basic emotions, forming the DEPTALK dataset. Speech recordings and their automatic transcriptions were processed using state-of-the-art pre-trained transformer-based models to generate embeddings. We first employed a conversation-level aggregation strategy, combining embeddings across each dialogue and classifying them with Extreme Gradient Boosting. A single model trained on all six conversations per participant outperformed emotion-specific models, achieving F1 scores of 0.566 for speech, 0.329 for text, and 0.648 for the multimodal fusion, indicating that aggregating emotionally diverse interactions exposes stronger depression cues. To capture temporal dynamics, we further implemented a turn-level aggregation strategy using Gated Recurrent Units and training on all conversations. This approach improved performance for text (F1 = 0.505) and maintained competitive results for speech (F1 = 0.541), although the multimodal GRU model (F1 = 0.556) did not surpass the best conversation-level model. Overall, findings suggest that in casual conversations, depressive symptoms are primarily conveyed through prosody, with the addition of semantic context further enhancing detection. This study advances the understanding of speech-based depression patterns in simulated social interactions and highlights the potential of using VHs for more objective depressive symptoms detection.

Open-ended questionnaires allow respondents to express freely, capturing richer information than close-ended formats, but they are harder to analyze. Recent natural language processing advancements enable automatic assessment of open-ended responses, yet its use in psychological classification is underexplored. This study proposes a methodology using pre-trained large language models (LLMs) for automatic classification of open-ended questionnaires, applied to autism spectrum disorder (ASD) classification via parental reports. We compare multiple training strategies using transcribed responses from 51 parents (26 with typically developing children, 25 with ASD), exploring variations in model fine-tuning, input representation, and specificity. Subject-level predictions are derived by aggregating 12 individual question responses. Our best approach achieved 84% subject-wise accuracy and 1.0 ROC-AUC using an OpenAI embedding model, per-question training, including questions in the input, and combining the predictions with a voting system. In addition, a zero-shot evaluation using GPT-4o was conducted, yielding comparable results, underscoring the potential of both compact, local models and large out-of-the-box LLMs. To enhance transparency, we explored interpretability methods. Proprietary LLMs like GPT-4o offered no direct explanation, and OpenAI embedding models showed limited interpretability. However, locally deployable LLMs provided the highest interpretability. This highlights a trade-off between proprietary models' performance and local models' explainability. Our findings validate LLMs for automatically classifying open-ended questionnaires, offering a scalable, cost-effective complement for ASD assessment. These results suggest broader applicability for psychological analysis of other conditions, advancing LLM use in mental health research.

Emotions play a critical role in numerous processes, including, but not limited to, social interactions. Consequently, the ability to evoke and recognize emotions is a challenging task with widespread implications, notably in the field of mental health assessment systems. However, up until now, emotional elicitation methods have not utilized simulated open social conversations. Our study introduces a comprehensive Virtual Human (VH), equipped with a realistic avatar and conversational abilities based on a Large Language Model. This architecture integrates psychological constructs—such as personality, mood, and attitudes—with emotional facial expressions, lip synchronization, and voice synthesis. All these features are embedded into a modular, cognitively-inspired framework, specifically designed for voice-based semi-guided emotional conversations in real time. The validation process involved an experiment with 64 participants interacting with six distinct VHs, each designed to provoke a different basic emotion. The system took an average of 4.44 s to generate the VH’s response. Participants assessed the naturalness and realism of the conversation, scoring averages of 4.61 and 4.44 out of 7, respectively. The VHs successfully generated the intended emotional valence in the users, while arousal was not evoked, though it could be recognized in the VHs. Our findings underscore the feasibility of employing VHs within affective computing to elicit emotions in socially and ecologically valid contexts. This development holds significant potential for application in sectors such as health, education, and marketing, among others.

Depressive symptoms involve dysfunctional social attitudes and heightened negative emotional states. Identifying biomarkers requires data collection in realistic environments that activate depression-specific phenomena. However, no previous research analysed biomarkers in combination with AI-powered conversational virtual humans (VH) for mental health assessment. This study aims to explore gaze and head movements patterns related to depressive symptoms during conversations with emotional VH. A total of 105 participants were evenly divided into a control group and a group of subjects with depressive symptoms (SDS). They completed six semi-guided conversations designed to evoke basic emotions. The VHs were developed using a cognitive-inspired framework, enabling real-time voice-based conversational interactions powered by a Large Language Model, and including emotional facial expressions and lip synchronization. They have embedded life-history, context, attitudes, emotions and motivations. Signal processing techniques were applied to obtain gaze and head movements features, and heatmaps were generated. Then, parametric and non-parametric statistical tests were applied to evaluate differences between groups. Additionally, a two-dimensional t-SNE embedding was created and combined with k-means clustering. Results indicate that SDS exhibited shorter blinks and longer saccades. The control group showed affiliative lateral head gyros and accelerations, while the SDS demonstrated stress-related back-and-forth movements. SDS also displayed the avoidance of eye contact. The exploratory multivariate statistical unsupervised learning achieved 72.3% accuracy. The present study analyse biomarkers in affective processes with multiple social contextual factors and information modalities in ecological environments, and enhances our understanding of gaze and head movements patterns in individuals with depressive symptoms, ultimately contributing to the development of more effective assessments and intervention strategies.

Facial emotion processing by the brain plays a decisive role in human social interactions. This signal helps us interpret and predict people's behaviours. However, other social signals such as human voices or human body odours may facilitate or impair the identification of facial expressions. Here we studied the effects of emotional human body odours on face processing by measuring evoked neural responses and brain connectivity using the electroencephalogram (EEG). We used an emotion recognition task in which the participants attributed an emotion (i.e. happy vs fearful) to a presented face image while simultaneously exposed to emotional body odours. First, we measured face related potentials (FRP)s including P100 and N170 components. Statistical analyses revealed significant differences among FRPs recorded in different odour conditions. Second, we used a hierarchical Bayesian approach including a group dynamic causal model (DCM) followed by parametric empirical Bayes (PEB) to characterize the brain network explaining differences between FRPs. Our preliminary results suggested that different brain networks contribute to neutral face processing in the presence of different emotional body odours.

Contains fulltext : 295424.pdf (Publisher’s version ) (Open Access)

Researchers increasingly use electrodermal activity (EDA) to assess emotional states, developing novel applications that include disorder recognition, adaptive therapy, and mental health monitoring systems. However, movement can produce major artifacts that affect EDA signals, especially in uncontrolled environments where users can freely walk and move their hands. This work develops a fully automatic pipeline for recognizing and correcting motion EDA artifacts, exploring the suitability of long short-term memory (LSTM) and convolutional neural networks (CNN). First, we constructed the EDABE dataset, collecting 74h EDA signals from 43 subjects collected during an immersive virtual reality task and manually corrected by two experts to provide a ground truth. The LSTM-1D CNN model produces the best performance recognizing 72% of artifacts with 88% accuracy, outperforming two state-of-the-art methods in sensitivity, AUC and kappa, in the test set. Subsequently, we developed a polynomial regression model to correct the detected artifacts automatically. Evaluation of the complete pipeline demonstrates that the automatically and manually corrected signals do not present differences in the phasic components, supporting their use in place of expert manual correction. In addition, the EDABE dataset represents the first public benchmark to compare the performance of EDA correction models. This work provides a pipeline to automatically correct EDA artifacts that can be used in uncontrolled conditions. This tool will allow to development of intelligent devices that recognize human emotional states without human intervention.

Attachment styles are known to have significant associations with mental and physical health. Specifically, insecure attachment leads individuals to higher risk of suffering from mental disorders and chronic diseases. The aim of this study is to develop an attachment recognition model that can distinguish between secure and insecure attachment styles from voice recordings, exploring the importance of acoustic features while also evaluating gender differences. A total of 199 participants recorded their responses to four open questions intended to trigger their attachment system using a web-based interrogation system. The recordings were processed to obtain the standard acoustic feature set eGeMAPS, and recursive feature elimination was applied to select the relevant features. Different supervised machine learning models were trained to recognize attachment styles using both gender-dependent and gender-independent approaches. The gender-independent model achieved a test accuracy of 58.88%, whereas the gender-dependent models obtained 63.88% and 83.63% test accuracy for women and men respectively, indicating a strong influence of gender on attachment style recognition and the need to consider them separately in further studies. These results also demonstrate the potential of acoustic properties for remote assessment of attachment style, enabling fast and objective identification of this health risk factor, and thus supporting the implementation of large-scale mobile screening systems.

Attention, control inhibition, and visual-spatial working memory represent the three basic sets of cognitive processes involving on executive functions (EF). Basic EF are relevant abilities in daily life that allow to control and monitor adapted behaviors in order to achieve specific goals. In the educational field, EF are related to academic achievement, social functioning, as well as the inhibition of maladaptive behaviors. Their impairment often leads to an incapacity to perform multiple and simultaneous mental activities, as well as to plan and monitor learning. The main aim of cognitive neuropsychology intervention is to identify effective methods that allow transferring trained strategies and abilities to daily life. Accordingly, virtual reality games (VRG) are showing ecological validity effectiveness in EF training. In this framework, the aim of this study was to examine the effectiveness of a VRG cooking-based for improving basic EF processing. 31 healthy subjects (M=24.3; SD=2.51) participated to 3 training sessions of 25 minutes each. Each session involved 6 VRG characterized by different levels of difficulties. Three traditional measures were administered to participants pre- and post-VRG: The Corsi test for assessing visual-spatial working memory, the Dual-task, and the Flanker task for attention and inhibition control respectively. The results reported a significant improvement of the three EF abilities after training, showing the potential effectiveness of a VRG along with the traditional measures. Future studies on students with learning disabilities are needed to compare performance and effectiveness.

Neuropsychological assessment has been traditionally made through paper-and-pencil batteries which usually are time-consuming, decontextualized, and non-ecological. These abilities play a critical role in education since they are very related to learning capacity, academic achievement, social functioning, as well as the inhibition of maladaptive behaviors. Meanwhile, serious games are being used in education and psychology to achieve assessments without these limitations, including neuropsychological assessments. While traditional tests can be analyzed with classical statistics, a large number of variables can be extracted from serious games, the analysis of which can be more complex. Machine learning can handle this large amount of information and find patterns that allow us to recognize behaviors. This study aimed to investigate whether machine learning could be used to improve predictive validity in applying a serious game for neuropsychological assessment. Results were based on 60 subjects, including 42 cognitive activities. The validation process showed best results on attention, memory, planning, and cognitive flexibility, achieving accuracies higher or equal to 0.8 and Cohen’s Kappas higher than 0.55, which implies that the Virtual Serious Game could be a valid tool to perform a neuropsychological evaluation along with traditional tests.

Abstract The validity of environmental simulations depends on their capacity to replicate responses produced in physical environments. However, very few studies validate navigation differences in immersive virtual environments, even though these can radically condition space perception and therefore alter the various evoked responses. The objective of this paper is to validate environmental simulations using 3D environments and head-mounted display devices, at behavioural level through navigation. A comparison is undertaken between the free exploration of an art exhibition in a physical museum and a simulation of the same experience. As a first perception validation, the virtual museum shows a high degree of presence. Movement patterns in both ‘museums’ show close similarities, and present significant differences at the beginning of the exploration in terms of the percentage of area explored and the time taken to undertake the tours. Therefore, the results show there are significant time-dependent differences in navigation patterns during the first 2 minutes of the tours. Subsequently, there are no significant differences in navigation in physical and virtual museums. These findings support the use of immersive virtual environments as empirical tools in human behavioural research at navigation level. Research highlights The latest generation HMDs show a high degree of presence. There are significant differences in navigation patterns during the first 2 minutes of a tour. Adaptation time need to be considered in future research. Training rooms need to be realistic, to avoid the ‘wow’ effect in the main experiment. Results support the use of Virtual Reality and the latest HMDs as empirical tools in human behavioural research at navigation level.

The objective of this study is to verify whether Virtual Reality together with relaxation techniques incorporated into it improve the students’ attention levels. To do so we made an intervention in an “Initial Vocational Qualification Program” group of Spanish high school in the Valencia metropolitan area. We had 13 students aged between 16 and 19 that participated in the study using an Oculus Rift virtual reality headset. These students were subject to an experimental environment that simulates a beach sunset. They had to perform breathing exercises and concentrate their attention on their breathing, the rhythm of the waves and on flowers of different colors that were appearing in the environment. Experimental results have revealed that levels of attention measured with the Trail Making Test (TMT) were improved when participants had the help of the virtual environment.

[EN] In the field of virtual reality (VR), many efforts have been made to analyze presence, the sense of being in the virtual world. However, it is only recently that functional magnetic resonance imaging (fMRI) has been used to study presence during an automatic navigation through a virtual environment. In the present work, our aim was to use fMRI to study the sense of presence during a VR-free navigation task, in comparison with visualization of photographs and videos (automatic navigations through the same environment). The main goal was to analyze the usefulness of fMRI for this purpose, evaluating whether, in this context, the interaction between the subject and the environment is performed naturally, hiding the role of technology in the experience. We monitored 14 right-handed healthy females aged between 19 and 25 years. Frontal, parietal and occipital regions showed their involvement during free virtual navigation. Moreover, activation in the dorsolateral prefrontal cortex was also shown to be negatively correlated to sense of presence and the postcentral parietal cortex and insula showed a parametric increased activation according to the condition-related sense of presence, which suggests that stimulus attention and self-awareness processes related to the insula may be linked to the sense of presence.

Acquired brain injury (ABI) is one of the main problems of disability and death in the world. Its incidence and survival rate are increasing annually. Thus, the number of chronic ABI patients is gradually growing. Traditionally, rehabilitation programs are applied to postacute and acute patients, but recent publications determine that chronic patients may benefit from rehabilitation. Also, in the last few years, the potential of virtual rehabilitation (VR) systems has been demonstrated. However, until now, no previous studies have been carried out to compare the evolution of chronic patients with acute patients in a VR program. To perform this study, we developed a VR system for ABI patients. The system, vestibular virtual rehabilitation (V2R), was designed with clinical specialists. V2R has been tested with 21 people ranging in age from 18 to 80 years old that were classified in two groups: chronic patients and acute patients. The results demonstrate a similar recovery for chronic and acute patients during the intervention period. Also, the results showed that chronic patients stop their improvement when they finish their training. This conclusion encourages us to direct our developments toward VR systems that can be easily integrated at home, allowing chronic patients to have a permanent VR training program.

The goal of the software proposed in this paper is to assist ophthalmologists in diagnosis and disease prevention, helping them to determine cardiovascular risk or other diseases where the vessels can be altered, as well as to monitor the pathology progression and response to different treatments. The performance of the tool has been evaluated by means of a double-blind study where its sensitivity, specificity, and reproducibility to discriminate between health fundus (without cardiovascular risk) and hypertensive patients has been calculated in contrast to an expert ophthalmologist opinion obtained through a visual inspection of the fundus image. An improvement of almost 20% has been achieved comparing the system results with the clinical visual classification.

The algorithm proposed in this paper allows to automatically segment the optic disc from a fundus image. The goal is to facilitate the early detection of certain pathologies and to fully automate the process so as to avoid specialist intervention. The method proposed for the extraction of the optic disc contour is mainly based on mathematical morphology along with principal component analysis (PCA). It makes use of different operations such as generalized distance function (GDF), a variant of the watershed transformation, the stochastic watershed, and geodesic transformations. The input of the segmentation method is obtained through PCA. The purpose of using PCA is to achieve the grey-scale image that better represents the original RGB image. The implemented algorithm has been validated on five public databases obtaining promising results. The average values obtained (a Jaccard's and Dice's coefficients of 0.8200 and 0.8932, respectively, an accuracy of 0.9947, and a true positive and false positive fractions of 0.9275 and 0.0036) demonstrate that this method is a robust tool for the automatic segmentation of the optic disc. Moreover, it is fairly reliable since it works properly on databases with a large degree of variability and improves the results of other state-of-the-art methods.

A growing number of empirical studies evaluate the influence of Mental Health (MH) technology on the clinical effectiveness, the therapeutic relationship (i.e., therapeutic alliance), and usability issues. However, to the authors’ knowledge, no studies have yet been performed regarding the influence of technology on the therapeutic process in terms of collaboration. This study evaluates the quality of collaboration between the client and therapist in Augmented Reality Exposure Therapy (ARET) context and the traditional, In Vivo Exposure Therapy (IVET) context with the Therapeutic Collaborative Scale (TCS). Twenty participants received an intensive session of cognitive behavioral therapy in either a technology-mediated therapeutic context or in a traditional therapeutic context. The results indicate that both therapeutic conditions show high collaboration scores. However, the asymmetry of roles between the therapist and the client under both conditions were detected. Also, a greater level of distraction was observed for therapists in ARET, which affected the quality of the therapists’ involvement in the therapeutic session. The implications of these results are discussed.

This paper presents an algorithm for a 3D segmentation of the aorta artery in magnetic resonance images (MRI). The purpose is to project the 3D segmented aorta in the patient's abdomen with an augmented reality (AR) system to help the surgeon in laparoscopic interventions. In order to obtain accurate results in the segmentation process a marker-controlled watershed algorithm is used. Since this method requires a robust gradient image and two marker sets, a preprocessing step is carried out in each image. The algorithm is automatic and the results are promising with a Jaccard coefficient (JC) of 0.8107 ± 0.0228.

The presence of metal artifacts in dental CT prevents the correct exploration and planning of dental interventions. This paper addresses a new paradigm in metal artifact reduction that uses the backprojected data available in the DICOM files. The method, based on variational image registration and morphological lambda reconstruction, enhances the image quality using not only the information of the artifacted image (horizontal approach) but also the information of adjoining images (vertical approach). Some preliminary results involving different CT scanners and patients are presented and discussed.

Augmented Reality (AR) appears as a promising technology to improve students motivation and interest and support the learning and teaching process in educational contexts. We present the collaborative development of an AR application to support the teaching of the digestive and circulatory systems. We developed this system with the support of a private Spanish school. The main objective of the application is to show the student in primary school, in the most accurate way, digestive and circulatory systems. By other hand, we also develop our own AR library, HUMANAR, in order to ensure the integration of AR into our game engine and to overcome some drawbacks present in some public libraries. Moreover, our system provides several advantages over the traditional learning as books, videos or practice with animal organs.

Augmented reality provides solutions and benefits in many areas of knowledge. In the field of education, we can apply this technology for learning contents and for developing skills in an engaging way. Engineering educators are aware of the need for spatial vision skills in order to project and interpret drawings and plans. We propose an educational kit for use at university-level with contents based on engineering graphics. Preliminary results of a validation study with first year Mechanical Engineering students indicate that this augmented reality training has a positive impact on students' spatial ability and learning for basic engineering graphics contents.

Augmented reality has been used in many fields, but it has not been used to treat psychological disorders. Augmented reality presents several advantages respect to: the traditional treatment of psychological disorders and virtual reality treatments. In this paper we present the first augmented reality system for the treatment of phobia to cockroaches. Our system has been developed using ARToolkit software. It has been tested with one patient and the results have been very satisfactory. At first of the exposure session the patient was not able to approach to a real cockroach and after the exposure session using our augmented reality system, the patient was able to approach to a real cockroach, to interact with it and to kill it by herself. This first result is very encouraging and it demonstrates that augmented reality exposure is effective for the treatment of this kind of phobias.

The watershed transform has interesting properties that make it useful for many different image segmentation applications: it is simple and intuitive, can be parallelized, and always produces a complete division of the image. However, when applied to medical image analysis, it has important drawbacks (oversegmentation, sensitivity to noise, poor detection of thin or low signal to noise ratio structures). We present an improvement to the watershed transform that enables the introduction of prior information in its calculation. We propose to introduce this information via the use of a previous probability calculation. Furthermore, we introduce a method to combine the watershed transform and atlas registration, through the use of markers. We have applied our new algorithm to two challenging applications: knee cartilage and gray matter/white matter segmentation in MR images. Numerical validation of the results is provided, demonstrating the strength of the algorithm for medical image segmentation.

This paper presents an e-commerce tool that extends the conventional online store with a new section called room planner, a web application which is embedded in the virtual store. It allows the specification of the geometry of the room, placement of the objects and selection of the point of view. Then a realistic picture of the scene can be obtained. This functionality is very suitable for the furniture and ceramics sectors. In order to generate the images a parallel radiosity illumination algorithm has been implemented, which can be used in low-cost platforms such as a cluster of PCs, so that these technologies are affordable also for SMEs.

Flying phobia (FP) might become a very incapacitating and disturbing problem in a person's social, working, and private areas. Psychological interventions based on exposure therapy have proved to be effective, but given the particular nature of this disorder they bear important limitations. Exposure therapy for FP might be excessively costly in terms of time, money, and efforts. Virtual reality (VR) overcomes these difficulties as different significant environments might be created, where the patient can interact with what he or she fears while in a totally safe and protected environment-the therapist's consulting room. This paper intends, on one hand, to show the different scenarios designed by our team for the VR treatment of FP, and on the other, to present the first results supporting the effectiveness of this new tool for the treatment of FP in a multiple baseline study.