AI literacy initiatives in K-12 education frequently emphasize the importance of AI ethics. However, many existing AI ethics curricula offer limited opportunities for students to examine ethical dilemmas or practice ethical decision-making with real-world scenarios. Prior work suggests that games can support ethics education by providing low-stakes, playful environments for exploration and reflection. In this paper, we present Orbiting the VirtueVerse, a scenario-based ethics game designed to introduce middle-school students to virtue ethics in the context of artificial intelligence. We report findings from a research workshop with 22 students aged 13–14. Participants learned about virtue ethics, played the game, and co-designed new scenarios for a future iteration of the game. Our findings suggest that game-based ethical scenarios are a promising approach for supporting students’ ethical understanding of AI. We conclude by offering design guidelines for effectively integrating ethical scenarios into educational games for AI literacy.

Immigrant-origin adolescents frequently encounter identity-related stressors that undermine school belonging, yet existing identity interventions require intensive facilitation that under-resourced schools cannot sustain. This study asks what challenges shape ethnic-racial identity (ERI) support under institutional constraints, and what design features a school-ready digital intervention should provide in response. We conducted six participatory workshops with 20 K-12 school counselors via a statewide association, using reflexive thematic analysis to identify three dimensions of ERI challenge-interpersonal, internal, and structural/contextual-and a consistent gap between what evidence-based interventions require and what schools can provide. These findings directly informed SelfQuest, a nine-chapter gamified identity-support prototype. This WiP contributes a practitioner-derived challenge codebook and a traceable derivation from counselor insights to prototype features. Classroom evaluation with immigrant-origin adolescents is the next step.

Artificial intelligence (AI) is increasingly present in children’s lives, yet few tools support developmentally appropriate AI literacy for grades K-3. This work examines the role of narrative in early AI literacy by directly comparing two versions of interactive game-based digital storybooks for children ages 6-9. The “Book+” condition combined an overarching story and characters with mini-games and scaffolded AI interactions, designed to be enjoyable, provide narrative context, and to give hands-on AI experience. We compared this with a “Game” condition that included the same learning goals, mini-games, and AI interactions but replaced the narrative with primarily instructional text. Across 57 participants, both conditions elicited high engagement, but “Book+” participants showed significantly greater learning gains and higher perceived knowledge. Qualitative findings revealed that while both groups enjoyed the creative AI mini-games, “Book+” participants more frequently used AI vocabulary in responses, connected concepts to the learning context, and expressed stronger emotional connection.

Children regularly negotiate questions of authority and control in home and school life, but little is known about how they believe robots should fit into these dynamics. We conducted a 75-minute design session with 17 children (ages 6-9) to examine when robots should take, share, or defer control, and how expectations shift when robots are framed as teachers, classmates, or mentees. Children resisted robot control, particularly in adult-regulated domains and areas tied to personal skill or self-expression. They were more open to robot control in domains where they felt less competent, or where robots, perceived as less legitimate authorities than humans, could substitute for adult control. Role framing further shaped expectations: teacher robots were granted autonomy, classmate robots were expected to act as peers, and mentee robots were expected to defer. These findings show that children apply context- and role-sensitive rules when negotiating control with robots. We conclude with design considerations for robots in children's everyday lives that respect children's agency, calibrate autonomy by domain, and align behavior with children's context-sensitive expectations.

Despite living in an increasingly connected world, social isolation is a prevalent issue today. While social robots have been explored as tools to enhance social connection through companionship, their potential as asynchronous social platforms for fostering connection towards humanity has received less attention. In this work, we introduce the design of a social support companion that facilitates the exchange of emotionally relevant stories and scaffolds reflection to enhance feelings of connection via five design dimensions. We investigate how social robots can serve as “social proxies” facilitating human stories, passing stories from other human narrators to the user. To this end, we conduct a real-world deployment of 40 robot stations in users' homes over the course of two weeks. Through thematic analysis of user interviews, we find that social proxy robots can foster connection towards other people's experiences via mechanisms such as identifying connections across stories or offering diverse perspectives. We present design guidelines from our study insights on the use of social robot systems that serve as social platforms to enhance human empathy and connection.

This paper reports on the creation and composition of a new corpus of children's speech, the Ohio Child Speech Corpus, which is publicly available on the Talkbank-CHILDES website. The audio corpus contains speech samples from 303 children ranging in age from 4 – 9 years old, all of whom participated in a seven-task elicitation protocol conducted in a science museum lab. In addition, an interactive social robot controlled by the researchers joined the sessions for approximately 60% of the children, and the corpus itself was collected in the peri‑pandemic period. Two analyses are reported that highlighted these last two features. One set of analyses found that the children spoke significantly more in the presence of the robot relative to its absence, but no effects of speech complexity (as measured by MLU) were found for the robot's presence. Another set of analyses compared children tested immediately post-pandemic to children tested a year later on two school-readiness tasks, an Alphabet task and a Reading Passages task. This analysis showed no negative impact on these tasks for our highly-educated sample of children just coming off of the pandemic relative to those tested later. These analyses demonstrate just two possible types of questions that this corpus could be used to investigate.

Artificial Intelligence (AI) is permeating almost every area of society, reshaping how many people, including youth, navigate the world. Despite the increased presence of AI, most people lack a baseline knowledge of how AI works. Moreover, social barriers often hinder equal access to AI courses, perpetuating disparities in participation in the field. To address this, it is crucial to design AI curricula that are effective, inclusive, and relevant, especially to learners from backgrounds that are historically excluded from working in tech. In this paper, we present AI for Wellbeing, a curriculum where students explore conversational AI and the ethical considerations around using it to promote wellbeing. We specifically designed content, educator materials, and educational technologies to meet the interests and needs of students and educators from diverse backgrounds. We piloted AI for Wellbeing in a 5-day virtual workshop with middle school teachers and students. Then, using a mixed-methods approach, we analyzed students' work and teachers' feedback. Our results suggest that the curriculum content and design effectively engaged students, enabling them to implement meaningful AI projects for wellbeing. We hope that the design of this curriculum and insights from our evaluation will inspire future efforts to create culturally relevant K-12 AI curricula.

In this research-to-practice full paper, we present the third iteration of our educational framework that advances AI-informed leadership - a much-needed competency in this era of rapid AI transformation. Our study aimed to evaluate our proposed content and pedagogy and whether it can be made widely accessible to non-technical leaders. We focused on modifying our existing curriculum and research protocol based on the feedback and learnings from our previous workshops. Our current workshop is part of an AI-Education program developed by MIT aiming to provide AI education to the U.S Air Force, which is one of the largest organizations in the US. Previous two iterations of the workshop aimed to offer an introduction to AI to U.S Air Force leaders, while the current iteration's goals are twofold: offering the updated content to U.S Air Force leaders, and supporting their capacity building by preparing a cohort of workshop trainers, who will lead the future iterations of the program inhouse. We conducted the workshop with over 50 participants, including 10 facilitators who were trained to run future workshops internally. Our comprehensive educational materials, including fa-cilitator guides, student-facing documents, and digital resources, support this scalable model. The results demonstrate significant improvements in AI knowledge and leadership competencies, including AI mindset, culture, and ethos. Our findings affirm the effectiveness of experiential learning methodologies and underscore the potential for scalable, sustainable AI education within large organizations through the facilitators' feedback.

In this paper, we introduce a novel conceptual model for a robot's behavioral adaptation in its long-term interaction with humans, integrating dynamic robot role adaptation with principles of flow experience from psychology. This conceptualization introduces a hierarchical interaction objective grounded in the flow experience, serving as the overarching adaptation goal for the robot. This objective intertwines both cognitive and affective sub-objectives and incorporates individual and group-level human factors. The dynamic role adaptation approach is a cornerstone of our model, highlighting the robot's ability to fluidly adapt its support roles-from leader to follower-with the aim of maintaining equilibrium between activity challenge and user skill, thereby fostering the user's optimal flow experiences. Moreover, this work delves into a comprehensive exploration of the limitations and potential applications of our proposed conceptualization. Our model places a particular emphasis on the multi-person HRI paradigm, a dimension of HRI that is both under-explored and challenging. In doing so, we aspire to extend the applicability and relevance of our conceptualization within the HRI field, contributing to the future development of adaptive social robots capable of sustaining long-term interactions with humans.

Today, Artificial Intelligence (AI) is prevalent in everyday life, with emerging technologies like AI companions, autonomous vehicles, and AI art tools poised to significantly transform the future. The development of AI curricula that shows people how AI works and what they can do with it is a powerful way to prepare everyone, and especially young learners, for an increasingly AI-driven world. Educators often employ robotic toolkits in the classroom to boost engagement and learning. However, these platforms are generally unsuitable for young learners and learners without programming expertise. Moreover, these platforms often serve as either programmable artifacts or pedagogical agents, rarely capitalizing on the opportunity to support students in both capacities. We designed Doodlebot, a mobile social robot for hands-on AI education to address these gaps. Doodlebot is an effective tool for exploring AI with grade school (K-12) students, promoting their understanding of AI concepts such as perception, representation, reasoning and generation. We begin by elaborating Doodlebot's design, highlighting its reliability, user-friendliness, and versatility. Then, we demonstrate Doodlebot's versatility through example curricula about AI character design, autonomous robotics, and generative AI accessible to young learners. Finally, we share the results of a preliminary user study with elementary school youth where we found that the physical Doodlebot platform was as effective and user-friendly as the virtual version. This work offers insights into designing interactive educational robots that can inform future AI curricula and tools.

Social-emotional learning (SEL) skills are essential for children to develop to provide a foundation for future relational and academic success. Using art as a medium for creation or as a topic to provoke conversation is a well-known method of SEL learning. Similarly, social robots have been used to teach SEL competencies like empathy, but the combination of art and social robotics has been minimally explored. In this paper, we present a novel child-robot interaction designed to foster empathy and promote SEL competencies via a conversation about art scaffolded by a social robot. Participants (N=11, age range: 7-11) conversed with a social robot about emotional and neutral art. Analysis of video and speech data demonstrated that this interaction design successfully engaged children in the practice of SEL skills, like emotion recognition and self-awareness, and greater rates of empathetic reasoning were observed when children engaged with the robot about emotional art. This study demonstrated that art-based reflection with a social robot, particularly on emotional art, can foster empathy in children, and interactions with a social robot help alleviate discomfort when sharing deep or vulnerable emotions.

The Feature Detection tool is a web-based activity that allows students to detect features in images and build their own rule-based classification algorithms. In this paper, we introduce the tool and share how it is incorporated into two, 45-minute lessons. The objective of the first lesson is to introduce students to the concept of feature detection, or how a computer can break down visual input into lower-level features. The second lesson aims to show students how these lower-level features can be incorporated into rule-based models to classify higher-order objects. We discuss how this tool can be used as a "first step" to the more complex concept ideas of data representation and neural networks.

As artificial intelligence (AI) becomes a prominent part of modern life, AI literacy is becoming important for all citizens, not just those in technology careers. Previous research in AI education materials has largely focused on the introduction of terminology as well as AI use cases and ethics, but few allow students to learn by creating their own machine learning models. Therefore, there is a need for enriching AI educational tools with more adaptable and flexible platforms for interested educators with any level of technical experience to utilize within their teaching material. As such, we propose the development of an open-source tool (Build-A-Bot) for students and teachers to not only create their own transformer-based chatbots based on their own course material but also learn the fundamentals of AI through the model creation process. The primary concern of this paper is the creation of an interface for students to learn the principles of artificial intelligence by using a natural language pipeline to train a customized model to answer questions based on their own school curriculums. The model uses contexts given by their instructor, such as chapters of a textbook, to answer questions and is deployed on an interactive chatbot/voice agent. The pipeline teaches students data collection, data augmentation, intent recognition, and question answering by having them work through each of these processes while creating their AI agent, diverging from previous chatbot work where students and teachers use the bots as black-boxes with no abilities for customization or the bots lack AI capabilities, with the majority of dialogue scripts being rule-based. In addition, our tool is designed to make each step of this pipeline intuitive for students at a middle-school level. Further work primarily lies in providing our tool to schools and seeking student and teacher evaluations.

Given the widespread prevalence of depression and its consequential impact on individuals and society, it is crucial to obtain objective measures for early diagnosis and intervention. As a multidisciplinary topic, these objective measures should be interpretable and accessible to health care professionals, ensuring effective collaboration and treatment planning in the realm of mental health care. Even though current automated depression diagnosis approaches improved over the last decade, a critical gap exists as they often lack affect-specificity and interpretability, limiting their practical application and potential impact on mental health care. In particular, interpretability from temporal activities from videos when deep models are used is not fully explored. In this study, we present a novel framework for analyzing Deep Neural Networks’ decisions when trained on facial videos, specifically focusing on automatic depression severity diagnosis. By fine-tuning Deep Convolutional Neural Networks (DCNN) pre-trained on Action Recognition datasets on depression severity facial videos from AVEC depression dataset, our framework is able to interpret the model’s saliency maps by examining face regions and temporal expression semantics. Our approach generates both visual and quantitative explanations for the model’s decisions, providing greater insight into its reasoning. In addition to this interpretability, our video-based modeling has improved upon previous single-face benchmarks for visual depression diagnosis, resulting in enhanced predictive performance. Overall, our work demonstrates the successful development of a framework capable of generating hypotheses from a facial model’s decisions while simultaneously improving depression’s predictive capabilities.

Escape rooms have become increasingly popular as a form of entertainment, in addition to being adopted by educators for their effectiveness in improving student engagement and learning. While they have been introduced in various educational contexts, from nursing to mathematics, and for different age groups, including K-12 and university students, little research has been conducted on the benefits of escape rooms for adult learning of artificial intelligence (AI). Furthermore, most escape room implementations lack relevance to real-world situations and challenges with using AI systems in the wild. This study explores the effectiveness of an escape room, DRONEscape, as a tool for teaching AI concepts to Air Force participants. The results suggest that escape rooms can most effectively facilitate engagement and collaboration, and have positive effects on learning AI concepts. This paper also provides considerations for improvements to future iterations of AI-themed escape rooms to enhance learning, collaboration, engagement, and enjoyment.

This research to practice full paper explores a new educational framework for AI-informed leadership and evaluates its curriculum and pedagogical approach through a novel, tailored, research instrument. Artificial Intelligence continues to rapidly transform many aspects of markets, solutions, and organizational culture across companies, agencies, and institutions in the public and private sectors. Within complex organizations, AI tools, technologies, and applications inform how leaders engage in strategy-making, management, operations, human resources, and professional education. Non-technical managers and executives are increasingly expected to lead teams to implement responsible AI solutions with the promise to improve efficiency, effectiveness, productivity, profitability, and more. AI is rapidly transforming organizational culture, requiring non-technical leaders to develop AI literacy and essential skills to lead teams in implementing responsible AI solutions. In the face of AI-driven change, business leaders need to be AI literate and develop their own essential skills, knowledge, procedures, and perspectives to successfully set vision and strategy to lead teams that can leverage AI to achieve inward-facing and outward-facing business goals. This presents challenges and opportunities to develop new pedagogical approaches and measures to prepare and assess business leaders' AI leadership skills - including understanding human-AI systems in the workplace and their responsible development and ethical use. There are also cultural and organizational behavior challenges in successfully adopting these new capabilities into a global and diverse human-AI workforce at scale. To advance these, we present an innovative hands-on AI leadership curriculum, where participants learn by making and team problem-solving, for United States Air Force (USAF) leaders to learn about AI and its responsible use in human-robot teaming with autonomous robots. We contribute new measures to assess their attitudinal shifts in AI leadership with respect to culture, mindsets, and ethics. We present a pilot study to evaluate our curriculum design and pedagogical approach to foster positive shifts in our AI leadership measures.

Social support plays a crucial role in managing and enhancing one's mental health and well-being. In order to explore the role of a robot's companion-like behavior on its therapeutic interventions, we conducted an eight-week-long deployment study with seventy participants to compare the impact of (1) acontrol robot with only assistant-like skills, (2) acoach-like robot with additional instructive positive psychology interventions, and (3) acompanion-like robot that delivered the same interventions in a peer-like and supportive manner. The companion-like robot was shown to be the most effective in building a positive therapeutic alliance with people, enhancing participants' well-being and readiness for change. Our work offers valuable insights into how companion AI agents could further enhance the efficacy of the mental health interventions by strengthening their therapeutic alliance with people for long-term mental health support.

Given the prevalence of depression worldwide and its major impact on society, several studies employed artificial intelligence modelling to automatically detect and assess depression. However, interpretation of these models and cues are rarely discussed in detail in the AI community, but have received increased attention lately. In this study, we aim to analyse the commonly selected features using a proposed framework of several feature selection methods and their effect on the classification results, which will provide an interpretation of the depression detection model. The developed framework aggregates and selects the most promising features for modelling depression detection from 38 feature selection algorithms of different categories. Using three real-world depression datasets, 902 behavioural cues were extracted from speech behaviour, speech prosody, eye movement and head pose. To verify the generalisability of the proposed framework, we applied the entire process to depression datasets individually and when combined. The results from the proposed framework showed that speech behaviour features (e.g. pauses) are the most distinctive features of the depression detection model. From the speech prosody modality, the strongest feature groups were F0, HNR, formants, and MFCC, while for the eye activity modality they were left-right eye movement and gaze direction, and for the head modality it was yaw head movement. Modelling depression detection using the selected features (even though there are only 9 features) outperformed using all features in all the individual and combined datasets. Our feature selection framework did not only provide an interpretation of the model, but was also able to produce a higher accuracy of depression detection with a small number of features in varied datasets. This could help to reduce the processing time needed to extract features and creating the model.

High-quality parent-child conversational interactions are crucial for children's social, emotional, and cognitive development. However, many children have limited exposure to these interactions at home. As increasingly accessible and scalable interventions in child development, interactive technologies, such as social robots, have great potential for facilitating parent-child interactions. However, such technology-based interventions are still underexplored, as the technologies' limited ability to understand the social-emotional dynamics of human dyadic interactions impedes their effective delivery of timely, adaptive interventions. To advance research on resolving this roadblock, we present a “dyadic affect in multimodal interaction-parent to child” (DAMI-P2C) dataset collected during a study of 34 parent-child pairs, where parents and children (3-7 years old) engaged in reading storybooks together. In contrast to existing public datasets for social-emotional behaviors in dyadic interactions, each instance for both participants in our dataset was annotated for affect by three labelers. Additionally, the dataset contains audiovisual recordings as well as each dyad's sociodemographic profiles, co-reading behaviors, affect labels, and body joints. We describe the dataset's main characteristics and provide a preliminary analysis of the interrelations between sociodemographic profiles, co-reading behaviors, and affect labels. The dataset provides us with useful insights into the computing and social science fields.

Abstract Personalized learning has a higher impact on students’ progress than traditional approaches. However, current resources required to implement personalization are scarce. This research aims to conceptualize and develop an autonomous robot tutor with personalization policy for preschool children aged between three to five years old. Personalization is performed by automatically adjusting the difficulty level of the lesson delivery and assessment, as well as adjusting the feedback based on the reaction of children. This study explores three child behaviors for the personalization policy: (i) academic knowledge (measured by the correctness of the answer), (ii) executive functioning of attention (measured by the orientation and the gaze direction of child’s body), and (iii) working memory or hesitation (measured by the time lag before the answer). Moreover, this study designed lesson content through interviews with teachers and deployed the personalization interaction policy through the NAO robot with five children in a case user study method. We qualitatively analyze the session observations and parent interviews, as well as quantitatively analyze knowledge gain through pre- and posttests and a parent questionnaire. The findings of the study reveal that the personalized interaction with the robot showed a positive potential in increasing the children’s learning gains and attracting their engagement. As general guidelines based on this pilot study, we identified additional personalization strategies that could be used for autonomous personalization policies based on each child’s behavior, which could have a considerable impact on child learning.

Despite the increase in awareness and support for mental health, college students' mental health is reported to decline every year in many countries. Several interactive technologies for mental health have been proposed and are aiming to make therapeutic service more accessible, but most of them only provide one-way passive contents for their users, such as psycho-education, health monitoring, and clinical assessment. We present a robotic coach that not only delivers interactive positive psychology interventions but also provides other useful skills to build rapport with college students. Results from our on-campus housing deployment feasibility study showed that the robotic intervention showed significant association with increases in students' psychological well-being, mood, and motivation to change. We further found that students' personality traits were associated with the intervention outcomes as well as their working alliance with the robot and their satisfaction with the interventions. Also, students' working alliance with the robot was shown to be associated with their pre-to-post change in motivation for better well-being. Analyses on students' behavioral cues showed that several verbal and nonverbal behaviors were associated with the change in self-reported intervention outcomes. The qualitative analyses on the post-study interview suggest that the robotic coach's companionship made a positive impression on students, but also revealed areas for improvement in the design of the robotic coach. Results from our feasibility study give insight into how learning users' traits and recognizing behavioral cues can help an AI agent provide personalized intervention experiences for better mental health outcomes.

Parent-child nonverbal communication plays a crucial role in understanding their relationships and assessing their interaction styles. However, prior works have seldom studied the exchange of these nonverbal cues between the dyad and focused on isolated cues from one person at a time. In contrast, this work analyzes both parents' and children's individual and dyadic nonverbal behaviors in relation to their four relationship characteristics, i.e., child temperament, parenting style, parenting stress, and home literacy environment. We utilize a state-of-the-art feature selection framework on a dataset of 31 parent-child interactions to automatically extract and select a set of temporal nonverbal behaviors as key indicators of the dyad's relationship characteristics. The results show that relationship characteristics were associated with both individuals' and dyads' nonverbal behaviors. This finding highlights the importance of accounting for both individual- and dyad-scale nonverbal behaviors when predicting dyadic relationship characteristics as well as the potential limitations of utilizing single persons' nonverbal data in isolation. It therefore motivates future work on this topic to take a holistic and relational approach. The dataset and extracted nonverbal data are made public to aid the development of automated detection tools for parent-child relationship characteristics that trains on visual recordings of their dyadic interactions.

Self-disclosure is an important part of mental health treatment process. As interactive technologies are becoming more widely available, many AI agents for mental health prompt their users to self-disclose as part of the intervention activities. However, most existing works focus on linguistic features to classify self-disclosure behavior, and do not utilize other multi-modal behavioral cues. We present analyses of people's non-verbal cues (vocal acoustic features, head orientation and body gestures/movements) exhibited during self-disclosure tasks based on the human-robot interaction data collected in our previous work. Results from the classification experiments suggest that prosody, head pose, and body postures can be independently used to detect self-disclosure behavior with high accuracy (up to 81%). Moreover, positive emotions, high engagement, self-soothing and positive attitudes behavioral cues were found to be positively correlated to self-disclosure. Insights from our work can help build a self-disclosure detection model that can be used in real time during multi-modal interactions between humans and AI agents.

Intent recognition models, which match a written or spoken input's class in order to guide an interaction, are an essential part of modern voice user interfaces, chatbots, and social robots. However, getting enough data to train these models can be very expensive and challenging, especially when designing novel applications such as real-world human-robot interactions. In this work, we first investigate how much training data is needed for high performance in an intent classification task. We train and evaluate BiLSTM and BERT models on various subsets of the ATIS and Snips datasets. We find that only 25 training examples per intent are required for our BERT model to achieve 94% intent accuracy compared to 98% with the entire datasets, challenging the belief that large amounts of labeled data are required for high performance in intent recognition. We apply this knowledge to train models for a real-world HRI application, character strength recognition during a positive psychology interaction with a social robot, and evaluate against the Character Strength dataset collected in our previous HRI study. Our real-world HRI application results also confirm that our model can produce 76% intent accuracy with 25 examples per intent compared to 80% with 100 examples. In a real-world scenario, the difference is only one additional error per 25 classifications. Finally, we investigate the limitations of our minimal data models and offer suggestions on developing high quality datasets. We conclude with practical guidelines for training BERT intent recognition models with minimal training data and make our code and evaluation framework available for others to replicate our results and easily develop models for their own applications.

A significant number of college students suffer from mental health issues that impact their physical, social, and occupational outcomes. Various scalable technologies have been proposed in order to mitigate the negative impact of mental health disorders. However, the evaluation for these technologies, if done at all, often reports mixed results on improving users' mental health. We need to better understand the factors that align a user's attributes and needs with technology-based interventions for positive outcomes. In psychotherapy theory, therapeutic alliance and rapport between a therapist and a client is regarded as the basis for therapeutic success. In prior works, social robots have shown the potential to build rapport and a working alliance with users in various settings. In this work, we explore the use of a social robot coach to deliver positive psychology interventions to college students living in on-campus dormitories. We recruited 35 college students to participate in our study and deployed a social robot coach in their room. The robot delivered daily positive psychology sessions among other useful skills like delivering the weather forecast, scheduling reminders, etc. We found a statistically significant improvement in participants' psychological wellbeing, mood, and readiness to change behavior for improved wellbeing after they completed the study. Furthermore, students' personality traits were found to have a significant association with intervention efficacy. Analysis of the post-study interview revealed students' appreciation of the robot's companionship and their concerns for privacy.

An estimated 350 million people worldwide are affected by depression. Using affective sensing technology, our long-term goal is to develop an objective multimodal system that augments clinical opinion during the diagnosis and monitoring of clinical depression. This paper steps towards developing a classification system-oriented approach, where feature selection, classification and fusion-based experiments are conducted to infer which types of behaviour (verbal and nonverbal) and behaviour combinations can best discriminate between depression and non-depression. Using statistical features extracted from speaking behaviour, eye activity, and head pose, we characterise the behaviour associated with major depression and examine the performance of the classification of individual modalities and when fused. Using a real-world, clinically validated dataset of 30 severely depressed patients and 30 healthy control subjects, a Support Vector Machine is used for classification with several feature selection techniques. Given the statistical nature of the extracted features, feature selection based on T-tests performed better than other methods. Individual modality classification results were considerably higher than chance level (83 percent for speech, 73 percent for eye, and 63 percent for head). Fusing all modalities shows a remarkable improvement compared to unimodal systems, which demonstrates the complementary nature of the modalities. Among the different fusion approaches used here, feature fusion performed best with up to 88 percent average accuracy. We believe that is due to the compatible nature of the extracted statistical features.

No studies have investigated cross-cultural and cross-language characteristics of depressed speech. We investigated the generalisability of a vocal biomarker-based approach to depression detection in clinical interviews recorded in three countries (Australia, the USA and Germany), two languages (German and English) and different accents (Australian and American). Several approaches to training and testing within and between datasets were evaluated. Using the same experimental protocol separately within each dataset, (cross-classification) accuracy was high.combining datasets, high accuracy was high again and consistent across language, recording environment, and culture. Training and testing between datasets, however, attenuated accuracy. These finding emphasize the importance of heterogeneous training sets for robust depression detection.

Clinical depression is a critical public health problem, with high costs associated to a person's functioning, mortality, and social relationships, as well as the economy overall. Currently, there is no dedicated objective method to diagnose depression. Rather, its diagnosis depends on patient self-report and the clinician's observation, risking a range of subjective biases. Our aim is to develop an objective affective sensing system that supports clinicians in their diagnosis and monitoring of clinical depression. In this PhD work, my approach is based on multimodal analysis, i.e. combinations of vocal affect, head pose and eye movement from a video-audio real-world clinically validated data. In addition, this work will investigate the cross-cultural generalization of depression characteristics from different languages and countries.

Depression is a common and disabling mental health disorder, which impacts not only on the sufferer but also their families, friends and the economy overall. Our ultimate aim is to develop an automatic objective affective sensing system that supports clinicians in their diagnosis and monitoring of clinical depression. Here, we analyse the performance of head pose and movement features extracted from face videos using a 3D face model projected on a 2D Active Appearance Model (AAM). In a binary classification task (depressed vs. non-depressed), we modelled low-level and statistical functional features for an SVM classifier using real-world clinically validated data. Although the head pose and movement would be used as a complementary cue in detecting depression in practice, their recognition rate was impressive on its own, giving 71.2% on average, which illustrates that head pose and movement hold effective cues in diagnosing depression. When expressing positive and negative emotions, recognising depression using positive emotions was more accurate than using negative emotions. We conclude that positive emotions are expressed less in depressed subjects at all times, and that negative emotions have less discriminatory power than positive emotions in detecting depression. Analysing the functional features statistically illustrates several behaviour patterns for depressed subjects: (1) slower head movements, (2) less change of head position, (3) longer duration of looking to the right, (4) longer duration of looking down, which may indicate fatigue and eye contact avoidance. We conclude that head movements are significantly different between depressed patients and healthy subjects, and could be used as a complementary cue.

Major depressive disorders are mental disorders of high prevalence, leading to a high impact on individuals, their families, society and the economy. In order to assist clinicians to better diagnose depression, we investigate an objective diagnostic aid using affective sensing technology with a focus on acoustic features. In this paper, we hypothesise that (1) classifying the general characteristics of clinical depression using spontaneous speech will give better results than using read speech, (2) that there are some acoustic features that are robust and would give good classification results in both spontaneous and read, and (3) that a `thin-slicing' approach using smaller parts of the speech data will perform similarly if not better than using the whole speech data. By examining and comparing recognition results for acoustic features on a real-world clinical dataset of 30 depressed and 30 control subjects using SVM for classification and a leave-one-out cross-validation scheme, we found that spontaneous speech has more variability, which increases the recognition rate of depression. We also found that jitter, shimmer, energy and loudness feature groups are robust in characterising both read and spontaneous depressive speech. Remarkably, thin-slicing the read speech, using either the beginning of each sentence or the first few sentences performs better than using all reading task data.

Accurate detection of depression from spontaneous speech could lead to an objective diagnostic aid to assist clinicians to better diagnose depression. Little thought has been given so far to which classifier performs best for this task. In this study, using a 60-subject real-world clinically validated dataset, we compare three popular classifiers from the affective computing literature - Gaussian Mixture Models (GMM), Support Vector Machines (SVM) and Multilayer Perceptron neural networks (MLP) - as well as the recently proposed Hierarchical Fuzzy Signature (HFS) classifier. Among these, a hybrid classifier using GMM models and SVM gave the best overall classification results. Comparing feature, score, and decision fusion, score fusion performed better for GMM, HFS and MLP, while decision fusion worked best for SVM (both for raw data and GMM models). Feature fusion performed worse than other fusion methods in this study. We found that loudness, root mean square, and intensity were the voice features that performed best to detect depression in this dataset.

Depression is a common and disabling mental health disorder, which impacts not only on the sufferer but also on their families, friends and the economy overall. Despite its high prevalence, current diagnosis relies almost exclusively on patient self-report and clinical opinion, leading to a number of subjective biases. Our aim is to develop an objective affective sensing system that supports clinicians in their diagnosis and monitoring of clinical depression. In this paper, we analyse the performance of eye movement features extracted from face videos using Active Appearance Models for a binary classification task (depressed vs. non-depressed). We find that eye movement low-level features gave 70% accuracy using a hybrid classifier of Gaussian Mixture Models and Support Vector Machines, and 75% accuracy when using statistical measures with SVM classifiers over the entire interview. We also investigate differences while expressing positive and negative emotions, as well as the classification performance in gender-dependent versus gender-independent modes. Interestingly, even though the blinking rate was not significantly different between depressed and healthy controls, we find that the average distance between the eyelids (`eye opening') was significantly smaller and the average duration of blinks significantly longer in depressed subjects, which might be an indication of fatigue or eye contact avoidance.

Depression is a serious psychiatric disorder that affects mood, thoughts, and the ability to function in everyday life. This pa-per investigates the characteristics of depressed speech for the purpose of automatic classification by analysing the effect of different speech features on the classification results. We anal-ysed voiced, unvoiced and mixed speech in order to gain a better understanding of depressed speech and to bridge the gap be-tween physiological and affective computing studies. This un-derstanding may ultimately lead to an objective affective sens-ing system that supports clinicians in their diagnosis and mon-itoring of clinical depression. The characteristics of depressed speech were statistically analysed using ANOVA and linked to their classification results using GMM and SVM. Features were extracted and classified over speech utterances of 30 clinically depressed patients against 30 controls (both gender-matched) in a speaker-independent manner. Most feature classification re-sults were consistent with their statistical characteristics, pro-viding a link between physiological and affective computing studies. The classification results from low-level features were slightly better than the statistical functional features, which in-dicates a loss of information in the latter. We found that both mixed and unvoiced speech were as useful in detecting depres-sion as voiced speech, if not better. Index Terms: depression, speech characteristics, mood classi-fication