Childhood mental health problems are impairing, predictive of health problems later in life, and becoming increasingly prevalent. A critical first step toward addressing this growing crisis is facilitating more widespread screening; however, gold-standard assessments remain subjective, time-consuming, largely unable to capture subthreshold conditions and comorbidity, and limited by access to the clinical experts needed to provide and interpret the results. In response, researchers have developed digital phenotype screening tools that capture and utilize objective physiological and behavioral measures to augment traditional mental health screening. However, the efficacy of these tools has traditionally been evaluated on their ability to predict mental health diagnoses. In contrast, in this work and in line with the Research Domain Criteria (RDoC) framework, we explore the utility of these objective measures for quantifying trans diagnostic severity of impairment across a range of widely used clinical measurements. Using canonical correlation analysis, we find that linear combinations of movement and audio features extracted from smartphone sensor data collected during short (< 7 minutes) objective assessments are significantly correlated with a range of widely used clinical measurements. These findings suggest that easy-to-collect objective physiological and behavioral measures are indicative of severity across a range of psychopathologies.

As the mental health of college students continues to worsen, identifying students who need additional support and timely, targeted intervention is vital for college campuses. Leveraging Oura rings, a validated consumer wearable, we identify and examine heart rate variability (HRV) trajectories in 233 first-semester college students to characterize patterns of physiological well-being. The dominant trajectory among students was a decline in HRV (43.25%), followed by students who fluctuated but ultimately improved their HRV (24.46%), and students who remained consistent around their baseline, but experienced low points at the end of October and the end of the semester (19.3%). The least common HRV trajectory was a continuous improvement across the semester (12.9%). Findings suggest strong associations between HRV trajectories and experiential backgrounds, including first-generation status and total reported adverse life experiences, as well as weekly self-reported stress levels, highlighting the potential of wearables for providing real-time insight that could enable targeted intervention for student well-being.

The past several decades have seen significant advances in monitoring older adults' health and well-being. However, creating viable, practical monitoring systems for informing caregivers requires understanding which behaviors and signs to track and what approaches best present that information. To investigate how technology can be leveraged to better augment caregivers' workflows, we take a multi-stage, qualitative approach to gain insights into the needs of caregivers and the older adults receiving care. Specifically, we use a series of domain expert interviews, cognitive walkthroughs, and semi-structured interviews with residents, and we synthesize our takeaways using thematic analysis at each phase. Our results show that this type of monitoring technology has great potential to reduce the effort needed by caregivers to complete their responsibilities and communicate with their teams. Additionally, we found that older adults are receptive to the technology but their privacy and autonomy must be prioritized for the sake of their mental wellbeing. These insights will facilitate greater intelligent interface development for Person-Centered Care by identifying important design considerations and vital features that require system support.

Increased virtual meeting software usage has allowed people to meet remotely in a more seamless fashion. However, compared to in-person meetings, valuable interaction cues such as impromptu group polling are less optimally executed due to increased difficulty in gauging remote participants, while also requiring prior meeting setup for automated counting with built-in polling tools. We propose a novel intelligent user interface approach for virtual meeting software that supports impromptu polling interactions by leveraging real-time hand gesture recognition and video filter feedback. We conducted studies to design and evaluate this intuitive gesture-based polling system with visual feedback. Our results demonstrated that our system was able to recognize attendees’ gestures and poll responses with reasonable accuracy, and showed improvements in hosts’ task workload performance. From our findings, our interface informs hosts of valuable results while maintaining organic gestural interaction cues with attendees similar to in-person meetings.

Nearly half of people prescribed medication to treat chronic or short-term conditions do not take their medicine as prescribed. This leads to worse treatment outcomes, higher hospital admission rates, increased healthcare costs, and increased morbidity and mortality rates. While some instances of medication non-adherence are a result of problems with the treatment plan or barriers caused by the health care provider, many are instances caused by patient-related factors such as forgetting, running out of medication, and not understanding the required dosages. This presents a clear need for patient-centered systems that can reliably increase medication adherence. To that end, in this work we describe an activity recognition system capable of recognizing when individuals take medication in an unconstrained, real-world environment. Our methodology uses a modified version of the Bagging ensemble method to suit unbalanced data and a classifier trained on the prediction probabilities of the Bagging classifier to identify when individuals took medication during a full-day study. Using this methodology we are able to recognize when individuals took medication with an F-measure of 0.77. Our system is a first step towards developing personal health interfaces that are capable of providing personalized medication adherence interventions.

Recent trends in computer-mediated communication (CMC) have not only led to expanded instant messaging through the use of images and videos but have also expanded traditional text messaging with richer content in the form of visual communication markers (VCMs) such as emoticons, emojis, and stickers. VCMs could prevent a potential loss of subtle emotional conversation in CMC, which is delivered by nonverbal cues that convey affective and emotional information. However, as the number of VCMs grows in the selection set, the problem of VCM entry needs to be addressed. Furthermore, conventional means of accessing VCMs continue to rely on input entry methods that are not directly and intimately tied to expressive nonverbal cues. In this work, we aim to address this issue by facilitating the use of an alternative form of VCM entry: hand gestures. To that end, we propose a user-defined hand gesture set that is highly representative of a number of VCMs and a two-stage hand gesture recognition system (trajectory-based, shape-based) that can identify these user-defined hand gestures with an accuracy of 82%. By developing such a system, we aim to allow people using low-bandwidth forms of CMCs to still enjoy their convenient and discreet properties while also allowing them to experience more of the intimacy and expressiveness of higher-bandwidth online communication.

The number of individuals living with chronic conditions continues to rise. As a result, a significant emphasis has been placed on both improving their quality of life as well as decreasing the cost and burden of caring for them. One particularly promising avenue for achieving this is the use of wearable devices, as they have become both affordable and reliable in recognizing fitness activities. However, while the existing algorithms reliably recognize physically intensive activities (e.g., walking vs. swimming), they fail to recognize personal hygiene actives that have more subtle differences (e.g., brushing teeth vs. washing hands). This research aims to develop novel features and intelligent, multi-stage algorithms that can reliably recognize such personal hygiene activities for chronic care. Additionally, we aim to further supplement this activity recognition with personalized interventions that enable individuals to manage their own personal health.

Design sketching is an important skill for designers, engineers, and creative professionals, as it allows them to express their ideas and concepts in a visual medium. Being a critical and versatile skill for many different disciplines, courses on design sketching are often taught in universities. Courses today predominately rely on pen and paper; however, this traditional pedagogy is limited by the availability of human instructors, who can provide personalized feedback. Using a stylus-based intelligent tutoring system called SketchTivity , we aim to eventually mimic the feedback given by an instructor and assess student-drawn sketches to give students insight into areas for improvement. To provide effective feedback to users, it is important to identify what aspects of their sketches they should work on to improve their sketching ability. After consulting with several domain experts in sketching, we came up with several classes of features that could potentially differentiate expert and novice sketches. Because improvement on one metric, such as speed, may result in a decrease in another metric, such as accuracy, the creation of a single score may not mean much to the user. We attempted to create a single internal score that represents overall drawing skill so that the system can track improvement over time and found that this score correlates highly with expert rankings. We gathered over 2,000 sketches from 20 novices and four experts for analysis. We identified key metrics for quality assessment that were shown to significantly correlate with the quality of expert sketches and provide insight into providing intelligent user feedback in the future.