Emotions, shaped by past experiences, significantly influence decision-making and goal pursuit. Traditional cognitive-behavioral techniques for personal development rely on mental imagery to envision ideal selves, but may be less effective for individuals who struggle with visualization. This paper introduces Emotional Self-Voice (ESV), a novel system combining emotionally expressive language models and voice cloning technologies to render customized responses in the user’s own voice. We investigate the potential of ESV to nudge individuals towards their ideal selves in a study with 60 participants. Across all three conditions (ESV, text-only, and mental imagination), we observed an increase in resilience, confidence, motivation, and goal commitment, and the ESV condition was perceived as uniquely engaging and personalized. We discuss the implications of designing generated self-voice systems as a personalized behavioral intervention for different scenarios.

Older adults have increasing difficulty with retrospective memory, hindering their abilities to perform daily activities and posing stress on caregivers to ensure their wellbeing. Recent developments in Artificial Intelligence (AI) and large context-aware multimodal models offer an opportunity to create memory support systems that assist older adults with common issues like object finding. This paper discusses the development of an AI-based, wearable memory assistant, MemPal, that helps older adults with a common problem, finding lost objects at home, and presents results from tests of the system in older adults' own homes. Using visual context from a wearable camera, the multimodal LLM system creates a real-time automated text diary of the person's activities for memory support purposes, offering object retrieval assistance using a voice-based interface. The system is designed to support additional use cases like context-based proactive safety reminders and recall of past actions. We report on a quantitative and qualitative study with N=15 older adults within their own homes that showed improved performance of object finding with audio-based assistance compared to no aid and positive overall user perceptions on the designed system. We discuss further applications of MemPal's design as a multi-purpose memory aid and future design guidelines to adapt memory assistants to older adults' unique needs.

With the rapid development of advanced machine learning methods for spatial reconstruction, it becomes important to understand the psychological and emotional impacts of such technologies on autobiographical memories. In a within-subjects study, we found that allowing users to walk through old spaces reconstructed from their videos significantly enhances their sense of traveling into past memories, increases the vividness of those memories, and boosts their emotional intensity compared to simply viewing videos of the same past events. These findings highlight that, regardless of the technological advancements, the immersive experience of VR can profoundly affect memory phenomenology and emotional engagement. As systems enabling immersive memory reconstruction become more ubiquitous, it is crucial to critically examine their effects on human cognition and perception of reality.

Rhythmic light, sound and haptic stimuli can improve cognition through neural entrainment and by modifying autonomic nervous system function. However, the effects and user experience of using wearables for inducing such rhythmic stimuli have been under-investigated. We conducted a study with 20 participants to understand the effects of rhythmic stimulation wearables on attention. We found that combined sound and light stimuli from a glasses device provided the strongest improvement to attention but were the least usable and socially acceptable. Haptic vibration stimuli from a wristband also improved attention and were the most usable and socially acceptable. Our field study (N=12) with haptic stimuli from a smartwatch showed that such systems can be easy to use and were used frequently in a range of contexts but more exploration is needed to improve the comfort. Our work contributes to developing future wearables to support attention and cognition.

People have to remember an ever-expanding volume of information. Wearables that use information capture and retrieval for memory augmentation can help but can be disruptive and cumbersome in real-world tasks, such as in social settings. To address this, we developed Memoro, a wearable audio-based memory assistant with a concise user interface. Memoro uses a large language model (LLM) to infer the user’s memory needs in a conversational context, semantically search memories, and present minimal suggestions. The assistant has two interaction modes: Query Mode for voicing queries and Queryless Mode for on-demand predictive assistance, without explicit query. Our study of (N=20) participants engaged in a real-time conversation, demonstrated that using Memoro reduced device interaction time and increased recall confidence while preserving conversational quality. We report quantitative results and discuss the preferences and experiences of users. This work contributes towards utilizing LLMs to design wearable memory augmentation systems that are minimally disruptive.

Scientific inquiry involves prediction, observation and explanation (POE) of phenomena and data. Appropriate guidance through these steps is essential for helping students learn and form positive attitudes towards science. Sensor-based education toolkits are becoming a popular way to provide this guidance, but they typically present different interfaces for measurement and learning materials which places a high cognitive demand on learners. To address this challenge, we developed a web application to integrate the scientific inquiry method where students are guided step-by-step, using a scaffolded-learning approach, through slide-based worksheets that provide direct interaction with real-time sensor measurements. We evaluate this approach through a qualitative analysis of data collected from two field studies in classrooms with a total of 42 students. We show that our approach encouraged positivity and further learning in science. Students displayed and expressed interest to conduct science experiments outside of class. We identify design implications for seamless learning, storytelling and integration of POE guided scientific inquiry with sensor-based toolkits.

With voice user interfaces (VUIs) becoming ubiquitous and speech synthesis technology maturing, it is possible to synthesise voices to resemble our friends and relatives (which we will collectively call 'kin') and use them on VUIs. However, designing such interfaces and investigating how the familiarity of kin voices affect user perceptions remain under-explored. Our surveys and interviews with 25 users revealed that VUIs using kin voices were perceived as more engaging, persuasive and safer yet eerier than VUIs using common virtual assistant voices. We then developed a technology probe, KinVoice, an Alexa-based VUI that was deployed in three households over two weeks. Users set reminders using KinVoice, which in turn, gave the reminders in synthesised kin voices. This was to explore users' needs, uncover challenges involved and inspire new applications. We discuss design guidelines for integrating familiar kin voices into VUIs, applications that benefit from its usage, and implications for balancing voice realism and usability with security and diversification.

Cognitive-affective states during learning or interactions with technologies is dependent on the mental effort of the learner and / or the cognitive load imposed by the system. Despite the growing research on the importance of understanding cognitive-affective state and their relationship to learning, measurement of such states during the learning process in Kindergartners is still unclear. While most assessments of learning and usability evaluations with Kindergartners focus on performance, self-reports and inferring from observable behaviours, they provide limited insights into the cognitive load and emotional state during the learning or interaction that are essential for a holistic picture of learning. Through a study with 18 kindergartners, we explore the feasibility of understanding cognitive-affective states associated with mental effort by triangulating the data obtained from observations, physiological markers, self-reports and performance as they performed tasks of varying mental effort. We present findings on the reliable markers within these sources across tasks. Results reveal that such a triangulation offers deeper insights into the cognitive-affective state of the learner. We believe this work would be a step towards better understanding of the learning process thereby facilitating instruction that is more aligned with the learner's cognitive-affective architecture as well as establishing guidelines for comprehensive usability / evaluation processes based on well-defined associations between child behaviour and child action.