Misinformation can spread rapidly in everyday conversation, where pausing to verify is not always possible. We envision a wearable system that bridges the timing gap between hearing a claim and forming a judgment. It uses ambient listening to detect verifiable claims, performs rapid web verification, and provides a subtle haptic nudge with a glanceable overview. A controlled study (N=34) simulated this approach and tested against a no-support baseline. Results show that instant, body-integrated feedback significantly improved real-time truth discernment and increased verification activity compared to unsupported fact-checking. However, it also introduced over-reliance when the system made errors, i.e. failed to flag false claims or flagged true claims as false. We contribute empirical evidence of improved discernment alongside insights into trust, effort, and user–system tensions in verification wearables.

Breaking negative mental health cycles, including rumination and recurring regrets, requires reflection that translates awareness into behavioral change. Grounded in the Transtheoretical Model (TTM) and Gross’s Emotion Regulation (ER) Process Model, we examine how Technologies Supporting Self-Reflection (TSR) bridge reflection and action. In a 15-day in-the-wild study (N = 20), participants used a voice-based journaling system to capture regrets and wishes and engaged in WhatIf-Planning, a novel structured reflection module integrating counterfactual thinking with if–then planning. Participants were randomized to either a free-form condition or a Gross-guided condition, which maps the five processes of Gross’s ER model into explicit journaling prompts. We contribute: (1) a unified reflection-to-action TSR system that operationalizes the Preparation stage of TTM to bridge Contemplation and Action, and (2) triangulated empirical evidence from an in-the-wild journaling study that first operationalizes Gross’s Process Model, revealing effects on coping flexibility and emotion regulation in daily life. Results show significant pre–post improvements in coping flexibility, indicating adaptive self-regulation across conditions, with the Gross-guided group generating more counterfactual alternatives, articulating concrete if–then action plans, and implementing more plans for self-driven change.

Marine ecosystems face unprecedented threats from climate change and plastic pollution, yet traditional environmental education often struggles to translate awareness into meaningful actions. This paper presents OceanChat, an interactive system leveraging large language models to create conversational AI agents represented as animated marine creatures, specifically a beluga whale, a jellyfish, and a seahorse, designed to promote pro-environmental behavior (PEB) and foster awareness through personalized dialogue. Through a between-subjects experiment (N=900), we compared three conditions: static scientific information, static character narratives, and interactive dialogue with AI-powered marine characters. Our analysis revealed that the Conversational Character Narrative condition significantly increased behavioral intentions and sustainable choice preferences compared to static approaches. The beluga whale character demonstrated consistently stronger emotional engagement across multiple measures, including perceived anthropomorphism and empathy. Our work extends research on sustainability interfaces facilitating PEB and offers design principles for creating emotionally resonant, intelligent AI characters.

Large Language Models (LLMs) are becoming increasingly ubiquitous in daily life, impacting decision-making across various domains. A substantial body of prior work has shown that individuals tend to evaluate positive predictions more favorably than negative ones—a phenomenon often referred to as the personal validation effect—across various non-AI prediction sources. Building on this foundation, we extend this well-established psychological effect to the context of LLM-based predictions, examining how prediction valence influences users’ perceptions when the source is an AI system. We investigate how positive AI-generated responses affect perceived validity, personalization, reliability, and usefulness of chatbot predictions, even when those predictions are fictitious and pre-scripted. In a study of 238 participants, positive predictions were perceived as significantly more valid (36% increase), personalized (42% increase), reliable (27% increase), and useful (22% increase) than negative predictions. These findings demonstrate that the personal validation effect persists in interactions with LLMs and underscore the substantial role of prediction valence in shaping user perceptions, with important implications for the design and deployment of AI systems across diverse applications.

Given the growing prevalence of fake information, including increasingly realistic AI-generated news, there is an urgent need to train people to better evaluate and detect misinformation. While interactions with AI have been shown to durably reduce people’s beliefs in false information, it is unclear whether these interactions also teach people the skills to discern false information themselves. We conducted a month-long study where 67 participants classified news headline-image pairs as real or fake, discussed their assessments with an AI system, followed by an unassisted evaluation of unseen news items to measure accuracy before, during, and after AI assistance. While AI assistance produced immediate improvements during AI-assisted sessions (+21% average), participants’ unassisted performance on new items declined significantly by 15.3% in week 4 compared to week 0. These results indicate that while AI may help immediately, it may ultimately degrade long-term misinformation detection abilities.

Connecting with one’s future self has been shown to enhance decision-making, improve academic performance, promote positive health outcomes, and elevate subjective quality of life. Yet traditional interventions rely on imagination or static visualizations that may not be the most effective. AI-generated digital twins offer a new approach, enabling people to engage in dialogue with a personalized representation of themselves decades ahead. However, it remains unclear how presentation modality shapes their psychological impact. We report a randomized between-subjects study (n = 92) comparing three modalities of an AI-generated future self (text, voice, and a photorealistic talking avatar) against a generic AI control. Our system integrated age progression, voice cloning, and facial animation to create personalized digital twins. All personalized modalities significantly strengthened participants’ connection to their future selves, particularly in how vividly and positively they could imagine who they will become. Although the avatar produced the largest gain in vividness, effects were comparable across modalities. Instead, subjective interaction quality, especially perceived persuasiveness, realism, and engagement, strongly predicted gains in future self-continuity and affect, indicating that experiential quality matters more than interface form. Conversation analysis revealed modality-specific patterns, with text emphasizing instrumental career planning and voice-based interactions eliciting more existential reflection. These findings indicate that effective future-self interventions do not necessarily rely on resource-intensive architecture and can scale through less demanding formats. At the same time, they raise ethical considerations about the implications of persuasive AI that engages users’ own identities.

Everyday conversations are more than exchanges of words—they reveal how people think, react, and adapt across situations. Through our speech we reveal the cognitive patterns that shape our behavior over time. Yet much of these patterns remains implicit: people operate through recurring heuristics and habits—some helpful, some limiting, many unnoticed. However, identifying such patterns requires self awareness that humans struggle with—and that current user modeling systems, often fragmented and narrowly tailored to specific tasks, fail to capture. Recognizing these patterns can enable user support systems to go beyond reactive assistance towards anticipatory support. We present Mind Mapper, an always-on wearable AI system that mines behavioral patterns from everyday real-life conversations. Mind Mapper employs a multi-stage LLM pipeline to generate, refine, and evaluate human-readable behavioral patterns. In a field study with 12 participants capturing over 700 hours of real-life conversational data, Mind Mapper generated behavioral patterns that participants consistently rated as accurate, unique, and helpful for reflection and behavior change. We further illustrate how such behavioral pattern modeling might enable new forms of human-AI interaction—anticipating user behavior through proactive interventions, adaptive content delivery, cognitive reframing, and behavioral simulation. Our results show the potential of always-on wearable systems with LLM-driven user models to support new forms of cognitively scaffolding, context-aware human-AI interactions.

Emotions, shaped by past experiences, significantly influence decision-making and goal pursuit. Traditional cognitive-behavioral techniques for personal development rely on mental imagery to envision ideal selves, but may be less effective for individuals who struggle with visualization. This paper introduces Emotional Self-Voice (ESV), a novel system combining emotionally expressive language models and voice cloning technologies to render customized responses in the user’s own voice. We investigate the potential of ESV to nudge individuals towards their ideal selves in a study with 60 participants. Across all three conditions (ESV, text-only, and mental imagination), we observed an increase in resilience, confidence, motivation, and goal commitment, and the ESV condition was perceived as uniquely engaging and personalized. We discuss the implications of designing generated self-voice systems as a personalized behavioral intervention for different scenarios.

CHI ’25, Yokohama, Japan

Older adults have increasing difficulty with retrospective memory, hindering their abilities to perform daily activities and posing stress on caregivers to ensure their wellbeing. Recent developments in Artificial Intelligence (AI) and large context-aware multimodal models offer an opportunity to create memory support systems that assist older adults with common issues like object finding. This paper discusses the development of an AI-based, wearable memory assistant, MemPal, that helps older adults with a common problem, finding lost objects at home, and presents results from tests of the system in older adults' own homes. Using visual context from a wearable camera, the multimodal LLM system creates a real-time automated text diary of the person's activities for memory support purposes, offering object retrieval assistance using a voice-based interface. The system is designed to support additional use cases like context-based proactive safety reminders and recall of past actions. We report on a quantitative and qualitative study with N=15 older adults within their own homes that showed improved performance of object finding with audio-based assistance compared to no aid and positive overall user perceptions on the designed system. We discuss further applications of MemPal's design as a multi-purpose memory aid and future design guidelines to adapt memory assistants to older adults' unique needs.

With the rapid development of advanced machine learning methods for spatial reconstruction, it becomes important to understand the psychological and emotional impacts of such technologies on autobiographical memories. In a within-subjects study, we found that allowing users to walk through old spaces reconstructed from their videos significantly enhances their sense of traveling into past memories, increases the vividness of those memories, and boosts their emotional intensity compared to simply viewing videos of the same past events. These findings highlight that, regardless of the technological advancements, the immersive experience of VR can profoundly affect memory phenomenology and emotional engagement. As systems enabling immersive memory reconstruction become more ubiquitous, it is crucial to critically examine their effects on human cognition and perception of reality.

Learning a motor skill is essential for many aspects of our lives. The complexity of some of these activities makes it hard for novices to understand through observation. Calligraphy writing is one such artistic practice where learners compare the visual differences between their writing and expert manuscripts and adjust until they have achieved similar results. We propose an accessible plotter-based system that guides the learner’s arm and hand in three directions with an actuated brush. It converts static Chinese calligraphy manuscripts to G-code that reproduces the calligrapher’s movement. Through a user study with twelve novice calligraphy learners, we validated the efficacy of our system as a learning tool that allows novices to gain an intuition of nuanced skills such as depth variation more effectively compared to watching a video recording of the same movement.

Fostering students’ interests in learning is considered to have many positive downstream effects. Large language models have opened up new horizons for generating content tuned to one’s interests, yet it is unclear in what ways and to what extent this customization could have positive effects on learning. To explore this novel dimension, we conducted a between-subjects online study (n=272) featuring different variations of a generative AI vocabulary learning app that enables users to personalize their learning examples. Participants were randomly assigned to control (sentence sourced from pre-existing text) or experimental conditions (generated sentence or short story based on users’ text input). While we did not observe a difference in learning performance between the conditions, the analysis revealed that generative AI-driven context personalization positively affected learning motivation. We discuss how these results relate to previous findings and underscore their significance for the emerging field of using generative AI for personalized learning.

Rhythmic light, sound and haptic stimuli can improve cognition through neural entrainment and by modifying autonomic nervous system function. However, the effects and user experience of using wearables for inducing such rhythmic stimuli have been under-investigated. We conducted a study with 20 participants to understand the effects of rhythmic stimulation wearables on attention. We found that combined sound and light stimuli from a glasses device provided the strongest improvement to attention but were the least usable and socially acceptable. Haptic vibration stimuli from a wristband also improved attention and were the most usable and socially acceptable. Our field study (N=12) with haptic stimuli from a smartwatch showed that such systems can be easy to use and were used frequently in a range of contexts but more exploration is needed to improve the comfort. Our work contributes to developing future wearables to support attention and cognition.

People have to remember an ever-expanding volume of information. Wearables that use information capture and retrieval for memory augmentation can help but can be disruptive and cumbersome in real-world tasks, such as in social settings. To address this, we developed Memoro, a wearable audio-based memory assistant with a concise user interface. Memoro uses a large language model (LLM) to infer the user’s memory needs in a conversational context, semantically search memories, and present minimal suggestions. The assistant has two interaction modes: Query Mode for voicing queries and Queryless Mode for on-demand predictive assistance, without explicit query. Our study of (N=20) participants engaged in a real-time conversation, demonstrated that using Memoro reduced device interaction time and increased recall confidence while preserving conversational quality. We report quantitative results and discuss the preferences and experiences of users. This work contributes towards utilizing LLMs to design wearable memory augmentation systems that are minimally disruptive.

We introduce “Future You,” an interactive, brief, single-session, digital chat intervention designed to improve future self-continuity-the degree of connection an individual feels with a temporally distant future selfa characteristic that is positively related to mental health and wellbeing. Our system allows users to chat with a relatable yet AI-powered virtual version of their future selves that is tuned to their future goals and personal qualities. To make the conversation realistic, the system generates a “future memory”-a unique backstory for each user-that creates a throughline between the user's present age (between 18–30) and their life at age 60. The “Future You” character also adopts the persona of an age-progressed image of the user. In our preregistered study$(\mathrm{N}=344)$, we found that after a brief interaction with the “Future You” character, users reported significantly decreased anxiety and increased future self-continuity compared to control conditions. This is the first study successfully demonstrating the use of personalized AI-generated characters to improve users' future self-continuity and wellbeing.

This research full paper describes a web-based online learning platform that delivers financial literacy lessons via talking head videos of AI-generated personas with two additional core features: LLM-powered proactive chat-based question-and-answer interactivity, and personal choice of the AI instructor from a list of distinct personas. We conducted two comparative studies with a total of 233 Thai students aged 1825, which aim to 1) investigate the impact of interactivity and instructor selection on the learning experience, and 2) further explore the underlying factors at play with instructor selection by introducing AI instructors' backstories as an extra intervention. We found that enabling interactivity significantly enhanced learning motivation, perceived learning facilitation, engagement, and virtual instructors' humanness compared to the passive setting. Providing learners with a choice of AI instructors provided minimal additional benefit. However, the learner's feeling of relatedness toward the instructor is a significant positive predictor of learning motivation, positive emotion, and agent credibility, while goal alignment with the agent correlates with perceived learning facilitation, and admiration corresponds with perceived agent humanness. These findings underscore the potential of interactive virtual instructors-ones that interactively encourage learners to reflect on the teaching materials throughout the lesson through two-way interaction-in enhancing motivational and experiential aspects of remote education, even if they do not significantly impact comprehension, and the importance of promoting learner's relatedness and goal alignment with the agent in boosting other aspects of the learning experience.

Despite significant historical progress, discrimination and social stigma continue to impact the lives of LGBTQIA+ individuals. The use of AI-generated virtual characters offers a unique opportunity to facilitate advocacy by engaging individuals in simulated conversations that can foster understanding, education, and empathy. This paper explores the potential of AI simulations to help individuals practice LGBTQIA+ advocacy, while also acknowledging the need for ethical considerations and addressing concerns about oversimplification or perpetuation of stereotypes. By combining technological innovation with a commitment to inclusivity, we aim to contribute to the ongoing struggle for equality in both the legal framework and the hearts and minds of the community. We present a study evaluating virtual characters driven by generative conversational AI simulating the social interactions surrounding “coming out of the closet”, a rite of passage associated with LGBTQIA+ communities. In our study, virtual characters embodied as queer individuals engage with users in a text-based conversation simulation paired with visual representations. We investigate how the interactions between the virtual characters and a user influence the user’s comfort, confidence, empathy and sympathy. The AI simulation includes distinct visual personas deployed in a series of conditions. We present findings from our deployments involving 307 users. Finally, we discuss the design implications of our work on the potential future of embodied, self-actuated and openly LGBTQIA+ intelligent agents.

Critical thinking is an essential human skill. Despite the importance of critical thinking, research reveals that our reasoning ability suffers from personal biases and cognitive resource limitations, leading to potentially dangerous outcomes. This paper presents the novel idea of AI-framed Questioning that turns information relevant to the AI classification into questions to actively engage users’ thinking and scaffold their reasoning process. We conducted a study with 204 participants comparing the effects of AI-framed Questioning on a critical thinking task; discernment of logical validity of socially divisive statements. Our results show that compared to no feedback and even causal AI explanations of an always correct system, AI-framed Questioning significantly increase human discernment of logically flawed statements. Our experiment exemplifies a future style of Human-AI co-reasoning system, where the AI becomes a critical thinking stimulator rather than an information teller.

This paper investigates how a smartphone-controlled olfactory wearable might improve memory recall. We conducted a within-subjects experiment with 32 participants using the device and without (control). In the experimental condition, bursts of odor were released during visuo-spatial memory navigation tasks, and replayed during sleep the following night in the subjects’ home. We found that compared to control, there was an improvement in memory performance when using the scent wearable in memory tasks that involved walking in a physical space. Furthermore, participants recalled more objects and translations when re-exposed to the same scent during the recall test, in addition to during sleep. These effects were statistically significant, and, in the object recall task, they also persisted for more than one week. This experiment demonstrates a potential practical application of olfactory interfaces that can interact with a user during wake as well as sleep to support memory.

Faced with public speaking anxiety, one common piece of advice is to picture the audience in a new light, using your mind’s eye. With Augmented Reality (AR) face filters, it becomes possible to literally change how one sees oneself or others. In this paper, we explore privately applied AR filters during online public speaking. Private means that these effects are only visible to the speaker. To investigate this possibly controversial concept, we conducted an online survey with 100 respondents to gather a diverse set of initial impressions, possible boundaries, and guidelines. Following this, we built a prototype of a private AR web-based video-calling application, and pilot-tested it with 16 participants to gain more in-depth insights. Based on our results, we outline key user perspectives and opportunities for the private application of AR face filters during online public speaking and discuss them in the context of previous literature on this topic.

Every human culture has developed practices and rituals associated with remembering people of the past - be it for mourning, cultural preservation, or learning about historical events. In this paper, we present the concept of “Living Memories”: interactive digital mementos that are created from journals, letters and data that an individual have left behind. Like an interactive photograph, living memories can be talked to and asked questions, making accessing the knowledge, attitudes and past experiences of a person easily accessible. To demonstrate our concept, we created an AI-based system for generating living memories from any data source and implemented living memories of the three historical figures “Leonardo Da Vinci”, “Murasaki Shikibu”, and “Captain Robert Scott”. As a second key contribution, we present a novel metrics scheme for evaluating the accuracy of living memory architectures and show the accuracy of our pipeline to improve over baselines. Finally, we compare the user experience and learning effects of interacting with the living memory of Leonardo Da Vinci to reading his journal. Our results show that interacting with the living memory, in addition to simply reading a journal, increases learning effectiveness and motivation to learn about the character.

The size and cost of electroencephalography (EEG) headsets have been decreasing at a steadfast pace. Prefrontal cortical activity is a promising input source that is also important for affect regulation. We created Joie, a joy-based EEG brain-computer interface (BCI) which uses prefrontal asymmetries associated with joyful thoughts as input to an endless runner game where the user’s character collects coins in response. In a lab study (20 participants, 15 training sessions per participant, up to two weeks of training), we found that our experiment group instructed to imagine positive music, winning awards, and similar strategies, demonstrated significantly greater ability in activating asymmetry compared to our placebo and control groups. In our analysis, Joie demonstrates the ability for prefrontal asymmetries to be used as input to an affective BCI and builds upon prior work in this area. Training these asymmetries can teach mental strategies that have applications in mental health.

Browsing multimedia objects, such as photos, videos, documents, and maps represents a frequent activity in a context of use where an end-user interacts on a large vertical display close to bystanders, such as a meeting in a corporate environment or a family display at home. In these contexts, mid-air gesture interaction is suitable for a large variety of end-users, provided that gestures are consistently mapped to similar functions across media types. We present Lui (Large User Interface), a ready-to-deploy and to-use application for browsing multimedia objects by consistent mid-air gesture interaction on a large display that is customizable by mapping new gesture classes to functions in real-time. The method followed to design the gesture interaction and to develop the application consists of four stages: (1) a contextual gesture elicitation study (23 participants × 18 referents = 414 proposed gestures) is conducted with the various media types to determine a consensus set satisfying consistency, (2) the continuous integration of this consensus set with gesture recognizers into a pipeline software architecture, (3) a comparative testing of these recognizers on the consensus set to configure the pipeline with the most efficient ones, and (4) an evaluation of the interface regarding its global quality and specific to the implemented gestures.

Video represents the majority of internet traffic today, driving a continual race between the generation of higher quality content, transmission of larger file sizes, and the development of network infrastructure. In addition, the recent COVID-19 pandemic fueled a surge in the use of video conferencing tools. Since videos take up considerable bandwidth ($\sim 100$Kbps to a few Mbps), improved video compression can have a substantial impact on network performance for live and pre-recorded content, providing broader access to multimedia content worldwide. We present a novel video compression pipeline, called Txt2Vid, which dramatically reduces data transmission rates by compressing webcam videos (“talking-head videos”) to a text transcript. The text is transmitted and decoded into a realistic reconstruction of the original video using recent advances in deep learning based voice cloning and lip syncing models. Our generative pipeline achieves two to three orders of magnitude reduction in the bitrate as compared to the standard audio-video codecs (encoders-decoders), while maintaining equivalent Quality-of-Experience based on a subjective evaluation by users ($n=242$) in an online study. The Txt2Vid framework opens up the potential for creating novel applications such as enabling audio-video communication during poor internet connectivity, or in remote terrains with limited bandwidth. The code for this work is available athttps://github.com/tpulkit/txt2vid.git.

Remote social interactions suffer from a loss of nonverbal cues used to build affiliation and connection. We propose the use of novel sensory channels for sharing social cues from interoceptive data through wearable devices that simulate the breathing and heartbeat patterns of another person, known to be linked to emotional perception and affect. We conducted a study with 16 participants testing the sharing of either heart rate or breathing rate through haptic or audio sensations. Participants experienced these sensations while watching videos of narrators describing personal experiences. We assessed the subjects’ feelings of affiliation and synchrony toward a narrator through surveys, interviews, and correlated physiological data. Our findings show that sensory devices that transfer interoceptive sensations, especially those below the level of conscious perception, can have a positive impact on feelings of connectedness. This has implications for the application of physiological channels in remote interactions to improve social connection.

This paper presents the results of a study with 134 participants to explore the effects of learning from an AI-generated virtual instructor that resembles a person one likes or admires. Given the important role instructors play in shaping learning experiences, as well as the recent surge in demand for online education, we investigate the potential for AI-generated instructors to motivate learning. Recent advances in generative AI have made it easy to create virtual instructors based on the likeness of a present-day, historical or fictional person, thereby enabling customization of video instructors based on the material, context and student. We found that while greater degrees of liking and admiration do not result in increased test scores, they can significantly improve students’ motivation towards learning, foster more positive emotions, and boost their appraisal of the AI-generated instructor as serving as an effective instructor.

Opportunities to evoke emotional experiences and modulate cognitive processes hold great importance in studying relations between emotion, cognition and behaviour as well as building technologies for maintaining positive mental health. We present the concept of Metasomatic Interactions as a new way of generating and controlling previously untapped embodied emotions using illusory sensations tuned to the prior bodily experiences. We present Frisson, a metasomatic interface built to elicit the sensations underlying the embodied emotion of aesthetic chills (i.e., goosebumps, psychogenic shivers). We present a user study (N = 14) in which the device evokes the psychogenic experience of aesthetic chills by simulating traversing thermal sensations across the spine when the illusory sensations overlap with the prior impression of aesthetic chills. The results encourage further testing of metasomatic interfaces for inducing emotions from the body. Using theories of embodied cognition, experimental results and this device, we present a discussion on characteristics of metasomatic interfaces, which we hope will inspire new categories of emotional prostheses in HCI.

The pairing of Virtual Reality technology with Physiological Sensing has gained much interest in clinical settings and beyond: from developing novel methods for diagnosis of perception and cognition impairments, biofeedback for anxiety treatment, to enhancing everyday practices such as self-guided meditation. However, conducting this type of research does not come without challenges. For example, accessing the equipment for recording data from the user and synchronizing physiological response data with the stimuli or interactive environment are not trivial tasks, and generating virtual content in response to the user’s real-time data is costly and complex. This paper presents Galea, a device for multi-modal signal acquisition able to measure the physiological response of a user when experiencing virtual content, enabling behavioral, affective computing , and human-computer interaction research and applications to access data from the Parasympathetic nervous system and Sympathetic nervous system simultaneously. We present a primer on detectable human physiology as an input source for Physiological Computing from the perspective of the signals available through our device. We describe the primary design considerations and circuit characterization results of in-vivo recordings from the wearer’s brain, eyes, heart, skin, and muscles. We also present an example to help contextualize how these signals can be used in a virtual reality setting. Galea makes working with physiological sensors in virtual reality more accessible and can offer a standard for inter and intra experiment data comparisons. Lastly, we discuss the importance and contributions of this work as well as future challenges that need to be considered.

Nonverbal vocalizations from non- and minimally speaking individuals who speak fewer than 20 words (mv* individuals) convey important communicative and affective information. While nonverbal vocalizations that occur amidst typical speech and infant vocalizations have been studied extensively in the literature, there is limited prior work on vocalizations by mv* individuals. Our work is among the first studies of the communicative and affective information expressed in nonverbal vocalizations by mv* children and adults. We collected labeled vocalizations in real-world settings with eight mv* communicators, with communicative and affective labels provided in-the-moment by a close family member. Using evaluation strategies suitable for messy, real-world data, we show that nonverbal vocalizations can be classified by function (with 4- and 5-way classifications) with F1 scores above chance for all participants. We analyze labeling and data collection practices for each participating family, and discuss the classification results in the context of our novel real-world data collection protocol. The presented work includes results from the largest classification experiments with nonverbal vocalizations from mv* communicators to date.

The problem of missing data has been persistent for a long time and poses a major obstacle in machine learning and statistical data analysis. Past works in this field have tried using various data imputation techniques to fill in the missing data, or training neural networks (NNs) with the missing data. In this work, we propose a simple yet effective approach that clusters similar input features together using hierarchical clustering and then trains proportionately split neural networks with a joint loss. We evaluate this approach on a series of benchmark datasets and show promising improvements even with simple imputation techniques. We attribute this to learning through clusters of similar features in our model architecture.

Most research works featuring AR and Brain-Computer Interface (BCI) systems are not taking advantage of the opportunities to integrate the two planes of data. Additionally, AR devices that use a Head-Mounted Display (HMD) face one major problem: constant closeness to a screen makes it hard to avoid distractions within the virtual environment. In this project, we reduced this distraction by including information about the current attentional state. We first introduce a clip-on solution for AR-BCI integration. A simple game was designed for the Microsoft HoloLens 2, which changed in real time according to the user's state of attention measured via electroencephalography (EEG). The system only responded if the attentional orientation was classified as “external.” Fourteen users tested the attention-aware system; we show that the augmentation of the interface improved the usability of the system. We conclude that more systems would benefit from clearly visualizing the user's ongoing attentional state as well as further efficient integration of AR and BCI headsets.

Reliability of machine learning (ML) systems is crucial in safety-critical applications such as healthcare, and uncertainty estimation is a widely researched method to highlight the confidence of ML systems in deployment. Sequential and parallel ensemble techniques have shown improved performance of ML systems in multi-modal settings by leveraging the feature sets together. We propose an uncertainty-aware boosting technique for multi-modal ensembling in order to focus on the data points with higher associated uncertainty estimates, rather than the ones with higher loss values. We evaluate this method on healthcare tasks related to Dementia and Parkinson's disease which involve real-world multi-modal speech and text data, wherein our method shows an improved performance. Additional analysis suggests that introducing uncertainty-awareness into the boosted ensembles decreases the overall entropy of the system, making it more robust to heteroscedasticity in the data, as well as better calibrating each of the modalities along with high quality prediction intervals. We open-source our entire codebase at https://github.com/usarawgi911//Uncertainty-aware-boosting.

Interventions aimed at promoting a growth mindset in children range from teaching about the brain's ability to change to playing computer games. In this work, we explore a novel approach to foster a growth mindset by means of interaction with a "magic hat" system which consists of using objects from sci-fi and pop-cultural references like Avengers or Star Wars. The artifacts are "enhanced" with embedded Electroencephalography (EEG) electrodes. In an initialization phase, the "magic hat" uses established Brain-Computer Interface algorithms to recognize certain mental processes of the child and the child is then able to use their brain signals to control a robot. We report on an experiment that validates the system with children who were asked to solve math problems. We evaluated their mindset before and after use of the system. In comparison with a control group, the children who used the system self-reported having a stronger growth mindset.

Human-Computer Integration (HInt) is an emerging paradigm in which computational and human systems are closely interwoven. Integrating computers with the human body is not new. however, we believe that with rapid technological advancements, increasing real-world deployments, and growing ethical and societal implications, it is critical to identify an agenda for future research. We present a set of challenges for HInt research, formulated over the course of a five-day workshop consisting of 29 experts who have designed, deployed and studied HInt systems. This agenda aims to guide researchers in a structured way towards a more coordinated and conscientious future of human-computer integration.

On-face wearables are currently limited to piercings, tattoos, or interactive makeup that aesthetically enhances the user, and have been minimally used for scent-delivery methods. However, on-face scent interfaces could provide an advantage for personal scent delivery in comparison with other modalities or body locations since they are closer to the nose. In this paper, we present the mechanical and industrial design details of a series of form factors for on-face olfactory wearables that are lightweight and can be adhered to the skin or attached to glasses or piercings. We assessed the usability of three prototypes by testing with 12 participants in a within-subject study design while they were interacting in pairs at a close personal distance. We compare two of these designs with an "off-face" olfactory necklace and evaluate their social acceptance, comfort as well as perceived odor intensity for both the wearer and observer.

Nonverbal vocalizations contain important affective and communicative information, especially for those who do not use traditional speech, including individuals who have autism and are non- or minimally verbal (nv/mv). Although these vocalizations are often understood by those who know them well, they can be challenging to understand for the community-at-large. This work presents (1) a methodology for collecting spontaneous vocalizations from nv/mv individuals in natural environments, with no researcher present, and personalized in-the-moment labels from a family member; (2) speaker-dependent classification of these real-world sounds for three nv/mv individuals; and (3) an interactive application to translate the nonverbal vocalizations in real time. Using support-vector machine and random forest models, we achieved speaker-dependent unweighted average recalls (UARs) of 0.75, 0.53, and 0.79 for the three individuals, respectively, with each model discriminating between 5 nonverbal vocalization classes. We also present first results for real-time binary classification of positive- and negative-affect nonverbal vocalizations, trained using a commercial wearable microphone and tested in real time using a smartphone. This work informs personalized machine learning methods for non-traditional communicators and advances real-world interactive augmentative technology for an underserved population.

A hard challenge for wearable systems is to measure differences in emotional valence, i.e. positive and negative affect via physiology. However, the stomach or gastric signal is an unexplored modality that could offer new affective information. We created a wearable device and software to record gastric signals, known as electrogastrography (EGG). An in-laboratory study was conducted to compare EGG with electrodermal activity (EDA) in 33 individuals viewing affective stimuli. We found that negative stimuli attenuate EGG's indicators of parasympathetic activation, or "rest and digest" activity. We compare EGG to the remaining physiological signals and describe implications for affect detection. Further, we introduce how wearable EGG may support future applications in areas as diverse as reducing nausea in virtual reality and helping treat emotion-related eating disorders.

Alzheimer's disease is estimated to affect around 50 million people worldwide and is rising rapidly, with a global economic burden of nearly a trillion dollars. This calls for scalable, cost-effective, and robust methods for detection of Alzheimer's dementia (AD). We present a novel architecture that leverages acoustic, cognitive, and linguistic features to form a multimodal ensemble system. It uses specialized artificial neural networks with temporal characteristics to detect AD and its severity, which is reflected through Mini-Mental State Exam (MMSE) scores. We first evaluate it on the ADReSS challenge dataset, which is a subject-independent and balanced dataset matched for age and gender to mitigate biases, and is available through DementiaBank. Our system achieves state-of-the-art test accuracy, precision, recall, and F1-score of 83.3% each for AD classification, and state-of-the-art test root mean squared error (RMSE) of 4.60 for MMSE score regression. To the best of our knowledge, the system further achieves state-of-the-art AD classification accuracy of 88.0% when evaluated on the full benchmark DementiaBank Pitt database. Our work highlights the applicability and transferability of spontaneous speech to produce a robust inductive transfer learning model, and demonstrates generalizability through a task-agnostic feature-space. The source code is available at https://github.com/wazeerzulfikar/alzheimers-dementia

Where does human agency remain in the era of automation and intelligent machineries? Technological affordances create instant ways of accomplishing challenging tasks, where a lot of individual agency is mitigated. In this installation, we present an autonomous musical instrument that invites volunteers to eventually take a subordinate role to machines. Over time, the instrument gradually raises its level of autonomy, eventually overriding through musical patterns impractical to play by users. Juxtaposing machine uprising and an artifact that manifests human creativity, we question the relationship between technology and human on the continuum of symbiosis and adversary, through orders of algorithmic complexity and intensity in the act of music.

Several research projects have recently explored the use of physiological sensors such as electroencephalography (EEG) or electrooculography (EOG) to measure the engagement and vigilance of a user in context of car driving. However, these systems still suffer from limitations such as an absence of a socially acceptable form-factor and use of impractical, gel-based electrodes. We present AttentivU, a device using both EEG and EOG for real-time monitoring of physiological data. The device is designed as a socially acceptable pair of glasses and employs silver electrodes. It also supports real-time delivery of feedback in the form of an auditory signal via a bone conduction speaker embedded in the glasses. A detailed description of the hardware design and proof of concept prototype is provided, as well as preliminary data collected from 20 users performing a driving task in a simulator in order to evaluate the signal quality of the physiological data.

Recently several research projects have explored using physiological sensors such as electroencephalography (EEG) or electrooculography (EOG) electrodes to measure the engagement of a user in different contexts and augment learning activities. However, these systems still suffer from limitations such as an absence of a socially acceptable design, or use of impractical gel-based electrodes. We present AttentivU, a device using both EEG and EOG for real-time monitoring of physiological data. The device is designed as a socially acceptable pair of glasses and employs silver electrodes as an alternative to the commonly used silver/silver chloride (Ag/AgCI) “wet” electrodes. A detailed description of the hardware design and proof of concept prototype is provided, as well as a side by side comparison of conventional wet electrodes.

Automatic and real-time sleep scoring is necessary to develop user interfaces that trigger stimuli in specific sleep stages. However, most automatic sleep scoring systems have been focused on offline data analysis. We present the first, real-time sleep staging system that uses deep learning without the need for servers in a smartphone application for a wearable EEG. We employ real-time adaptation of a single channel Electroencephalography (EEG) to infer from a Time-Distributed Convolutional Neural Network (CNN). Polysomnography (PSG) -the gold standard for sleep staging-requires a human scorer and is both complex and resource-intensive. Our work demonstrates an end-to-end, smartphone-based pipeline that can infer sleep stages in just single 30-second epochs, with an overall accuracy of 83.5% on 20-fold cross validation for 5-stage classification of sleep stages using the open Sleep-EDF dataset. For comparison, inter-rater reliability among sleep-scoring experts is about 80% (Cohen's k=0\pmb.68 to \pmb0.76). We further propose an on-device metric independent of the deep learning model which increases the average accuracy of classifying deep-sleep (N3) to more than 97.2% on 4 test nights using power spectral analysis.

We present a small and lightweight wearable device that enhances virtual reality experiences and reduces cybersickness by means of galvanic vestibular stimulation (GVS). GVS is a specific way to elicit vestibular reflexes that has been used for over a century to study the function of the vestibular system. In addition to GVS, we support physiological sensing by connecting heart rate, electrodermal activity and other sensors to our wearable device using a plug and play mechanism. An accompanying Android app communicates with the device over Bluetooth (BLE) for transmitting the GVS stimulus to the user through electrodes attached behind the ears. Our system supports multiple categories of virtual reality applications with different types of virtual motion such as driving, navigating by flying, teleporting, or riding. We present a user study in which participants (N = 20) experienced significantly lower cybersickness when using our device and rated experiences with GVS-induced haptic feedback as significantly more immersive than a no-GVS baseline.

This paper studies the feasibility of using low-cost motion sensors to provide opportunistic heart rate assessments from ballistocardiographic signals during restful periods of daily life. Three wearable devices were used to capture peripheral motions at specific body locations (head, wrist, and trouser pocket) of 15 participants during five regular workdays each. Three methods were implemented to extract heart rate from motion data and their performance was compared to those obtained with an FDA-cleared device. With a total of 1358 h of naturalistic sensor data, our results show that providing accurate heart rate estimations from peripheral motion signals is possible during relatively "still" moments. In our real-life workplace study, the head-mounted device yielded the most frequent assessments (22.98% of the time under 5 beats per minute of error) followed by the smartphone in the pocket (5.02%) and the wrist-worn device (3.48%). Most importantly, accurate assessments were automatically detected by using a custom threshold based on the device jerk. Due to the pervasiveness and low cost of wearable motion sensors, this paper demonstrates the feasibility of providing opportunistic large-scale low-cost samples of resting heart rate.

Virtual reality can help realize mediated social experiences where distance disappears and we interact as richly with those around the world as we do with those in the same room. The design of social virtual experiences presents a challenge for remotely located users with room-scale setups like those afforded by recent commodity virtual reality devices. Since users inhabit different physical spaces that may not be the same size, a mapping to a shared virtual space is needed for creating experiences that allow everyone to use real walking for locomotion. We designed three mapping techniques that enable users from diverse room-scale setups to interact together in virtual reality. Results from our user study (N = 26) show that our mapping techniques positively influence the perceived degree of togetherness and copresence while the size of each user's tracked space influences individual presence.

Physically walking in virtual reality can provide a satisfying sense of presence. However, natural locomotion in virtual worlds larger than the tracked space remains a practical challenge. Numerous redirected walking techniques have been proposed to overcome space limitations but they often require rapid head rotation, sometimes induced by distractors, to keep the scene rotation imperceptible. We propose a design methodology of seamlessly integrating redirection into the virtual experience that takes advantage of the perceptual phenomenon of inattentional blindness. Additionally, we present four novel visibility control techniques that work with our design methodology to minimize disruption to the user experience commonly found in existing redirection techniques. A user study (N = 16) shows that our techniques are imperceptible and users report significantly less dizziness when using our methods. The illusion of unconstrained walking in a large area (16 x 8m) is maintained even though users are limited to a smaller (3.5 x 3.5m) physical space.

Today's children are growing up with smart toys, Internet-connected devices that use artificial intelligence to drive interactive play. In a prior research study, we found that children ages 4--10 perceive these toys as worthy of trust [5]. This leads us to inquire if children in this age range could be directly influenced by these devices. In this work, we used a conformity test and a disobedience task to study how children are influenced by a talking doll. We found that the doll could influence children to change their judgments about moral transgressions, however it was unsuccessful in persuading children to disobey an instruction. Finally, we analyzed children's perceptions of the smart toy and discusses implications of this work for future child-agent interaction.

The ability to relax is sometimes challenging to achieve, nevertheless it is extremely important for mental and physical health, particularly to effectively manage stress and anxiety. We propose a virtual reality experience that integrates a wearable, low-cost EEG headband and an olfactory necklace that passively promotes relaxation. The physiological response was measured from the EEG signal. Relaxation scores were computed from EEG frequency bands associated with a relaxed mental state using an entropy-based signal processing approach. The subjective perception of relaxation was determined using a questionnaire. A user study involving 12 subjects showed that the subjective perception of relaxation increased by 26.1 % when using a VR headset with the olfactory necklace, compared to not being exposed to any stimulus. Similarly, the physiological response also increased by 25.0 %. The presented work is the first Virtual Reality Therapy system that uses scent in a wearable manner and proves its effectiveness to increase relaxation in everyday life situations.

With virtual reality head-mounted displays rapidly becoming accessible to mass audiences, there is growing interest in new forms of natural input techniques to enhance immersion and engagement for players. Research has explored physiological input for enhancing immersion in single player games through indirectly controlled signals like heart rate or galvanic skin response. In this paper, we propose breathing as a directly controlled physiological signal that can facilitate unique and engaging play experiences through natural interaction in single and multiplayer virtual reality games. Our study (N = 16) shows that participants report a higher sense of presence and find the gameplay more fun and challenging when using our breathing actions. From study observations and analysis we present five design strategies that can aid virtual reality game designers interested in using directly controlled forms of physiological input.

Virtual and augmented reality headsets are unique as they have access to our facial area: an area that presents an excellent opportunity for always-available input and insight into the user's state. Their position on the face makes it possible to capture bio-signals as well as facial expressions. This paper introduces the PhysioHMD, a software and hardware modular interface built for collecting affect and physiological data from users wearing a head-mounted display. The PhysioHMD platform is a flexible architecture enables researchers and developers to aggregate and interprets signals in real-time, and use those to develop novel, personalized interactions and evaluate virtual experiences. Offering an interface that is not only easy to extend but also is complemented by a suite of tools for testing and analysis. We hope that PhysioHMD can become a universal, publicly available testbed for VR and AR researchers.

Embodied theories of language propose that the way we communicate verbally is grounded in our body. Nevertheless, the way a second language is conventionally taught does not capitalize on kinesthetic modalities. The tracking capabilities of room-scale virtual reality systems afford a way to incorporate kinesthetic learning in language education. We present Words in Motion, a virtual reality language learning system that reinforces associations between word-action pairs by recognizing a student's movements and presenting the corresponding name of the performed action in the target language. Results from a user study involving 57 participants suggest that the kinesthetic approach in virtual reality has less immediate learning gain in comparison to a text-only condition and no immediate difference with participants in a non-kinesthetic virtual reality condition. However, virtual kinesthetic learners showed significantly higher retention rates after a week of exposure than all other conditions and higher performance than non-kinesthetic virtual reality learners. Positive correlation between the times a word-action pair was executed and the times a word was remembered by the subjects, supports that virtual reality can impact language learning by leveraging kinesthetic elements.

We present a wearable interface that allows a user to silently converse with a computing device without any voice or any discernible movements - thereby enabling the user to communicate with devices, AI assistants, applications or other people in a silent, concealed and seamless manner. A user's intention to speak and internal speech is characterized by neuromuscular signals in internal speech articulators that are captured by the AlterEgo system to reconstruct this speech. We use this to facilitate a natural language user interface, where users can silently communicate in natural language and receive aural output (e.g - bone conduction headphones), thereby enabling a discreet, bi-directional interface with a computing device, and providing a seamless form of intelligence augmentation. The paper describes the architecture, design, implementation and operation of the entire system. We demonstrate robustness of the system through user studies and report 92% median word accuracy levels.

Most interfaces of our interactive devices such as phones and laptops are flat and are built as external devices in our environment, disconnected from our bodies. Therefore, we need to carry them with us in our pocket or in a bag and accommodate our bodies to their design by sitting at a desk or holding the device in our hand. We propose Hand Range Interface, an input surface that is always at our fingertips. This body-centric interface is a semi-sphere attached to a user's wrist, with a radius the same as the distance from the wrist to the index finger. We prototyped the concept in virtual reality and conducted a user study with a pointing task. The input surface can be designed as rotating with the wrist or fixed relative to the wrist. We evaluated and compared participants' subjective physical comfort level, pointing speed and pointing accuracy on the interface that was divided into 64 regions. We found that the interface whose orientation was fixed had a much better performance, with 41.2% higher average comfort score, 40.6% shorter average pointing time and 34.5% lower average error. Our results revealed interesting insights on user performance and preference of different regions on the interface. We concluded with a set of guidelines for future designers and developers on how to develop this type of new body-centric input surface.

Various forms of wearable robotics challenge the notion of the human body, in that the robots render the acquired capabilities in physical forms. However, majority of such systems are designed for specific purposes, where rapidly changing environments pose a diverse set of problems that are difficult to solve with a single interface. To address this, we propose a modular hardware platform that allows its users or designers to build and customize wearable robots. The process of building an augmentation is simply to connect actuator and sensor blocks and attach them to the body. The current list of designed components includes servomotor modules and sensor modules, that can be programmed to incorporate additional electronics for desired sensing capabilities. Our electrical and mechanical connector designs can be extended to utilize any motors within afforded power, size, and weight constraints. We also show how our platform can be used in various applications, in addition to how the proposed design can be extended as well as challenges for future systems.

We present Oasis, a novel system for automatically generating immersive and interactive virtual reality environments for single and multiuser experiences. Oasis enables real-walking in the generated virtual environment by capturing indoor scenes in 3D and mapping walkable areas. It makes use of available depth information for recognizing objects in the real environment which are paired with virtual counterparts to leverage the physicality of the real world, for a more immersive virtual experience. Oasis allows co-located and remotely located users to interact seamlessly and walk naturally in a shared virtual environment. Experiencing virtual reality with currently available devices can be cumbersome due to presence of objects and furniture which need to be removed every time the user wishes to use VR. Our approach is new, in that it allows casual users to easily create virtual reality environments in any indoor space without rearranging furniture or requiring specialized equipment, skill or training. We demonstrate our approach to overlay a virtual environment over an existing physical space through fully working single and multiuser systems implemented on a Tango tablet device.

The sense of smell is perhaps the most pervasive of all senses, but it is also one of the least understood and least exploited in HCI. We present Essence, the first olfactory computational necklace that can be remotely controlled through a smartphone and can vary the intensity and frequency of the released scent based on biometric or contextual data. This paper discusses the role of smell in designing pervasive systems that affect one's mood and cognitive performance while being asleep or awake. We present a set of applications for this type of technology as well as the implementation of the olfactory display and the supporting software. We also discuss the results of an initial test of the prototype that show the robustness and usability of Essence while wearing it for long periods of time in multiple environments.

Printflatables is a design and fabrication system for human-scale, functional and dynamic inflatable objects. We use inextensible thermoplastic fabric as the raw material with the key principle of introducing folds and thermal sealing. Upon inflation, the sealed object takes the expected three dimensional shape. The workflow begins with the user specifying an intended 3D model which is decomposed to two dimensional fabrication geometry. This forms the input for a numerically controlled thermal contact iron that seals layers of thermoplastic fabric. In this paper, we discuss the system design in detail, the pneumatic primitives that this technique enables and merits of being able to make large, functional and dynamic pneumatic artifacts. We demonstrate the design output through multiple objects which could motivate fabrication of inflatable media and pressure-based interfaces.

BIO: Pattie Maes is the Alexander W. Dreyfoos (1954) Professor in MIT's Program in Media Arts and Sciences and associate head of the Program in Media Arts and Sciences. She founded and directs the Media Lab's Fluid Interfaces research group. Previously, she founded and ran the Software Agents group. Prior to joining the Media Lab, Maes was a visiting professor and a research scientist at the MIT Artificial Intelligence Lab. She holds bachelor's and PhD degrees in computer science from the Vrije Universiteit Brussel in Belgium. Her areas of expertise are human-computer interaction and artificial intelligence. Maes is the editor of three books, and is an editorial board member and reviewer for numerous professional journals and conferences. She has received several awards: FastCompany named her one of 50 most influential designers (2011). Newsweek magazine named her one of the "100 Americans to watch for" in the year 2000; TIME Digital selected her as a member of the Cyber-Elite, the top 50 technological pioneers of the high-tech world; the World Economic Forum honored her with the title "Global Leader for Tomorrow"; Ars Electronica awarded her the 1995 World Wide Web category prize; and in 2000 she was recognized with the "Lifetime Achievement Award" by the Massachusetts Interactive Media Council. She also received an honorary doctorate from the Vrije Universiteit Brussel in Belgium. Her 2009 TED talk is among the most watched TED talks ever. In addition to her academic endeavors, Maes has been active as an entrepreneur as cofounder of several venture-backed companies including Firefly Networks (sold to Microsoft) and Open Ratings (sold to Dun & Bradstreet). She remains an advisor and investor to several MIT spinoffs.

Affective virtual spaces are of interest in many virtual reality applications such as education, wellbeing, rehabilitation, and entertainment. In this paper we present Auris, a system that attempts to generate affective virtual environments from music. We use music as input because it inherently encodes emotions that listeners readily recognize and respond to. Creating virtual environments is a time consuming and labor-intensive task involving various skills like design, 3D modeling, texturing, animation, and coding. Auris helps make this easier by automating the virtual world generation task using mood and content extracted from song audio and lyrics data respectively. Our user study results indicate virtual spaces created by Auris successfully convey the mood of the songs used to create them and achieve high presence scores with the potential to provide novel experiences of listening to music.

GalVR is a navigation interface that uses galvanic vestibular stimulation (GVS) during walking to cause users to turn from their planned trajectory. We explore GalVR for collaborative navigation in a two-player virtual reality (VR) game. The interface affords a novel game design that exploits the differences in first and third person perspectives, allowing VR and non-VR users to share a play experience. By introducing interdependence arising from dissimilar points of view, players can uniquely contribute to the shared experience based on their roles. We detail the design of our asymmetrical game, Dark Room and present some insights from a pilot study. Trust emerged as the defining factor for successful play.

Physical interfaces with actuation capability enable the design of wearable devices that augment human physical capabilities. Extra machine joints integrated to our biological body may allow us to achieve additional skills through programmatic reconfiguration of the joints. To that end, we present a wearable multi-joint interface that offers "synergistic interactions" by providing additional fingers, structural supports, and physical user interfaces. Motions of the machine joints can be controlled via interfacing with our muscle signals, as a direct extension of our body. On the basis of implemented applications, we demonstrate our design guidelines for creating a desirable human-machine synergy -- that enhances our innate capabilities, not replacing or obstructing, and also without enforcing the augmentation. Finally we describe technical details of our muscle-based control method and implementations of the presented applications.

Exploratory Learning with Virtual Companions Sharing Attention and Context (EVA) is a concept for mediated teaching and learning that sits at the intersection of exploratory learning, telepresence, and attention awareness. The companion teacher is informed about the attentional state and environment of the learner, and can refer directly to this environment through marking or annotation. To the learner, the companion is virtual -- either human or automatic -- and, if human, either physically copresent or remote. The content and style of presentation are tailored to the learner's momentary level of interest or focus, and her attention can be guided to salient environmental elements (e.g. visual) in order to convey desired information. We define a design space for such systems, which applies to learning in Augmented Reality and Virtual Reality, and can be employed as a framework for design and evaluation. We demonstrate this through trials with two proof-of-concept systems, one in AR and one in VR, with a human companion. We conclude that the EVA design space defines a powerful set of systems for learning and finish by presenting guidelines for making such systems maximally effective.

The Experience Sampling Method is widely used for collecting self-report responses from people in natural settings. While most traditional approaches rely on using a phone to trigger prompts and record information, wearable devices now offer new opportunities that may improve this method. This research quantitatively and qualitatively studies the experience sampling process on head-worn and wrist-worn wearable devices, and compares them to the traditional "smartphone in the pocket." To enable this work, we designed and implemented a custom application to provide similar prompts across the three types of devices and evaluated it with 15 individuals for five days (75 days total), in the context of real-life stress measurement. We found significant differences in response times across devices, and captured tradeoffs in interaction types, screen size, and device familiarity that can affect both users' experience and the reports made by users.

The sense of agency (SoA) describes the feeling of being the author and in control of one's movements. It is closely linked to automated aspects of sensorimotor control and understood to depend on one's ability to monitor the details of one's movements. As such SoA has been argued to be a critical component of self-awareness in general and contribute to presence in virtual reality environments in particular. A common approach to investigating SoA is to ask participants to perform goal-directed movements and introducing spatial or temporal visuomotor mismatches in the feedback. Feedback movements are traditionally either switched with someone else's movements using a 2D video-feed or modified by providing abstracted feedback about one's actions on a computer screen. The aim of the current study was to quantify conscious monitoring and the SoA for ecologically valid, three dimensional feedback of the participants' actual limb and movements. This was achieved by displaying an Infra-Red (IR) feed of the participants' upper limbs in an augmented virtuality environment (AVE) using a head-mounted display (HMD). Movements could be fed back in real-time (46ms system delay) or with an experimental delay of up to 570ms. As hypothesized, participant's SoA decreased with increasing temporal visuomotor mismatches (p<;.001), replicating previous findings and extending them to AVEs. In-line with this literature, we report temporal limits of 222±60ms (50% psychometric threshold) in N=28 participants. Our results demonstrate the validity of the experimental platform by replicating studies in SoA both qualitatively and quantitatively. We discuss our findings in relation to the use of virtual and mixed reality in research and implications for neurorehabilitation therapies.

Drawing as a means of expression has evolved over time, as, and through a means of computation: Since pre-historic time, humankind has been involved in drawing through a myriad forms of mediums that, over many years, have evolved to be increasingly computation-driven. However, they largely continue to remain constrained to human body scale and aesthetics, while computer technology now allows a more synergistic and collaborative expression between human and machine. In our installation, we engage audience with a drone-based drawing system that applies a person's pen drawing at different scales in different styles. The unrestricted and programmable motion of the proxy can institute various artistic distortions in real-time, creating a new dynamic medium of creative expression.

We present GyroVR, head worn flywheels designed to render inertia in Virtual Reality (VR. Motions such as flying, diving or floating in outer space generate kinesthetic forces onto our body which impede movement and are currently not represented in VR. We simulate those kinesthetic forces by attaching flywheels to the users head, leveraging the gyroscopic effect of resistance when changing the spinning axis of rotation. GyroVR is an ungrounded, wireless and self contained device allowing the user to freely move inside the virtual environment. The generic shape allows to attach it to different positions on the users body. We evaluated the impact of GyroVR onto different mounting positions on the head (back and front) in terms of immersion, enjoyment and simulator sickness. Our results show, that attaching GyroVR onto the users head (front of the Head Mounted Display (HMD)) resulted in the highest level of immersion and enjoyment and therefore can be built into future VR HMDs, enabling kinesthetic forces in VR.

Mobile phone-based Virtual Reality (VR) is rapidly growing as a platform for stereoscopic 3D and non-3D digital content and applications. The ability to track eye gaze in these devices would be a tremendous opportunity on two fronts: firstly, as an interaction technique, where interaction is currently awkward and limited, and secondly, for studying human visual behavior. We propose a method to add eye gaze tracking to these existing devices using their on-board display and camera hardware, with a minor modification to the headset enclosure. We present a proof-of-concept implementation of the technique and show results demonstrating its feasibility. The software we have developed will be made available as open source to benefit the research community.

Fine motor skills like finger/hand manipulations are essential for playing musical instruments and these skills require a great amount of time and effort to acquire. Researchers have been introducing haptic feedback systems in order to facilitate the process of learning motor skills but little research has expanded the possibility of applying to the field of musical instruments. Hence, we developed a system called "MoveMe" that provides three-dimensional haptic support for playing a musical instrument. The system guides a user's hands as if someone else was holding their hands to help a beginner play a musical instrument. With the system, an expert can pre-record his/her movements so that a beginner can play it back later as necessary. Alternatively, the system connects an expert and a beginner via two haptic robots and the expert can, in real time, guide and correct the beginner's movement. In addition to those functionalities, we introduce a new proficiency metric provided by force feedback. A master can evaluate how much a beginner has improved using both audio feedback as well as this new force-based metric. Through the experiments that we conducted, we found that our system is effective in terms of playing a song at a correct speed and rhythm.

Data-Objects introduces the idea of re-designing physical objects that a person uses every day to act as tactile affective interfaces. Data-Objects are a means to provide users information about their use of different everyday objects and how it affects them. We do this by re-designing the objects to embed information in the physical body of the object itself without hampering its original functional capability. By 3D printing the body in a set period of time, differences in the information represented on the physical body over the time are aimed at highlighting patterns of how the usage of that object has been affecting the user. The overwhelming digital information that we are exposed to and the disconnect that it has from what it is representing makes the relation of the information to the different aspects of our life not effective. We think that using physical forms of objects to provide information can make the data more meaningful, enhancing the value of the object beyond its intended function. Physical forms can provide subliminal and tactile feedback to the users as they use the objects throughout the day, without specific visual attention. Being present physically ensures that people are more conscious of the data and patterns, and makes the data visible to other people as well.

Accessing printed text in a mobile context is a major challenge for the blind. A preliminary study with blind people reveals numerous difficulties with existing state-of-the-art technologies including problems with alignment, focus, accuracy, mobility and efficiency. In this paper, we present a finger-worn device, FingerReader, that assists blind users with reading printed text on the go. We introduce a novel computer vision algorithm for local-sequential text scanning that enables reading single lines, blocks of text or skimming the text with complementary, multimodal feedback. This system is implemented in a small finger-worn form factor, that enables a more manageable eyes-free operation with trivial setup. We offer findings from three studies performed to determine the usability of the FingerReader.

Ubiquitous computing has been focusing on creating smart agents that are submerged into everyday environments, however, recent development on physical computing is demanding a shift from calm computing to a physically engaging form. Computing is no more limited to increasing our comfort through passive and pervasive deployment, they can now be created as being more actively and physically intermeshed into our tasks. We present L'evolved, autonomous ubiquitous utilities that assist in user tasks through active physical participation. They not only dynamically adapt to individual user needs and actions, but also work in close tandem with the users. Among explorations on potential applications, we harness drone technology to realize the design and implementation of example utilities that afford free motions and computational controls. Through various use scenarios of those exemplary utilities, we show how this new form of smart agents promises new ways of interacting with our physical environments. We also discuss design implications and technical details of our implementations.

Wearable commodities are able to extend beyond the temporal span of a particular community event, offering omnipresent vehicles for producing icebreaking interaction opportunities. We introduce a novel platform, which generates social affordances to facilitate community organizers in aggregating social interaction among unacquainted, collocated members beyond initial hosted gatherings. To support these efforts, we present functional work-in-progress prototypes for Social Textiles, wearable computing textiles which enable social messaging and peripheral social awareness on non-emissive digitally linked shirts. The shirts serve as catalysts for different social depths as they reveal common interests (mediated by community organizers), based on the physical proximity of users. We provide 3 key scenarios, which demonstrate the user experience envisioned with our system. We present a conceptual framework, which shows how different community organizers across domains such as universities, brand communities and digital self-organized communities can benefit from our technology.

clayodor (\klei-o-dor\) is a clay-like malleable material that changes smell based on user manipulation of its shape. This work explores the tangibility of shape changing materials to capture smell, an ephemeral and intangible sensory input. We present the design of a proof-of-concept prototype, and discussions on the challenges of navigating smell though form.

Forces of gravity and the concept of a fixed center of mass severely limit the opportunities for product design and user interaction with those products. In this paper, we present L-Shift, a fictional material in which the weight distribution of physical objects is manipulated dynamically. It is a material that shifts its phase between solid and liquid and, therefore, allows for changes in weight distribution and rigidity. This gives designers many more options, as products can be re-shaped, re-balanced and stiffened as needed. Through prototyping with sodium acetate -- a material that has properties close to the proposed fictional one -- we explore user scenarios and future possibilities. We envision that it is possible to manipulate such physical properties of objects dynamically, which will open up new opportunities for designing interactive and more responsive products.

The huge influx of mobile display devices is transforming computing into multi-device interaction, demanding a fluid mechanism for using multiple devices in synergy. In this paper, we present a novel interaction system that allows a collocated large display and a small handheld device to work together. The smartphone acts as a physical interface for near-surface interactions on a computer screen. Our system enables accurate position tracking of a smartphone placed on or over any screen by displaying a 2D color pattern that is captured using the smartphone's back-facing camera. As a result, the smartphone can directly interact with data displayed on the host computer, with precisely aligned visual feedback from both devices. The possible interactions are described and classified in a framework, which we exemplify on the basis of several implemented applications. Finally, we present a technical evaluation and describe how our system is unique compared to other existing near-surface interaction systems. The proposed technique can be implemented on existing devices without the need for additional hardware, promising immediate integration into existing systems.

We present an augmented airbrush that allows novices to experience the art of spray painting. Inspired by the thriving field of smart tools, our handheld device uses 6DOF tracking, augmentation of the airbrush trigger, and a specialized algorithm to restrict the application of paint to a preselected reference image. Our device acts both as a physical spraying device and as an intelligent assistive tool, providing simultaneous manual and computerized control. Unlike prior art, here the virtual simulation guides the physical rendering ( inverse rendering ), allowing for a new spray painting experience with singular physical results. We present our novel hardware design, control software, and a user study that verifies our research objectives.

We present the design, and evaluation of WaaZam, a video mediated communication system designed to support creative play in customized environments. Users can interact together in virtual environments composed of digital assets layered in 3D space. The goal of the project is to support creative play and increase social engagement during video sessions of geographically separated families. We try to understand the value of customization for individual families with children ages 6-12. We present interviews with creativity experts, a pilot study and a formal evaluation of families playing together in four conditions: separate windows, merged windows, digital play sets, and customized digital environments. We found that playing in the same video space enables new activities and increases social engagement for families. Customization allows families to modify scenes for their needs and support more creative play activities that embody the imagination of the child.

Flexpad is an interactive system that combines a depth camera and a projector to transform sheets of plain paper or foam into flexible, highly deformable, and spatially aware handheld displays. We present a novel approach for tracking deformed surfaces from depth images in real time. It captures deformations in high detail, is very robust to occlusions created by the user's hands and fingers, and does not require any kind of markers or visible texture. As a result, the display is considerably more deformable than in previous work on flexible handheld displays, enabling novel applications that leverage the high expressiveness of detailed deformation. We illustrate these unique capabilities through three application examples: curved cross-cuts in volumetric images, deforming virtual paper characters, and slicing through time in videos. Results from two user studies show that our system is capable of detecting complex deformations and that users are able to perform them quickly and precisely.

This paper details the design and implementation of a new type of display technology. Display Blocks are a response to two major limitations of current displays: dimensional compression and physical-digital disconnect. Each Display Block consists of six organic light emitting diode (OLED) screens, arranged in a cubic form factor. We explore the possibilities that this type of display holds for data visualization, manipulation and exploration. To this end, we accompany our design with a set of initial applications that leverage the form factor of the displays. We hope that this work shows the promise of display technologies which use their form factor as a cue to understanding their content.

The Perifoveal Display (see Figure 1) is a visualization display for complex, real-time, dynamic data such as stock market data, traffic or control room as well as virtual 3D environments. The system takes advantage of the unique properties of the human perceptive system, which is capable of perceiving a high degree of detail in the foveal area, but has a unique more subliminal type of perception of movement and brightness in the peripheral area. The Perifoveal Display varies how data is visualized based on the user's viewing direction. Data in the center of the user's focus is displayed in a lot of detail. Movement and change in brightness as well as amount of detail and size highlight important data changes that fall into the periphery. The results of our user study show that the system is able to support the user while observing complex data.

This paper presents the design, implementation, and evaluation of an interactive tabletop system that supports co-located meeting capture and asynchronous search and review of past meetings. The goal of the project is to evaluate the design of a conference table that augments the everyday work patterns of small collaborative groups by incorporating an integrated annotation system. We present a holistic design that values hardware ergonomics, supports heterogeneous input modalities, generates a memory of all user interactions, and provides access to historical data on and off the table. We present a user evaluation that assesses the usefulness of the input modalities and software features, and validates the effectiveness of the MemTable system as a tool for assisting memory recall.

SPARSH presents a seamless way of passing data among multiple users and devices. The user touches a data item they wish to copy from a device, conceptually saving it in the user's body. Next, the user touches the other device they want to paste/pass the saved content. SPARSH uses touch-based interactions as indications for what to copy and where to pass it. Technically, the actual transfer of media happens via the information cloud. Accompanying video shows some of the SPARSH scenarios.

SPARSH explores a novel interaction method to seamlessly transfer data among multiple users and devices in a fun and intuitive way. The user touches a data item they wish to copy from a device, conceptually saving in the user's body. Next, the user touches the other device they want to paste/pass the saved content. SPARSH uses touch-based interactions as indications for what to copy and where to pass it. Technically, the actual transfer of media happens via the information cloud.

As personal projection devices become more common they will be able to support a range of exciting and unexplored social applications. We present a novel system and method that enables playful social interactions between multiple projected characters. The prototype consists of two mobile projector-camera systems, with lightly modified existing hardware, and computer vision algorithms to support a selection of applications and example scenarios. Our system allows participants to discover the characteristics and behaviors of other characters projected in the environment. The characters are guided by hand movements, and can respond to objects and other characters, to simulate a mixed reality of life-like entities.

Pattie Maes is an associate professor in MIT's Program in Media Arts and Sciences and associate head of the Program in Media Arts and Sciences. She founded and directs the Media Lab's Fluid Interfaces research group ‹http://fluid. media.mit.edu/› which develops technologies for seamless integration of the digital world and physical world. Previously, she founded and ran the Software Agents group ‹http://fluid.media.mit.edu/›. Prior to joining the Media Lab, Maes was a visiting professor and a research scientist at the MIT Artificial Intelligence Lab. She holds bachelor's and PhD degrees in computer science from the Vrije Universiteit Brussel in Belgium. Her areas of expertise are human-computer interaction and intelligent user interfaces. Maes is the editor of three books, and is an editorial board member and reviewer for numerous professional journals and conferences. She has received several awards: /Newsweek/ magazine named her one of the “100 Americans to watch for” in the year 2000; /TIME/ Digital selected her as a member of the Cyber-Elite, the top 50 technological pioneers of the hightech world; the World Economic Forum honored her with the title “Global Leader for Tomorrow”; Ars Electronica awarded her the 1995 World Wide Web category prize; and in 2000 she was recognized with the “Lifetime Achievement Award” by the Massachusetts Interactive Media Council.

In this note, we present SixthSense, a wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information. By using a tiny projector and a camera coupled in a pendant like mobile wearable device, SixthSense sees what the user sees and visually augments surfaces, walls or physical objects the user is interacting with; turning them into just-in-time information interfaces. SixthSense attempts to free information from its confines by seamlessly integrating it with the physical world.

Surfaces capable of modulating permeability have long been used in architecture for environmental control, but have remained largely unexplored as information displays. The advent of new shape changing materials and construction techniques promises to change this. In this paper, we describe Shutters: a curtain composed of actuated louvers that can be individually addressed for precise control of ventilation, daylight incidence and information display. We discuss related work, the underlying design principles behind Shutters, engineering details and application scenarios in architecture and fashion. We conclude with a comparative visual study for the use of permeability in kinetic and shadow displays and provide directions for future work.

We present 'Quickies: Intelligent Sticky Notes', an attempt to bring one of the most useful inventions of the 20th century into the digital age: the ubiquitous sticky notes. Sticky notes help us manage our-to-do lists, tag our objects and documents and capture short reminders or information that we may need in the near future. 'Quickies' enrich the experience of using sticky notes by allowing them to be tracked and managed more effectively. Quickies are stickies that have intelligence and the ability to remind us about the task we ought to perform or to provide us at the right time with the information we captured in the past. The project explores how the use of Artificial Intelligence, RFID, and ink recognition technologies can make it possible to create intelligent sticky notes that can be searched, located, can send reminders and messages, and more broadly, can help us to seamlessly connect our physical and digital experiences.

In this paper we describe Sprout I/O, a novel haptic interface for tactile and visual communication. Sprout I/O combines textiles and shape-memory alloys to create a soft and kinetic membrane with truly co-located input and output. We describe implementation details, the affordances made possible by the use of smart materials in human computer interaction and possible applications for this technology.

Mobile communication devices, such as mobile phones and networked personal digital assistants (PDAs), allow users to be constantly connected and communicate anywhere and at any time, often resulting in personal and private communication taking place in public spaces. This private -- public contrast can be problematic. As a remedy, we promote intimate interfaces: interfaces that allow subtle and minimal mobile interaction, without disruption of the surrounding environment. In particular, motionless gestures sensed through the electromyographic (EMG) signal have been proposed as a solution to allow subtle input in a mobile context. In this paper we present an expansion of the work on EMG-based motionless gestures including (1) a novel study of their usability in a mobile context for controlling a realistic, multimodal interface and (2) a formal assessment of how noticeable they are to informed observers. Experimental results confirm that subtle gestures can be profitably used within a multimodal interface and that it is difficult for observers to guess when someone is performing a gesture, confirming the hypothesis of subtlety.

This paper outlines Siftables, a novel platform that applies technology and methodology from wireless sensor networks to tangible user interfaces in order to yield new possibilities for human-computer interaction. Siftables are compact devices with sensing, graphical display, and wireless communication. They can be physically manipulated as a group to interact with digital information and media. We discuss the unique affordances that a sensor network user interface (SNUI) such as Siftables provides, as well as the resulting directness between the physical interface and the data being manipulated. We conclude with a description of some gestural language primitives that we are currently prototyping with Siftables.

In this paper we present the linguistic design aspects subTextile, a hardware platform and visual programming language created with the goal of distilling the domain of programming to a set of primitives that would allow novice users to create dynamic and interesting behaviors for interactive art, architecture, and industrial design artifacts. Design exists in every aspect of human society, but the knowledge structure we have moved towards makes it difficult to acquire expertise in a wide set of skills. This discrepancy suggests the need for tools which encapsulate expert knowledge while allowing end-users to design using these "packaged skills," and this is what the subTextile language attempts to explore. We show an approach for distilling out a minimal set of linguistic primitives that can succinctly encapsulate the full expressive capability of a lower-level language by focusing not on the end effects, but the activity we expect the user to perform with the tool.

Mobile devices are generally used in public, where the user is surrounded by others not involved in the interaction. Audible notification cues are often a cause of unnecessary disruption and distraction both for co-located people and even for the user to whom they are directed. We present a wearable peripheral display embedded in eyeglasses that delivers subtle, discreet and unobtrusive cues. The display is personal and intimate; it delivers visual cues in the wearers' periphery without disrupting their immediate environment. A user study conducted to validate the design reveals that the display is effective and subtle in notifying users. Experimental results show, with significance, that the cues can be designed to meet specific levels of visibility and disruption for the wearer, so that some cues are less noticeable when the user is not under high workload, which is highly desirable in many practical circumstances. Hence, peripheral notification displays can provide an effective solution for designing socially acceptable notification displays, unobtrusive to the user and the immediate environment.

Popular online social networks such as Friendster and MySpace do more than simply reveal the superficial structure of social connectedness — the rich meanings bottled within social network profiles themselves imply deeper patterns of culture and taste. If these latent semantic fabrics of taste could be harvested formally, the resultant resource would afford completely novel ways for representing and reasoning about web users and people in general. This paper narrates the theory and technique of such a feat — the natural language text of 100,000 social network profiles were captured, mapped into a diverse ontology of music, books, films, foods, etc., and machine learning was applied to infer a semantic fabric of taste. Taste fabrics bring us closer to improvisational manipulations of meaning, and afford us at least three semantic functions — the creation of semantically flexible user representations, cross-domain taste-based recommendation, and the computation of taste-similarity between people — whose use cases are demonstrated within the context of three applications — the InterestMap, Ambient Semantics, and IdentityMirror. Finally, we evaluate the quality of the taste fabrics, and distill from this research reusable methodologies and techniques of consequence to the semantic mining and Semantic Web communities.

A key to improving at any task is frequent feedback from people whose opinions we care about: our family, friends, mentors, and the experts. However, such input is not usually available from the right people at the time it is needed most, and attaining a deep understanding of someone else's perspective requires immense effort. This paper introduces a technological solution.We present a novel method for automatically modeling a person's attitudes and opinions, and a proactive interface called "What Would They Think?" which offers the just-in-time perspectives of people whose opinions we care about, based on whatever the user happens to be reading or writing. In the application, each person is represented by a "digital persona," generated from an automated analysis of personal texts (e.g. weblogs and papers written by the person being modeled) using natural language processing and commonsense-based textual-affect sensing.In user studies, participants using our application were able to grasp the personalities and opinions of a panel of strangers more quickly and deeply than with either of two baseline methods. We discuss the theoretical and pragmatic implications of this research to intelligent user interfaces.

In the near future, dynamic pricing will be a common competitive maneuver. In this age of digital markets, sellers in electronic marketplaces can implement automated and frequent adjustments to prices and can easily imagine how this will increase their revenue by selling to buyers "at the right time, at the right price." But at present, most sellers do not have an adequate understanding of the performance of dynamic pricing algorithms in their marketplaces. This paper addresses this concern by analyzing the performance of two adaptive pricing algorithms. We study the behavior of these algorithms within the Learning Curve Simulator, a platform for analyzing dynamic pricing strategies in finite markets assuming various buyer behaviors. The goals of our research are twofold: (i) to explore the use of simulation as a tool to aid in the development of dynamic pricing strategies; and (ii) to explicitly identify the market conditions under which our example strategies, Goal-Directed and Derivative-Following, are successful.

This paper examines seller strategies for dynamic pricing in an auction-driven marketplace.Specifically, this paper focuses on the airline industry, a field experienced in demand forecasting and dynamic pricing capabilities.Our goal is to discover the relevant factors when a seller dynamically evaluates incoming bids on a finite number of goods.We present two adaptive pricing strategies and evaluate them using a market simulator.

This paper describes MARI (Multi-Attribute Resource Intermediary), a project which proposes to improve online marketplaces, specifically those that involve the buying and selling of non-tangible goods and services. MARI is an intermediary architecture intended as a generalized platform for the specification and brokering of heterogeneous goods and services. MARI makes it possible for both buyers and sellers alike to more holistically and comprehensively specify relative preferences for the transaction partner, as well as for the attributes of the product in question, making price just one of a multitude of possible factors influencing the decision to trade. Ultimately, we expect that the ability to make such specifications will result in a more efficient, richer, and integrative transaction experience.

Inspired by Hill and Hollans original work [7], we have been developing a theory of interaction history and building tools to apply this theory to navigation in a complex information space. We have built a series of tools - map, paths, annota- tions and signposts - based on a physical-world navigation metaphor. These tools have been in use for over a year. Our user study involved a controlled browse task and showed that users were able to get the same amount of work done with significantly less effort.

The Internet enables groups of people throughout the world to interact to discuss issues, get assistance, learn, and socialize. However, when there are thousands of loosely defined groups in which a user could potentially participate, the problem becomes finding the groups of most interest. In this paper we focus on the domain of Internet Relay Chat real-time text messaging, and describe a “social butterfly” agent called Butterfly that samples available conversational groups and recommends ones of interest. We discuss Butterfly’s motivation, usage, realworld design constraints, implementation, and results. Finally, we introduce work in progress on a multi-agent approach that has grown out of our experience with Butterfly.

Article Intelligent software Share on Author: Pattie Maes MIT Media Laboratory, E15-305, 20 Ames Street, Cambridge MA MIT Media Laboratory, E15-305, 20 Ames Street, Cambridge MAView Profile Authors Info & Claims IUI '97: Proceedings of the 2nd international conference on Intelligent user interfacesJanuary 1997 Pages 41–43https://doi.org/10.1145/238218.238283Online:06 January 1997Publication History 11citation716DownloadsMetricsTotal Citations11Total Downloads716Last 12 Months36Last 6 weeks4 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

The cumbersome nature of wired interfaces and the limited nature of the interaction with graphical objects has so far limited the range of application of virtual environments. We discuss the design and implementation of a novel system, called ALIVE, which allows wireless full-body interaction between a human participant and a rich graphical world inhabited by autonomous agents. Based on results obtained with real users, the paper argues that this kind of system can provide more complex and very different experiences than traditional virtual reality systems. The ALIVE system significantly broadens the range of potential applications of virtual reality systems; in particular the paper discusses novel applications in the area of training and teaching, entertainment and last but not least, digital assistants or interface agents.>

This paper describes a technique for making personalized recommendations from any type of database to a user based on similarities between the interest profile of that user and those of other users.In particular, we discuss the implementation of a networked system called Ringo, which makes personalized recommendations for music albums and artists.Ringo's database of users and artists grows dynamically as more people use the system and enter more information.Four different algorithms for making recommendations by using social information filtering were tested and compared.We present quantitative and qualitative results obtained from the use of Ringo by more than 2000 people.

No abstract available.

Traditional machine vision assumes that the vision system recovers a complete, labeled description of the world [10]. Recently, several researchers have criticized this model and proposed an alternative model that considers perception as a distributed collection of task-specific, context-driven visual routines [1, 12]. Some of these researchers have argued that in natural living systems these visual routines are the product of natural selection [11]. So far, researchers have hand-coded task-specific visual routines for actual implementations (e.g., [3]). In this article we propose an alternative approach in which visual routines for simple tasks are created using an artificial evolution approach. We present results from a series of runs on actual camera images, in which simple routines were evolved using genetic programming techniques [7]. The results obtained are promising: The evolved routines are able to process correctly up to 93% of the test images, which is better than any algorithm we were able to write by hand.

One category of research in Artificial Life is concerned with modeling and building so-called adaptive autonomous agents, which are systems that inhabit a dynamic, unpredictable environment in which they try to satisfy a set of time-dependent goals or motivations. Agents are said to be adaptive if they improve their competence at dealing with these goals based on experience. Autonomous agents constitute a new approach to the study of Artificial Intelligence (AI), which is highly inspired by biology, in particular ethology, the study of animal behavior. Research in autonomous agents has brought about a new wave of excitement into the field of AI. This paper reflects on the state of the art of this new approach. It attempts to extract its main ideas, evaluates what contributions have been made so far, and identifies its current limitations and open problems.

This paper brings some perspective to various concepts in computational reflection. A definition of computational reflection is presented, the importance of computational reflection is discussed and the architecture of languages that support reflection is studied. Further, this paper presents a survey of some experiments in reflection which have been performed. Examples of existing procedural, logic-based and rule-based languages with an architecture for reflection are briefly presented. The main part of the paper describes an original experiment to introduce a reflective architecture in an object-oriented language. It stresses the contributions of this language to the field of object-oriented programming and illustrates the new programming style made possible. The examples show that a lot of programming problems that were previously handled on an ad hoc basis, can in a reflective architecture be solved more elegantly.