Chronic liver disease can lead to neurological conditions that result in coma or death. Although early detection can allow for intervention, testing is infrequent and unstandardized. Beacon is a device for at-home patient self-measurement of cognitive function via critical flicker frequency, which is the frequency at which a flickering light appears steady to an observer. This paper presents our efforts in iterating on Beacon's hardware and software to enable at-home use, then reports on an at-home deployment with 21 patients taking measurements over 6 weeks. We found that measurements were stable despite being taken at different times and in different environments. Finally, through interviews with 15 patients and 5 hepatologists, we report on participant experiences with Beacon, preferences around how CFF data should be presented, and the role of caregivers in helping patients manage their condition. Informed by our experiences with Beacon, we further discuss design implications for home health devices.

We present AdHocProx, a system that uses device-relative, inside-out sensing to augment co-located collaboration across multiple devices, without recourse to externally-anchored beacons – or even reliance on WiFi connectivity.

Speech is inappropriate in many situations, limiting when voice control can be used. Most unvoiced speech text entry systems can not be used while on-the-go due to movement artifacts. Using a dental retainer with capacitive touch sensors, SilentSpeller tracks tongue movement, enabling users to type by spelling words without voicing. SilentSpeller achieves an average 97% character accuracy in offline isolated word testing on a 1164-word dictionary. Walking has little effect on accuracy; average offline character accuracy was roughly equivalent on 107 phrases entered while walking (97.5%) or seated (96.5%). To demonstrate extensibility, the system was tested on 100 unseen words, leading to an average 94% accuracy. Live text entry speeds for seven participants averaged 37 words per minute at 87% accuracy. Comparing silent spelling to current practice suggests that SilentSpeller may be a viable alternative for silent mobile text entry.

Deviant eating behavior such as skipping meals and consuming unhealthy meals has a significant association with mental well-being in college students. However, there is more to what an individual eats. While eating patterns form a critical component of their mental well-being, insights and assessments related to the interplay of eating patterns and mental well-being remain under-explored in theory and practice. To bridge this gap, we use an existing real-time eating detection system that captures context during meals to examine how college students’ eating context associates with their mental well-being, particularly their affect, anxiety, depression, and stress. Our findings suggest that students’ irregularity or skipping meals negatively correlates with their mental well-being, whereas eating with family and friends positively correlates with improved mental well-being. We discuss the implications of our study in designing dietary intervention technologies and guiding student-centric well-being technologies.

Debugging printed circuit boards (PCBs) requires frequent context switching and spatial pattern matching between software design files and physical boards. To reduce this overhead, we conduct a series of interviews with electrical engineers to understand their workflows, around which we design a set of AR interaction techniques, we call Augmented Silkscreen, to streamline identification, localization, annotation, and measurement tasks. We then run a set of remote user studies with illustrative video sketches and simulated PCB tasks to compare our interactions with current practices, finding that our techniques reduce completion times. Based on these quantitative results, as well as qualitative feedback from our participants, we offer design recommendations for the implementation of these interactions on a future, deployable AR system.

As wearable devices move toward the face (i.e. smart earbuds, glasses), there is an increasing need to facilitate intuitive interactions with these devices. Current sensing techniques can already detect many mouth-based gestures; however, users’ preferences of these gestures are not fully understood. In this paper, we investigate the design space and usability of mouth-based microgestures. We first conducted brainstorming sessions (N=16) and compiled an extensive set of 86 user-defined gestures. Then, with an online survey (N=50), we assessed the physical and mental demand of our gesture set and identified a subset of 14 gestures that can be performed easily and naturally. Finally, we conducted a remote Wizard-of-Oz usability study (N=11) mapping gestures to various daily smartphone operations under a sitting and walking context. From these studies, we develop a taxonomy for mouth gestures, finalize a practical gesture set for common applications, and provide design guidelines for future mouth-based gesture interactions.

Gaze tracking is an essential component of next generation displays for virtual reality and augmented reality applications. Traditional camera-based gaze trackers used in next generation displays are known to be lacking in one or multiple of the following metrics: power consumption, cost, computational complexity, estimation accuracy, latency, and form-factor. We propose the use of discrete photodiodes and light-emitting diodes (LEDs) as an alternative to traditional camera-based gaze tracking approaches while taking all of these metrics into consideration. We begin by developing a rendering-based simulation framework for understanding the relationship between light sources and a virtual model eyeball. Findings from this framework are used for the placement of LEDs and photodiodes. Our first prototype uses a neural network to obtain an average error rate of 2.67° at 400 Hz while demanding only 16 mW. By simplifying the implementation to using only LEDs, duplexed as light transceivers, and more minimal machine learning model, namely a light-weight supervised Gaussian process regression algorithm, we show that our second prototype is capable of an average error rate of 1.57° at 250 Hz using 800 mW.

Washing hands is one of the easiest yet most effective ways to prevent spreading illnesses and diseases. However, not adhering to thorough handwashing routines is a substantial problem worldwide. For example, in hospital operations lack of hygiene leads to healthcare associated infections. We present WristWash, a wrist-worn sensing platform that integrates an inertial measurement unit and a Hidden Markov Model-based analysis method that enables automated assessments of handwashing routines according to recommendations provided by the World Health Organization (WHO). We evaluated Wrist-Wash in a case study with 12 participants. WristWash is able to successfully recognize the 13 steps of the WHO handwashing procedure with an average accuracy of 92% with user-dependent models, and with 85% for user-independent modeling. We further explored the system's robustness by conducting another case study with six participants, this time in an unconstrained environment, to test variations in the hand-washing routine and to show the potential for real-world deployments.

We extend the interaction space of low-cost mobile virtual reality (VR) by introducing bidirectional scrolling and discrete selection using magnetic sensing. Our design uses the original Google Cardboard v1 input components, modifying only the cardboard mounted on the side. Users slide the magnetized washer around a circular track on the outer layer, which drags a magnet on the inner layer across asymmetric patterned ridges. The phone's magnetometer detects the position of the magnet as it moves around the track and slots into each ridge, emulating a click wheel. The phone's accelerometer is used to recognize center button taps. We compare our system against the current best practice (gaze) with 12 participants across four VR navigation and selection tasks. Finally, we demonstrate our system robustly handles continuous input, despite some minor deterioration of the cardboard, using a motorized rig over an 8-hour period.

Teleconferencing is touted to be one of the main and most powerful uses of virtual reality (VR). While subtle facial movements play a large role in human-to-human interactions, current work in the VR space has focused on identifying discrete emotions and expressions through coarse facial cues and gestures. By tracking and representing the fluid movements of facial elements as continuous range values, users are able to more fully express themselves. In this work, we present Buccal, a simple yet effective approach to inferring continuous lip and jaw motions by measuring deformations of the cheeks and temples with only 5 infrared proximity sensors embedded in a mobile VR headset. The signals from these sensors are mapped to facial movements through a regression model trained with ground truth labels recorded from a webcam. For a streamlined user experience, we train a user independent model that requires no setup process. Finally, we demonstrate the use of our technique to manipulate the lips and jaw of a 3D face model in real-time.

In a piano lesson, a student often imitates the teacher--s playing in terms of speed, dynamics, and fingering. While this learning model leverages one's visual and even audial perception for emulation, it still lacks an important component of piano playing -- the tactile sensation. We seek to convey the tactile sensations of the teacher's keystrokes and then signal the student's corresponding fingers. We implemented an instrumented fingerless glove called Tactile Teacher to detect finger taps on hard surfaces. Since finger taps generate acoustic signals and cause vibrations, we embedded three vibration sensors on the glove and use machine learning algorithms to analyze the data from the sensors. After a brief training procedure, this prototype can accurately identify single finger tap in a very good performance at above 89% accuracy, and two finger taps resulted in accuracy around 85%.