EMA systems.

We describe a smartphone/smartwatch system to evaluate anomia in individuals with aphasia by using audio-recording-based ecological momentary assessments. The system delivers object-naming assessments to a participant's smartwatch, whereby a prompt signals the availability of images of these objects on the watch screen. Participants attempt to speak the names of the images that appear on the watch display out loud and into the watch as they go about their lives. We conducted a three-week feasibility study with six participants with mild to moderate aphasia. Participants were assigned to either a nine-item (four prompts per day with nine images) or single-item (36 prompts per day with one image each) ecological momentary assessment protocol. Compliance in recording an audio response to a prompt was approximately 80% for both protocols. Qualitative analysis of the participants' interviews suggests that the participants felt capable of completing the protocol, but opinions about using a smartwatch were mixed. We review participant feedback and highlight the importance of considering a population's specific cognitive or motor impairments when designing technology and training protocols.

Spinal cord injury (SCI) affects the mobility of 250,000 people per year worldwide. Physical activity (PA) in individuals with SCI is positively associated with improved mental and physical health outcomes. Mobile technologies have been developed to motivate individuals with SCI to increase PA using activity tracking and real-time feedback. We conducted semi-structured interviews and participatory design sessions with 15 manual wheelchair users with SCI and eight of their family members/friends to investigate user impressions of future technologies that might use computer-mediated, sensor-triggered communication to motivate PA. We assessed barriers to PA and how context-aware communication could help overcome them. Participants with SCI expressed that PA tracking and communication technologies must be tailored to their specific needs. Further analysis revealed that context-aware messaging could help participants with SCI connect with others to initiate timely conversations about overcoming PA barriers, and to provide encouragement to meet their PA goals. We discuss opportunities to empower individuals with SCI with regards to PA using tailored, context-aware communication.

Intersubject variability in accelerometer-based activity recognition may significantly affect classification accuracy, limiting a reliable extension of methods to new users. In this paper, we propose an approach for personalizing classification rules to a single person. We demonstrate that the method improves activity detection from wrist-worn accelerometer data on a four-class recognition problem of interest to the exercise science community, where classes are ambulation, cycling, sedentary, and other. We extend a previously published activity classification method based on support vector machines so that it estimates classification uncertainty. Uncertainty is used to drive data label requests from the user, and the resulting label information is used to update the classifier. Two different datasets-one from 33 adults with 26 activity types, and another from 20 youth with 23 activity types-were used to evaluate the method using leave-one-subject-out and leave-one-group-out cross validation. The new method improved overall recognition accuracy up to 11% on average, with some large person-specific improvements (ranging from -2% to +36%). The proposed method is suitable for online implementation supporting real-time recognition systems.

data collection for assessment of behaviors, states, and contexts. Questions are prompted during everyday life using an individual's mobile device, thereby reducing recall bias and increasing validity over other self-report methods such as retrospective recall. We describe a microinteraction-based EMA method ("micro" EMA, or μEMA) using smartwatches, where all EMA questions can be answered with a quick glance and a tap - nearly as quickly as checking the time on a watch. A between-subjects, 4-week pilot study was conducted where μEMA on a smartwatch (n=19) was compared with EMA on a phone (n=14). Despite an ≈8 times increase in the number of interruptions, μEMA had a significantly higher compliance rate, completion rate, and first prompt response rate, and μEMA was perceived as less distracting. The temporal density of data collection possible with μEMA could prove useful in ubiquitous computing studies.

This paper extends previous work automatically detecting stereotypical motor movements (SMM) in individuals on the autism spectrum. Using three-axis accelerometer data obtained through wearable wireless sensors, we compare recognition results for two different classifiers -- Support Vector Machine and Decision Tree -- in combination with different feature sets based on time-frequency characteristics of accelerometer data. We use data collected from six individuals on the autism spectrum who participated in two different studies conducted three years apart in classroom settings, and observe an average accuracy across all participants over time ranging from 81.2% (TPR: 0.91; FPR: 0.21) to 99.1% (TPR: 0.99; FPR: 0.01) for all combinations of classifiers and feature sets. We also provide analyses of kinematic parameters associated with observed movements in an attempt to explain classifier-feature specific performance. Based on our results, we conclude that real-time, person-dependent, adaptive algorithms are needed in order to accurately and consistently measure SMM automatically in individuals on the autism spectrum over time in real-word settings.

Accurate, real-time measurement of energy expended during everyday activities would enable development of novel health monitoring and wellness technologies. A technique using three miniature wearable accelerometers is presented that improves upon state-of-the-art energy expenditure (EE) estimation. On a dataset acquired from 24 subjects performing gym and household activities, we demonstrate how knowledge of activity type, which can be automatically inferred from the accelerometer data, can improve EE estimates by more than 15% when compared to the best estimates from other methods.

Individuals with Autism Spectrum Disorders (ASD) frequently engage in stereotyped and repetitive motor movements. Automatically detecting these movements in real-time using comfortable, miniature wireless sensors could advance autistic research and enable new intervention tools for the classroom that help children and their caregivers monitor and cope with this potentially problematic class of behavior. We present activity recognition results for stereotypical hand flapping and body rocking using data collected from six children with ASD repeatedly observed in both laboratory and classroom settings. In the classroom, an overall recognition accuracy of 88.6% (TP: 0.85; FP: 0.08) was achieved using three sensors. Challenges encountered when applying machine learning to this domain, as well as implications for the development of real-time classroom interventions and research tools, are discussed.

Stereotypical motor movements are one of the most common and least understood behaviors occurring in individuals with Autism Spectrum Disorder (ASD). Problems with traditional methods for measuring movement stereotypy make it difficult to accurately determine when and why these behaviors occur. The current research overcomes previous measurement problems by utilizing wireless accelerometers and pattern recognition software to automatically and reliably detect stereotypical motor movements such as body rocking and hand flapping in children with ASD.

The potential for sensor-enabled mobile devices to proactively present information when and where users need it ranks among the greatest promises of ubiquitous computing. Unfortunately, mobile phones, PDAs, and other computing devices that compete for the user's attention can contribute to interruption irritability and feelings of information overload. Designers of mobile computing interfaces, therefore, require strategies for minimizing the perceived interruption burden of proactively delivered messages. In this work, a context-aware mobile computing device was developed that automatically detects postural and ambulatory activity transitions in real time using wireless accelerometers. This device was used to experimentally measure the receptivity to interruptions delivered at activity transitions relative to those delivered at random times. Messages delivered at activity transitions were found to be better received, thereby suggesting a viable strategy for context-aware message delivery in sensor-enabled mobile computing devices.

Ubiquitous, context-aware computer systems may ultimately enable computer applications that naturally and usefully respond to a user's everyday activity. Although new algorithms that can automatically detect context from wearable and environmental sensor systems show promise, many of the most flexible and robust systems use probabilistic detection algorithms that require extensive libraries of training data with labeled examples. In this paper, we describe the need for such training data and some challenges we have identified when trying to collect it while testing three context-detection systems for ubiquitous computing and mobile applications.

Healthcare systems in developed countries are experiencing severe financial stress as age demographics shift upward, leading to a larger percentage of older adults needing care. One way to potentially reduce or slow spiraling medical costs is to use technology, not only to cure sickness, but also to promote wellness throughout all stages of life, thereby avoiding or deferring expensive medical treatments. Ubiquitous computing and context-aware algorithms offer a new healthcare opportunity and a new set of research challenges: exploiting emerging consumer electronic devices to motivate healthy behavior as people age by presenting "just-in-time" information at points of decision and behavior.

A probabilistic framework for representing and visually recognizing complex multi-agent action is presented. Motivated by work in model-based object recognition and designed for the recognition of action from visual evidence, the representation has three components: (1) temporal structure descriptions representing the temporal relationships between agent goals, (2) belief networks for probabilistically representing and recognizing individual agent goals from visual evidence, and (3) belief networks automatically generated from the temporal structure descriptions that support the recognition of the complex action. We describe our current work on recognizing American football plays from noisy trajectory data.

A real-time tracking algorithm that uses contextual information is described. The method is capable of simultaneously tracking multiple, non-rigid objects when erratic movement and object collisions are common. A closed-world assumption is used to adaptively select and weight image features used for correspondence. Results of algorithm testing and the limitations of the method are discussed. The algorithm has been used to track children in an interactive, narrative playspace.

A new approach to tracking weakly modeled objects in a semantically rich domain is presented. We define a closed-world as a space-time region of an image sequence in which the complete taxonomy of objects is known, and in which each pixel should be explained as belonging to one of those objects. Given contextual object information, context-specific features can be dynamically selected as the basis for tracking. A context-specific feature is one that has been chosen based upon the context to maximize the chance of successful tracking between frames. Our work is motivated by the goal of video annotation-the semi-automatic generation of symbolic descriptions of action taking place in a contextually-rich dynamic scene. We describe how contextual knowledge in the "football domain" can be applied to closed-world football player tracking and present the details of our implementation. We include tracking results based on hundreds of images that demonstrate the wide range of tracking situations the algorithm successfully handles as well as a few examples of where the algorithm fails.>

A method for incorporating intensity edge information into the recovery of occlusion regions using a pixel-based stereo algorithm is presented. The authors review the construction of disparity space images and their use to solve the stereo occlusion problem. The authors show the relationship between intensity edges and the disparity space images, and extend their stereo technique to use information about intensity discontinuities at occlusion edges. The combination of ground control points and edge information yields excellent occlusion regions.