OBJECTIVE: Digital behavior change interventions (DBCIs) are feasibly effective tools for addressing physical activity. However, in-depth understanding of participants' long-term engagement with DBCIs remains sparse. Since the effectiveness of DBCIs to impact behavior change depends, in part, upon participant engagement, there is a need to better understand engagement as a dynamic process in response to an individual's ever-changing biological, psychological, social, and environmental context. METHODS: The year-long micro-randomized trial (MRT) HeartSteps II provides an unprecedented opportunity to investigate DBCI engagement among ethnically diverse participants. We combined data streams from wearable sensors (Fitbit Versa, i.e., walking behavior), the HeartSteps II app (i.e. page views), and ecological momentary assessments (EMAs, i.e. perceived intrinsic and extrinsic motivation) to build the idiographic models. A system identification approach and a fluid analogy model were used to conduct autoregressive with exogenous input (ARX) analyses that tested hypothesized relationships between these variables inspired by Self-Determination Theory (SDT) with DBCI engagement through time. RESULTS: Data from 11 HeartSteps II participants was used to test aspects of the hypothesized SDT dynamic model. The average age was 46.33 (SD=7.4) years, and the average steps per day at baseline was 5,507 steps (SD=6,239). The hypothesized 5-input SDT-inspired ARX model for app engagement resulted in a 31.75 % weighted RMSEA (31.50 % on validation and 31.91 % on estimation), indicating that the model predicted app page views almost 32 % better relative to the mean of the data. Among Hispanic/Latino participants, the average overall model fit across inventories of the SDT fluid analogy was 34.22 % (SD=10.53) compared to 22.39 % (SD=6.36) among non-Hispanic/Latino Whites, a difference of 11.83 %. Across individuals, the number of daily notification prompts received by the participant was positively associated with increased app page views. The weekend/weekday indicator and perceived daily busyness were also found to be key predictors of the number of daily application page views. CONCLUSIONS: This novel approach has significant implications for both personalized and adaptive DBCIs by identifying factors that foster or undermine engagement in an individual's respective context. Once identified, these factors can be tailored to promote engagement and support sustained behavior change over time.

Diverse disciplines, including Human-Computer Interaction have explored how mobile health (mHealth) applications can transform healthcare and health promotion. Increasingly, research has explored how mHealth tools can promote healthy behaviors within vulnerable populations-groups that disproportionately experience barriers to wellness. We conducted a systematic review of 83 papers from diverse disciplines to characterize the design and impact of mHealth tools in low-socioeconomic (low-SES) and racial/ethnic minority individuals. Our findings highlight that the diversity within low-SES and racial/ethnic minority groups was not reflected in the populations studied. Most studies focused on improving the health of individuals, often neglecting factors at the community and society levels that influence health disparities. Moreover, few improvements in health outcomes were demonstrated. We further discuss factors that acted as barriers and facilitators of mHealth intervention adoption. Our findings highlight trends that can drive critically needed digital health innovations for vulnerable populations.

Although the positive effects of exercise on the well-being and quality of independent living for older adults are well accepted, many elderly individuals lack access to exercise facilities, or the skills and motivation to perform exercise at home. To provide a more engaging environment that promotes physical activity, various fitness applications have been proposed. Many of the available products, however, are geared toward a younger population and are not appropriate or engaging for an older population. To address these issues, we developed an automated interactive exercise coaching system using the Microsoft Kinect. The coaching system guides users through a series of video exercises, tracks and measures their movements, provides real-time feedback, and records their performance over time. Our system consists of exercises to improve balance, flexibility, strength, and endurance, with the aim of reducing fall risk and improving performance of daily activities. In this paper, we report on the development of the exercise system, discuss the results of our recent field pilot study with six independently living elderly individuals, and highlight the lessons learned relating to the in-home system setup, user tracking, feedback, and exercise performance evaluation.

In principle, connected sensors allow effortless long-term self-monitoring of health and wellness that can help maintain health and quality of life. However, data collected in the “wild” may be noisy and contain outliers, e.g., due to uncontrolled sources or data from different persons using the same device. The removal of the “outliers” is therefore critical for accurate interpretation of the data. In this paper we study the detection and elimination of outliers in self-weighing time series data obtained from connected weight scales. We examined three techniques: (1) a method based on autoregressive integrated moving average (ARIMA) time series modelling, (2) median absolute deviation (MAD) scale estimate, and (3) a method based on Rosner statistics. We applied these methods to both a data set with real outliers and a clean data set corrupted with simulated outliers. The results suggest that the simple MAD algorithm and ARIMA performed well with both test sets while the Rosner statistics was significantly less effective. In addition, the ARIMA approach appeared to be significantly less sensitive to long periods of missing data than MAD and Rosner statistics.

Sensory-motor performance is indicative of both cognitive and physical function. The Halstead-Reitan finger tapping test is a measure of sensory-motor speed commonly used to assess function as part of a neuropsychological evaluation. Despite the widespread use of this test, the underlying motor and cognitive processes driving tapping behavior during the test are not well characterized or understood. This lack of understanding may make clinical inferences from test results about health or disease state less accurate because important aspects of the task such as variability or fatigue are unmeasured. To overcome these limitations, we enhanced the tapper with a sensor that enables us to more fully characterize all the aspects of tapping. This modification enabled us to decompose the tapping performance into six component phases and represent each phase with a set of parameters having clear functional interpretation. This results in a set of 29 total parameters for each trial, including change in tapping over time, and trial-to-trial and tap-to-tap variability. These parameters can be used to more precisely link different aspects of cognition or motor function to tapping behavior. We demonstrate the benefits of this new instrument with a simple hypothesis-driven trial comparing single and dual-task tapping.

Early and reliable detection of cognitive decline is one of the most important challenges of current healthcare. In this project, we developed an approach whereby a frequently played computer game can be used to assess a variety of cognitive processes and estimate the results of the pen-and-paper trail making test (TMT)--known to measure executive function, as well as visual pattern recognition, speed of processing, working memory, and set-switching ability. We developed a computational model of the TMT based on a decomposition of the test into several independent processes, each characterized by a set of parameters that can be estimated from play of a computer game designed to resemble the TMT. An empirical evaluation of the model suggests that it is possible to use the game data to estimate the parameters of the underlying cognitive processes and using the values of the parameters to estimate the TMT performance. Cognitive measures and trends in these measures can be used to identify individuals for further assessment, to provide a mechanism for improving the early detection of neurological problems, and to provide feedback and monitoring for cognitive interventions in the home.

Loneliness is a common condition in elderly associated with severe health consequences including increased mortality, decreased cognitive function, and poor quality of life. Identifying and assisting lonely individuals is therefore increasingly important-especially in the home setting-as the very nature of loneliness often makes it difficult to detect by traditional methods. One critical component in assessing loneliness unobtrusively is to measure time spent out-of-home, as loneliness often presents with decreased physical activity, decreased motor functioning, and a decline in activities of daily living, all of which may cause decrease in the amount of time spent outside the home. Using passive and unobtrusive in-home sensing technologies, we have developed a methodology for detecting time spent out-of-home based on logistic regression. Our approach was both sensitive (0.939) and specific (0.975) in detecting time out-of-home across over 41,000 epochs of data collected from four subjects monitored for at least 30 days each in their own homes. In addition to linking time spent out-of-home to loneliness, (r = -0.44, p = 0.011) as measured by the UCLA Loneliness Index, we demonstrate its usefulness in other applications such as uncovering general behavioral patterns of elderly and exploring the link between time spent out-of-home and physical activity ( r = 0.415, p = 0.031), as measured by the Berkman Social Disengagement Index.

Unexpected stimuli are a challenge to any machine learning algorithm. Here, we identify distinct types of unexpected events when general-level and specific-level classifiers give conflicting predictions. We define a formal framework for the representation and processing of incongruent events: Starting from the notion of label hierarchy, we show how partial order on labels can be deduced from such hierarchies. For each event, we compute its probability in different ways, based on adjacent levels in the label hierarchy. An incongruent event is an event where the probability computed based on some more specific level is much smaller than the probability computed based on some more general level, leading to conflicting predictions. Algorithms are derived to detect incongruent events from different types of hierarchies, different applications, and a variety of data types. We present promising results for the detection of novel visual and audio objects, and new patterns of motion in video. We also discuss the detection of Out-Of- Vocabulary words in speech recognition, and the detection of incongruent events in a multimodal audiovisual scenario.

Quality of sleep is an important attribute of an individual's health state and its assessment is therefore a useful diagnostic feature. Changes in the patterns of motor activities during sleep can be a disease marker, or can reflect various abnormal physiological and neurological conditions. Presently, there are no convenient, unobtrusive ways to assess quality of sleep outside of a clinic. This paper describes a system for unobtrusive detection of movement in bed that uses load cells installed at the corners of a bed. The system focuses on identifying when a movement occurs based on the forces sensed by the load cells. The movement detection approach estimates the energy in each load cell signal over short segments to capture the variations caused by movement. The accuracy of the detector is evaluated using data collected in the laboratory. The detector is capable of detecting voluntary movements in bed while the subjects were awake, with an average equal error rate of 3.22% (+/-0.54). Its performance is invariant with respect to the individual's characteristics, e.g., weight, as well as those of the bed. The simplicity of the resulting algorithms and their relative insensitivity to the weight and height of the monitored individual make the approach practical and easily deployable in residential and clinical settings.

The human neural responses associated with cognitive events, referred as event related potentials (ERPs), can provide reliable inference for target image detection. Incremental learning has been widely investigated to deal with large datasets. To solve the problem of data growing over time in cross session studies, we apply an incremental learning support vector machines (SVM) method on single-trial ERP detection for identifying targets in satellite images. We implement the incremental learning SVM by keeping only the support vectors, instead of all the data, from the previous sessions and incorporating them with the data of the current session. Thus the incremental learning dramatically reduces the computational load. The results demonstrate that the incremental learning ERP detection system performs as well as the naive method, which uses only the current training session, and the batch mode, which uses all training data. Furthermore, it is more computationally efficient, which allows it to better cope with a continuous stream of EEG data.

This paper focuses on resolving a number of issues that appear when the performance of human speech recognition is compared to that of automatic speech recognition. In particular human experimental data suggest that the resulting error is a product of the individual streams. On the other hand, Bayesian combination requires a multiplication of the estimates of prior probabilities and likelihoods. We show that, in principle, there is no discrepancy. The product of errors is a performance measure and human and machine performance may be consistent with this empirically established regularity. The product of probabilities is step in an algorithm to achieve the performance that may or may not be consistent with the product of errors. The main problem is that most of prior discussions failed to distinguish the performance measures from the estimates of the parameters used in the algorithm.

This paper describes an approach for target image search using human brain signals generated by perceptual processes in the brain. The human brain generates event related potentials (ERPs) in response to critical events, such as interesting/novel visual stimuli in the form of a target image. In this paper, we describe experiments involving six professional image analysts and summarize the ERP detection performance as they search for targets within a large image database. We develop a disjoint windowing scheme for data preprocessing to discard irrelevant and redundant information from the raw data to get clean training data. We apply support vector machines to detect ERPs and conduct 10-fold cross validation for parameter regularization. The results demonstrate that the ERP pattern recognition can provide reliable inference for image triage.

It is of prime importance in everyday human life to cope with and respond appropriately to events that are not foreseen by prior experience. Machines to a large extent lack the ability to respond appropriately to such inputs. An important class of unexpected events is defined by incongruent combinations of inputs from different modalities and therefore multimodal information provides a crucial cue for the identification of such events, e.g., the sound of a voice is being heard while the person in the field-of-view does not move her lips. In the project DIRAC ("Detection and Identification of Rare Audio-visual Cues") we have been developing algorithmic approaches to the detection of such events, as well as an experimental hardware platform to test it. An audio-visual platform ("AWEAR" - audio-visual wearable device) has been constructed with the goal to help users with disabilities or a high cognitive load to deal with unexpected events. Key hardware components include stereo panoramic vision sensors and 6-channel worn-behind-the-ear (hearing aid) microphone arrays. Data have been recorded to study audio-visual tracking, a/v scene/object classification and a/v detection of incongruencies.

Unexpected stimuli are a challenge to any machine learning algorithm. Here we identify distinct types of unexpected events, focusing on 'incongruent events' - when 'general level' and 'specific level' classifiers give conflicting predictions. We define a formal framework for the representation and processing of incongruent events: starting from the notion of label hierarchy, we show how partial order on labels can be deduced from such hierarchies. For each event, we compute its probability in different ways, based on adjacent levels (according to the partial order) in the label hierarchy . An incongruent event is an event where the probability computed based on some more specific level (in accordance with the partial order) is much smaller than the probability computed based on some more general level, leading to conflicting predictions. We derive algorithms to detect incongruent events from different types of hierarchies, corresponding to class membership or part membership. Respectively, we show promising results with real data on two specific problems: Out Of Vocabulary words in speech recognition, and the identification of a new sub-class (e.g., the face of a new individual) in audio-visual facial object recognition.

In this paper, we describe a system for detecting encephalography (EEG) signatures of visual recognition events evoked in a single trial during rapid serial visual presentation (RSVP). In order to investigate the viability of nonlinear approaches in EEG detection and assess the performance comparison, we applied three classifiers (linear logistic regression model, Laplacian classifier, and spectral maximum mutual information projection) in the detection tasks. The EEG was recorded using 32 electrodes during the rapid image presentation (50ms/100ms per image). Subjects were instructed to push a button when they recognize a target image. The results suggest that while the detection of single trial EEG-based recognition is possible, taking advantage of the nonlinear techniques requires data representation that would overcome the non-stationarity of the EEG signals.

Power spectrum density (PSD) of electroencephalogram (EEG) signals is a widely used feature for brain computer interfaces (BCI). Usually, PSD features are integrated over different frequency bands, such as delta, theta, alpha, beta, gamma, which are based on well-established interpretations of EEG signals in prior experimental and clinical contexts. However, these predefined frequency bands do not necessarily relate to the optimal features for various BCI applications. In this paper, we propose an alternative feature dimensionality reduction method, which automatically determines the optimal number and the range of frequency bands. We applied the proposed method on EEG classification in the context of Augmented Cognition (AugCog) using BCI. The experimental results show that the proposed method can extract more robust features than features manually extracted from predefined frequency bands.

We describe a model of the auditory system in which a population of spiking neurons with limited sampling rates represents the magnitude and phase of high bandwidth auditory signals. The basic premise of this model is based on the fact that each peripheral auditory neuron appears to have a very narrow band tuning characteristics. The signal in each narrow-band channel can, therefore, be sampled at frequencies that are much lower than the center frequency of the band, e.g., < 50 Hz and consistent with the capabilities of neurons. The new idea here is that the system can use non-uniform sampling, consistent with the refractory periods of the neurons, to capture both the amplitude of the modulation and the phase of the carrier signal. The computational model described in this paper consists of a short-term FFT analysis combined with overlap-add and a sampling process where magnitude is digitized but phase is represented using a temporal code of spiking neurons. The coding/decoding mechanism is using knowledge of the properties of the refractory period. We show that this model can represent arbitrary signals, but redundant signals such as speech are represented with higher accuracy than uncorrelated noise. We note that this basic coding approach may be useful for representation of signals in situation where binary representation is not feasible.

A useful enhanced vision system (EVS) for aircraft landing guidance not only has to provide the pilots a reliable image of the scene by fusing several sensor images in real time, but should also give them additional information such as attitude, navigation and hazard signals for safe landing in all weather conditions. Here, a biologically inspired EVS is proposed. The algorithms of the core modules of the systems, namely, the pre-processing and image retrieval are discussed in this paper. A FPGA version of the association network algorithm used in the application is discussed and its performance compared with the PC system.

The U.S. has experienced a rapid growth in the use of computers by elders. E-mail, Web browsing, and computer games are among the most common routine activities for this group of users. In this paper, we describe techniques for unobtrusively monitoring naturally occurring computer interactions to detect sustained changes in cognitive performance. Researchers have demonstrated the importance of the early detection of cognitive decline. Users over the age of 75 are at risk for medically related cognitive problems and confusion, and early detection allows for more effective clinical intervention. In this paper, we present algorithms for inferring a user's cognitive performance using monitoring data from computer games and psychomotor measurements associated with keyboard entry and mouse movement. The inferences are then used to classify significant performance changes, and additionally, to adapt computer interfaces with tailored hints and assistance when needed. These methods were tested in a group of elders in a residential facility.

We describe a novel adaptive method that achieves robustness in pattern classification by combining a large number of weak classifiers. The individual classifiers are trained on subsets of features of the training samples and the output classification is obtained by a weighted sum of the individual weak classifiers. When the classifier is applied to the test set, the combination weights are adaptively adjusted in accordance with the agreement among the individual classifiers. We evaluated the performances of several different combination methods using simulated data and the results proved to be robust.

The influence that language has on contextual interpretations cannot be ignored by computer systems that strive to be context aware. Rather, once systems are designed to perceive language and other forms of human action, these interpretative processes will of necessity be context dependent. As an example, we illustrate how people simply and naturally create new contexts by naming and referring. We then describe Rasa, a mixed-reality system that observes and understands how users in a military command post create such contexts as part of the process of maintaining situational awareness. In such environments, commander's maps are covered with Post-it(r) notes. These paper artifacts are contextually transformed to represent units in the field by the application of multimodal language. Rasa understands this language, thereby allowing paper- based tools to become the basis for digital interaction. Finally, we argue that architectures for such context-aware systems will need to be built to process the inherent ambiguity and uncertainty of human input in order to be effective.

A new highly parallel approach to automatic recognition of speech, inspired by early Fletcher's research on Articulation Index, and based on independent probability estimates in several sub-bands of the available speech spectrum, is presented. The approach is especially suitable for situations when part of the spectrum of speech is corrupted. In such cases, it can yield an order-of-magnitude improvement in the error rate over a conventional full-band recognizer. 1.

Quadtrees are a compact hierarchical method of representation of images. In this paper, we explore a number of hierarchical image representations as applied to binary images, of which quadtrees are a single exemplar. We discuss quadtrees, binary trees, and an adaptive hierarchical method. Extending these methods into the third dimension of time results in several other methods. All of these methods are discussed in terms of time complexity, worst case and average compression of random images, and compression results on binary images derived from natural scenes. The results indicate that quadtrees are the most effective for two-dimensional images, but the adaptive algorithms are more effective for dynamic image sequences.