Supporting student success requires collaboration among multiple stakeholders. Researchers have explored machine learning models for academic performance prediction; yet key challenges remain in ensuring these models are interpretable, equitable, and actionable within real-world educational support systems. First, many models prioritize predictive accuracy but overlook human-centered principles, limiting trust among students and reducing their usefulness for educators and institutional decision-makers. Second, most models require at least a month of data before making reliable predictions, delaying opportunities for early intervention. Third, current models primarily rely on sporadically collected, classroom-derived data, missing broader behavioral patterns that could provide more continuous and actionable insights. To address these gaps, we present three modeling approaches-LR, 1D-CNN, and MTL-1D-CNN-to classify students as low or high academic performers. We evaluate them based on explainability , fairness , and generalizability to assess their alignment with key social values. Using behavioral and self-reported data collected within the first week of two Spring terms, we demonstrate that these models can identify at-risk students as early as week one. However, trade-offs across human-centered principles highlight the complexity of designing predictive models that effectively support multi-stakeholder decision-making and intervention strategies. We discuss these trade-offs and their implications for different stakeholders, outlining how predictive models can be integrated into student support systems. Finally, we examine broader socio-technical challenges in deploying these models and propose future directions for advancing human-centered, collaborative academic prediction systems.

Despite a rich history of investigating smartphone overuse intervention techniques, AI-based just-in-time adaptive intervention (JITAI) methods for overuse reduction are lacking. We develop Time2Stop, an intelligent, adaptive, and explainable JITAI system that leverages machine learning to identify optimal intervention timings, introduces interventions with transparent AI explanations, and collects user feedback to establish a human-AI loop and adapt the intervention model over time. We conducted an 8-week field experiment (N=71) to evaluate the effectiveness of both the adaptation and explanation aspects of Time2Stop. Our results indicate that our adaptive models significantly outperform the baseline methods on intervention accuracy (>32.8% relatively) and receptivity (>8.0%). In addition, incorporating explanations further enhances the effectiveness by 53.8% and 11.4% on accuracy and receptivity, respectively. Moreover, Time2Stop significantly reduces overuse, decreasing app visit frequency by 7.0 ∼ 8.9%. Our subjective data also echoed these quantitative measures. Participants preferred the adaptive interventions and rated the system highly on intervention time accuracy, effectiveness, and level of trust. We envision our work can inspire future research on JITAI systems with a human-AI loop to evolve with users.

Smartphone overuse is related to a variety of issues such as lack of sleep and anxiety. We explore the application of Self-Affirmation Theory on smartphone overuse intervention in a just-in-time manner. We present TypeOut, a just-in-time intervention technique that integrates two components: an in-situ typing-based unlock process to improve user engagement, and self-affirmation-based typing content to enhance effectiveness. We hypothesize that the integration of typing and self-affirmation content can better reduce smartphone overuse. We conducted a 10-week within-subject field experiment (N=54) and compared TypeOut against two baselines: one only showing the self-affirmation content (a common notification-based intervention), and one only requiring typing non-semantic content (a state-of-the-art method). TypeOut reduces app usage by over 50%, and both app opening frequency and usage duration by over 25%, all significantly outperforming baselines. TypeOut can potentially be used in other domains where an intervention may benefit from integrating self-affirmation exercises with an engaging just-in-time mechanism.

Recent research has demonstrated the capability of behavior signals captured by smartphones and wearables for longitudinal behavior modeling. However, there is a lack of a comprehensive public dataset that serves as an open testbed for fair comparison among algorithms. Moreover, prior studies mainly evaluate algorithms using data from a single population within a short period, without measuring the cross-dataset generalizability of these algorithms. We present the first multi-year passive sensing datasets, containing over 700 user-years and 497 unique users’ data collected from mobile and wearable sensors, together with a wide range of well-being metrics. Our datasets can support multiple cross-dataset evaluations of behavior modeling algorithms’ generalizability across different users and years. As a starting point, we provide the benchmark results of 18 algorithms on the task of depression detection. Our results indicate that both prior depression detection algorithms and domain generalization techniques show potential but need further research to achieve adequate cross-dataset generalizability. We envision our multi-year datasets can support the ML community in developing generalizable longitudinal behavior modeling algorithms.

Continuous passive sensing of daily behavior from mobile devices has the potential to identify behavioral patterns associated with different aspects of human characteristics. This paper presents novel analytic approaches to extract and understand these behavioral patterns and their impact on predicting adaptive and maladaptive personality traits. Our machine learning analysis extends previous research by showing that both adaptive and maladaptive traits are associated with passively sensed behavior providing initial evidence for the utility of this type of data to study personality and its pathology. The analysis also suggests directions for future confirmatory studies into the underlying behavior patterns that link adaptive and maladaptive variants consistent with contemporary models of personality pathology.

Although some research highlights the benefits of behavioral routines for individual functioning, other research indicates that routines can reflect an individual's inflexibility and lower well-being. Given conflicting accounts on the benefits of routine, research is needed to examine how routineness versus flexibility in health-related behaviors correspond to personality traits, health, and occupational outcomes. We adopt a nonlinear dynamical systems approach to understanding routine using automatically sensed health-related behaviors collected from 483 information workers over a roughly two-month period. We utilized multidimensional recurrence quantification analysis to derive a measure of health regularity (routineness) from measures of daily step count, sleep duration, and heart rate variability (which relates to stress). Participants also completed measures of personality, health, and job performance at the start of the study and for two months via Ecological Momentary Assessments. Greater regularity was associated with higher neuroticism, lower agreeableness, and greater interpersonal and organizational deviance. Importantly, these results were independent of overall levels of each health indicator in addition to demographics. It is often believed that routine is desirable, but the results suggest that associations with routineness are more nuanced, and wearable sensors can provide insights into beneficial health behaviors.

This paper presents a computational framework for modeling biobehavioral rhythms - the repeating cycles of physiological, psychological, social, and environmental events - from mobile and wearable data streams. The framework incorporates four main components: mobile data processing, rhythm discovery, rhythm modeling, and machine learning. We evaluate the framework with two case studies using datasets of smartphone, Fitbit, and OURA smart ring to evaluate the framework’s ability to (1) detect cyclic biobehavior, (2) model commonality and differences in rhythms of human participants in the sample datasets, and (3) predict their health and readiness status using models of biobehavioral rhythms. Our evaluation demonstrates the framework’s ability to generate new knowledge and findings through rigorous micro- and macro-level modeling of human rhythms from mobile and wearable data streams collected in the wild and using them to assess and predict different life and health outcomes.

We present HulaMove, a novel interaction technique that leverages the movement of the waist as a new eyes-free and hands-free input method for both the physical world and the virtual world. We first conducted a user study (N=12) to understand users’ ability to control their waist. We found that users could easily discriminate eight shifting directions and two rotating orientations, and quickly confirm actions by returning to the original position (quick return). We developed a design space with eight gestures for waist interaction based on the results and implemented an IMU-based real-time system. Using a hierarchical machine learning model, our system could recognize waist gestures at an accuracy of 97.5%. Finally, we conducted a second user study (N=12) for usability testing in both real-world scenarios and virtual reality settings. Our usability study indicated that HulaMove significantly reduced interaction time by 41.8% compared to a touch screen method, and greatly improved users’ sense of presence in the virtual world. This novel technique provides an additional input method when users’ eyes or hands are busy, accelerates users’ daily operations, and augments their immersive experience in the virtual world.

Abstract Taking an action research approach, we engaged in fieldwork with school-based behavioral health care teams to: observe record keeping practices, design and deploy a prototype system addressing key challenges, and reflect on its use. We describe the challenges of capturing behavioral data using both paper and electronic records. Creating records of behaviors requires direct observation, and as a result the record keeping responsibility is challenging to distribute across a care team. Behavioral data on paper must be transferred and prepared for reporting, both inside the organization and to stakeholders outside of the organization. In prototyping a computerized working record, we targeted user needs for capturing details of a behavioral incident in the moment. Challenges persisted through the transition from paper to our prototype, and based on these empirical findings over two years of fieldwork, we present five tensions in representing behavioral data in an electronic health record. These tensions reflect the differences between entering behavioral data into the record for intraorganizational use versus interorganizational use.

We present a machine learning approach that uses data from smartphones and fitness trackers of 138 college students to identify students that experienced depressive symptoms at the end of the semester and students whose depressive symptoms worsened over the semester. Our novel approach is a feature extraction technique that allows us to select meaningful features indicative of depressive symptoms from longitudinal data. It allows us to detect the presence of post-semester depressive symptoms with an accuracy of 85.7% and change in symptom severity with an accuracy of 85.4%. It also predicts these outcomes with an accuracy of >80%, 11–15 weeks before the end of the semester, allowing ample time for pre-emptive interventions. Our work has significant implications for the detection of health outcomes using longitudinal behavioral data and limited ground truth. By detecting change and predicting symptoms several weeks before their onset, our work also has implications for preventing depression.

Past research regarding on-body interaction typically requires custom sensors, limiting their scalability and generalizability. We propose EarBuddy, a real-time system that leverages the microphone in commercial wireless earbuds to detect tapping and sliding gestures near the face and ears. We develop a design space to generate 27 valid gestures and conducted a user study (N=16) to select the eight gestures that were optimal for both human preference and microphone detectability. We collected a dataset on those eight gestures (N=20) and trained deep learning models for gesture detection and classification. Our optimized classifier achieved an accuracy of 95.3%. Finally, we conducted a user study (N=12) to evaluate EarBuddy's usability. Our results show that EarBuddy can facilitate novel interaction and that users feel very positively about the system. EarBuddy provides a new eyes-free, socially acceptable input method that is compatible with commercial wireless earbuds and has the potential for scalability and generalizability

College students every day decide and plan how to best spend their time to balance academic, physical, and social goals under uncertainty. This process is likely suboptimal where long-term life satisfaction and success is not guaranteed, and poor decision-making may lead to longer-term problems like depression. To support everyday planning, we introduce activity recommendation, a novel method that combines artificial intelligence, machine learning, and a psychology-informed approach to automatically generate activity-recommendations that optimize long-term life satisfaction. We tested our method with an existing dataset and derived activity recommendations for depressed and non-depressed students. We evaluated the recommendations through interviews with college students who rated the suggestions positively. Our model can be optimized for different goals and domains and is easy to interpret. Our results demonstrate the feasibility of our approach and lay the groundwork towards implementing a live system.

The integration of the fifth generation (5G) networks and artificial intelligence (AI) benefits to create a more holistic and better connected ecosystem for industries. User profiling has become an important issue for industries to improve company profit. In the 5G era, smartphone applications have become an indispensable part in our everyday lives. Users determine what apps to install based on their personal needs, interests, and tastes, which is likely shaped by their genders-the behavioral, cultural, or psychological traits typically associated with their sex. It is possible to profile users' gender based simply on a single snapshot of apps installed on their smartphones. With this inference based on easy to access data, we can make smartphone systems more user-friendly, and provide better personalized products and services. In this article, we explore such possibilities through an empirical study on a large-scale dataset of installed app lists from 15 000 Android users. More specifically, we investigate the following research questions: 1) What differences between females and males can be explored from installed app lists? 2) Can user gender be reliably inferred from a snapshot of apps installed? Which snapshot feature(s) are the most predictive? What is the best combination of features for building the gender prediction model? 3) What are the limitations of a gender prediction model based solely on a snapshot of apps installed on a smartphone? We find significant gender differences in app type, function, and icon design. We then extract the corresponding features from a snapshot of apps installed to infer the gender of each user. We assess the gender predictive ability of individual features and combinations of different features. We achieve an accuracy of 76.62% and area under the curve of 84.23% with the best set of features, outperforming the existing work by around 5% and 10%, respectively. Finally, we perform an error analysis on misclassified users and discussed the implications and limitations of this article.

The ubiquitous use of social media enables researchers to obtain self-recorded longitudinal data of individuals in real-time. Because this data can be collected in an inexpensive and unobtrusive way at scale, social media has been adopted as a “passive sensor” to study human behavior. However, such research is impacted by the lack of homogeneity in the use of social media, and the engineering challenges in obtaining such data. This paper proposes a statistical framework to leverage the potential of social media in sensing studies of human behavior, while navigating the challenges associated with its sparsity. Our framework is situated in a large-scale in-situ study concerning the passive assessment of psychological constructs of 757 information workers wherein of four sensing streams was deployed - bluetooth beacons, wearable, smartphone, and social media. Our framework includes principled feature transformation and machine learning models that predict latent social media features from the other passive sensors. We demonstrate the efficacy of this imputation framework via a high correlation of 0.78 between actual and imputed social media features. With the imputed features we test and validate predictions on psychological constructs like personality traits and affect. We find that adding the social media data streams, in their imputed form, improves the prediction of these measures. We discuss how our framework can be valuable in multimodal sensing studies that aim to gather comprehensive signals about an individual's state or situation.

People eat every day and biting is one of the most fundamental and natural actions that they perform on a daily basis. Existing work has explored tooth click location and jaw movement as input techniques, however clenching has the potential to add control to this input channel. We propose clench interaction that leverages clenching as an actively controlled physiological signal that can facilitate interactions. We conducted a user study to investigate users' ability to control their clench force. We found that users can easily discriminate three force levels, and that they can quickly confirm actions by unclenching (quick release). We developed a design space for clench interaction based on the results and investigated the usability of the clench interface. Participants preferred the clench over baselines and indicated a willingness to use clench-based interactions. This novel technique can provide an additional input method in cases where users' eyes or hands are busy, augment immersive experiences such as virtual/augmented reality, and assist individuals with disabilities.

Improving mobile keyboard typing speed increases in value as more tasks move to a mobile setting. Autocorrect reduces the time it takes to manually fix typing errors, which results in typing speed increase. However, recent user studies uncovered an unexplored side-effect: participants' aversion to typing errors despite autocorrect. We present a computational model of typing on keyboards with autocorrect, which enables precise study of expert typists' aversion to typing errors on such keyboards. Unlike empirical typing studies that last days, our model evaluates this phenomenon for any autocorrect accuracy in seconds. We show that typists' aversion to typing errors imposes a limit on upper bound typing speeds, even for highly accurate autocorrect. Our findings motivate future keyboard designs that reduce typists' aversion to typing errors to increase typing speeds.

Parkinson's disease (PD) is the second most common neurodegenerative disorder, impacting an estimated seven to ten million people worldwide. Measuring the symptoms and progress of the disease, and medication effectiveness is currently performed using subjective measures and visual estimation. We developed and evaluated a mobile application, STOP for tracking hand's motor symptoms, and a medication journal for recording medication intake. We followed 13 PD patients from two countries for a 1-month long real-world deployment. We found that PD patients are willing to use digital tools, such as STOP, to track their medication intake and symptoms, and are also willing to share such data with their caregivers and medical personnel to improve their own care.

A deep understanding of how discrimination impacts psychological health and well-being of students could allow us to better protect individuals at risk and support those who encounter discrimination. While the link between discrimination and diminished psychological and physical well-being is well established, existing research largely focuses on chronic discrimination and long-term outcomes. A better understanding of the short-term behavioral correlates of discrimination events could help us to concretely quantify such experiences, which in turn could support policy and intervention design. In this paper we specifically examine, for the first time, what behaviors change and in what ways in relation to discrimination. We use actively-reported and passively-measured markers of health and well-being in a sample of 209 first-year college students over the course of two academic quarters. We examine changes in indicators of psychological state in relation to reports of unfair treatment in terms of five categories of behaviors: physical activity, phone usage, social interaction, mobility, and sleep. We find that students who encounter unfair treatment become more physically active, interact more with their phone in the morning, make more calls in the evening, and spend more time in bed on the day of the event. Some of these patterns continue the next day. Our results further our understanding of the impact of discrimination and can inform intervention work.

Parkinson's disease (PD) is the second most common neurodegenerative disorder, impacting an estimated seven to ten million people worldwide. It is commonly accepted that improving medication adherence alleviates symptoms and maintains motor capabilities. Not following the medication regimen (e.g., skipping or over-medicating) may worsen side-effects, which mislead clinicians and patients. We developed and evaluated a mobile application, STOP, for screening the PD symptoms and medication intake. It contains a game for tracking the PD symptoms, and a medication journal for recording medical intake and adherence. We conducted a 1-month long real-world deployment with 13 PD patients from two countries. We found that the application medication adherence tracking provides non-bias information, and users are receptive to share such data with their care and medical personnel.

A large plethora of models to predict human mobility exists in the literature. The problem of how to select the most appropriate model to solve a specific mobility prediction task has however received only little attention. Yet, a wrong model choice may lead to severe performance losses. In this paper, we address the model selection problem in human mobility prediction and make the following contributions. We present SELECTOR, a generic framework to explore human mobility data and compute both population models and individual models to predict human mobility. The former are models that are adapted to the characteristics of an entire population of users and can be used to overcome the cold-start problem. The latter are prediction models optimized for individual users. We present and analyze the results obtainable using SELECTOR on the Nokia data set, which is one of the largest and richest, publicly available data sets of human mobility data. We show that for many users, generic population models can be used in place of individual models with negligible performance losses. Yet for about 25 percent of the users, individual models perform at least three percentage points better than population models. Thereby, we show that the use of phone context data does not lead to significantly better performance of human mobility predictors with respect to the case in which only temporal and spatial features are used. We further observe that the population models we derive are robust against the demographics of the users and that building different population models for different periods of the day leads to performance improvements. We make SELECTOR publicly available to allow other researchers and practitioners to explore further mobility data sets and to embed the code base of SELECTOR in their applications.

Accurate real-time depth camera-based tracking of limbs, fingers and faces would be of great use to the field of Sign Language Recognition (SLR). While aspects of depth-based tracking have been applied to SLR, technological limitations have previously forced trade-offs between the resolution necessary to track finger positions and the field of view necessary to track the signer's body. Only recently, with improvements in cameras and computing power, have algorithms been developed which boast the capability of maintaining accurate finger tracking over an appropriately sized volume of space. In this paper, we employ the publicly available Sphere-Mesh [1] hand tracking algorithm to collect and recognize ASL handshapes. In doing so, we demonstrate recognition rates comparable to other state of the art handshape classifiers using simple naíve Bayesian classifiers that can run in real-time.

Text entry is an increasingly important activity for mobile device users. As a result, increasing text entry speed of expert typists is an important design goal for physical and soft keyboards. Mathematical models that predict text entry speed can help with keyboard design and optimization. Making typing errors when entering text is inevitable. However, current models do not consider how typists themselves reduce the risk of making typing errors (and lower error frequency) by typing more slowly. We demonstrate that users respond to costly typing errors by reducing their typing speed to minimize typing errors. We present a model that estimates the effects of risk aversion to errors on typing speed. We estimate the magnitude of this speed change, and show that disregarding the adjustments to typing speed that expert typists use to reduce typing errors leads to overly optimistic estimates of maximum errorless expert typing speeds.

An ability to detect behaviors that negatively impact people's wellbeing and show people how they can correct those behaviors could enable technology that improves people's lives. Existing supervised machine learning approaches to detect and generate such behaviors require lengthy and expensive data labeling by domain experts. In this work, we focus on the domain of routine behaviors, where we model routines as a series of frequent actions that people perform in specific situations. We present an approach that bypasses labeling each behavior instance that a person exhibits. Instead, we weakly label instances using people's demonstrated routine. We classify and generate new instances based on the probability that they belong to the routine model. We illustrate our approach on an example system that helps drivers become aware of and understand their aggressive driving behaviors. Our work enables technology that can trigger interventions and help people reflect on their behaviors when those behaviors are likely to negatively impact them.

The recent profusion of sensors has given consumers and researchers the ability to collect significant amounts of data. However, understanding sensor data can be a challenge, because it is voluminous, multi-sourced, and unintelligible. Nonetheless, intelligent systems, such as activity recognition, require pattern analysis of sensor data streams to produce compelling results; machine learning (ML) applications enable this type of analysis. However, the number of ML experts able to proficiently classify sensor data is limited, and there remains a lack of interactive, usable tools to help intermediate users perform this type of analysis. To learn which features these tools must support, we conducted interviews with intermediate users of ML and conducted two probe-based studies with a prototype ML and visual analytics system, Gimlets. Our system implements ML applications for sensor-based time-series data as a novel domain-specific prototype that integrates interactive visual analytic features into the ML pipeline. We identify future directions for usable ML systems based on sensor data that will enable intermediate users to build systems that have been prohibitively difficult.

Though tactile maps have been shown to be useful tools for visually impaired individuals, their availability has been limited by manufacturing and design costs. In this paper, we present a system that uses 3D printing to (1) make tactile maps more affordable to produce, (2) allow visually impaired individuals to independently design and customize maps, and (3) provide interactivity using widely available mobile devices. Our system consists of three parts: a web interface, a modeling algorithm, and an interactive touchscreen application. Our web interface, hosted at www.tactilemaps.net, allows visually impaired individuals to create maps of any location on the globe while specifying (1) what features to map, (2) how the features should be represented by textures, and (3) where to place markers and labels. Our modeling algorithm accommodates user specifications to create map models with (1) multiple layers of continuously varying textures and (2) markers of various geometric shapes or braille characters. Our interactive application uses a novel approach to 3D printing tactile maps using conductive filament to provide touchscreen overlays that allow users to dynamically interact with the maps on a wide range of mobile devices. This paper details the implementation of our system. We also present findings from a user study validating the usability of our mapping interface and the utility of the maps produced. Finally, we discuss the limitations of our current implementation and the plans we have to improve our system based on feedback from our user study and additional interviews.

Human routines are blueprints of behavior, which allow people to accomplish purposeful repetitive tasks at many levels, ranging from the structure of their day to how they drive through an intersection. People express their routines through actions that they perform in the particular situations that triggered those actions. An ability to model routines and understand the situations in which they are likely to occur could allow technology to help people improve their bad habits, inexpert behavior, and other suboptimal routines. However, existing routine models do not capture the causal relationships between situations and actions that describe routines. Our main contribution is the insight that byproducts of an existing activity prediction algorithm can be used to model those causal relationships in routines. We apply this algorithm on two example datasets, and show that the modeled routines are meaningful-that they are predictive of people's actions and that the modeled causal relationships provide insights about the routines that match findings from previous research. Our approach offers a generalizable solution to model and reason about routines.

The ability to quickly interact with any nearby appliance from a mobile device would allow people to perform a wide range of one-time tasks (e.g., printing a document in an unfamiliar office location). However, users currently lack this capability, and must instead manually configure their devices for each appliance they want to use. To address this problem, we created Snap-To-It, a system that allows users to opportunistically interact with any appliance simply by taking a picture of it. Snap-To-It shares the image of the appliance a user wants to interact with over a local area network. Appliances then analyze this image (along with the user's location and device orientation) to see if they are being "selected," and deliver the corresponding control interface to the user's mobile device. Snap-To-It's design was informed by two technology probes that explored how users would like to select and interact with appliances using their mobile phone. These studies highlighted the need to be able to select hardware and software via a camera, and identified several novel use cases not supported by existing systems (e.g., interacting with disconnected objects, transferring settings between appliances). In this paper, we show how Snap-To-It's design is informed by our probes and how developers can utilize our system. We then show that Snap-To-It can identify appliances with over 95.3% accuracy, and demonstrate through a two-month deployment that our approach is robust to gradual changes to the environment.

Timebanking is a growing type of peer-to-peer service exchange, but is hampered by the effort of finding good transaction partners. We seek to reduce this effort by using a Matching Algorithm for Service Transactions (MAST). MAST matches transaction partners in terms of similarity of interests and complementarity of abilities and needs. We present an experiment involving data and participants from a real timebanking network, that evaluates the acceptability of MAST, and shows that such an algorithm can retrieve matches that are subjectively better than matches based on matching the category of people's historical offers or requests to the category of a current transaction request.

There is a significant gap in the body of research on cross-device interfaces. Research has largely focused on enabling them technically, but when and how users want to use cross-device interfaces is not well understood. This paper presents an exploratory user study with XDBrowser, a cross-device web browser we are developing to enable non-technical users to adapt existing single-device web interfaces for cross-device use while viewing them in the browser. We demonstrate that an end-user customization tool like XDBrowser is a powerful means to conduct user-driven elicitation studies useful for understanding user preferences and design requirements for cross-device interfaces. Our study with 15 participants elicited 144 desirable multi-device designs for five popular web interfaces when using two mobile devices in parallel. We describe the design space in this context, the usage scenarios targeted by users, the strategies used for designing cross-device interfaces, and seven concrete mobile multi-device design patterns that emerged. We discuss the method, compare the cross-device interfaces from our users and those defined by developers in prior work, and establish new requirements from observed user behavior. In particular, we identify the need to easily switch between different interface distributions depending on the task and to have more fine-grained control over synchronization.

It is well established that delays in system response time negatively impact productivity, error rates and user satisfaction. What is less clear is the degree to which these effects deter users from engaging with a system. Usability guidelines provide rough response time targets for minimizing these effects across various types of interactions. However, developers faced with technical limitations or cost constraints that prevent them from meeting such targets are given no data with which to estimate the impact that system response delays will have on user engagement. In this work, we demonstrate a methodology for using crowd sourcing platforms to examine (1) the relative impacts of different delay types and (2) the effects of marginal changes in system response times. We compare two common network delay types, those caused by limited bandwidth (increased download times) and those caused by network latency (lag in responsiveness), and present how these delays reduce engagement in the context of a crowd sourced image classification task. Furthermore, we model how financial incentives interact with system response delays to impact user engagement. Finally, we show how such models can be used to optimize the cost of system design choices.

Previous work on muscle activity sensing has leveraged specialized sensors such as electromyography and force sensitive resistors. While these sensors show great potential for detecting finger/hand gestures, they require additional hardware that adds to the cost and user discomfort. Past research has utilized sensors on commercial devices, focusing on recognizing gross hand gestures. In this work we present Serendipity, a new technique for recognizing unremarkable and fine-motor finger gestures using integrated motion sensors (accelerometer and gyroscope) in off-the-shelf smartwatches. Our system demonstrates the potential to distinguish 5 fine-motor gestures like pinching, tapping and rubbing fingers with an average f1-score of 87%. Our work is the first to explore the feasibility of using solely motion sensors on everyday wearable devices to detect fine-grained gestures. This promising technology can be deployed today on current smartwatches and has the potential to be applied to cross-device interactions, or as a tool for research in fields involving finger and hand motion.

When health services involve long-term treatment over months or years, providers have the ability, not present in acute emergency care, to collaboratively reflect on clients' changing health data and adjust interventions. In this paper, we discuss temporality as a factor in the design of health information technology. We define a temporal spectrum ranging from time-critical services that benefit from standardization to long-term services that require more flexibility. We provide empirical evidence from fieldwork that we performed in organizations providing long-term behavioral and mental health services for children. Our fieldwork in this context complements and provides contrasts to previous CSCW studies performed in time-critical hospital settings. Current literature shows a bias toward standardized records and routines in the implementation of health information technology, a policy that may not be appropriate for long-term health services. We discuss how the design of information systems should vary based on temporal factors.

Hospital readmissions are a major problem facing health care systems today, costing Medicare alone US$26 billion each year. Being readmitted is associated with significantly shorter survival, and is often preventable. Predictors of readmission are still not well understood, particularly those under the patient's control: behavioral risk factors. Our work evaluates the ability of behavioral risk factors, specifically Fitbit-assessed behavior, to predict readmission for 25 postsurgical cancer inpatients. Our results show that sum of steps, maximum sedentary bouts, frequency, and low breaks in sedentary times during waking hours are strong predictors of readmission. We built two models for predicting readmissions: Steps-only and Behavioral model that adds information about sedentary behaviors. The Behavioral model (88.3%) outperforms the Steps-only model (67.1%), illustrating the value of passively collected information about sedentary behaviors. Indeed, passive monitoring of behavior data, i.e., mobility, after major surgery creates an opportunity for early risk assessment and timely interventions.

Understanding smartphone users is fundamental for creating better smartphones, and improving the smartphone usage experience and generating generalizable and reproducible research. However, smartphone manufacturers and most of the mobile computing research community make a simplifying assumption that all smartphone users are similar or, at best, constitute a small number of user types, based on their behaviors. Manufacturers design phones for the broadest audience and hope they work for all users. Researchers mostly analyze data from smartphone-based user studies and report results without accounting for the many different groups of people that make up the user base of smartphones. In this work, we challenge these elementary characterizations of smartphone users and show evidence of the existence of a much more diverse set of users. We analyzed one month of application usage from 106,762 Android users and discovered 382 distinct types of users based on their application usage behaviors, using our own two-step clustering and feature ranking selection approach. Our results have profound implications on the reproducibility and reliability of mobile computing studies, design and development of applications, determination of which apps should be pre-installed on a smartphone and, in general, on the smartphone usage experience for different types of users.

Activity recognition is a key component of context-aware computing to support people's physical activity, but conventional approaches often lack in their generalizability and scalability due to problems of diversity in how individuals perform activities, overfitting when building activity models, and collection of a large amount of labeled data from end users. To address these limitations, we propose a semipopulation-based approach that exploits activity models trained from other users; therefore, a new user does not need to provide a large volume of labeled activity data. Instead of relying on any additional information from users like their weight or height, our approach directly measures the fitness of others' models on a small amount of labeled data collected from the new user. With these shared activity models among users, we compose a hybrid model of Bayesian networks and support vector machines to accurately recognize the activity of the new user. On activity data collected from 28 people with a diversity in gender, age, weight, and height, our approach produced an average accuracy of 83.4% (kappa: 0.852), compared with individual and (standard) population models that had accuracies of 77.3% (kappa: 0.79) and 77.7% (kappa: 0.743), respectively. Through an analysis on the performance of our approach and users' demographic information, our approach outperforms others that rely on users' demographic information for recognizing their activities, which may contradict the commonly held belief that physically similar people would have similar activity patterns.

Due to the cut throat competition among organizations, business process improvement is now an everyday activity. A relentless activity that makes business processes more complex than ever. As they get more complex, the improvement rounds become time-consuming, costly and the quality of each outcome is put into jeopardy, which is somehow paradoxical with the concept of improvement. In this paper, we propose a business process improvement technique based on ubiquitous computing. First, we couple business processes with ubiquitous computing and define a ubiquitous business process. Then, we explain how ubiquitous computing positively impacts the performance metrics of business processes. Afterwards, we set a specification for designing ubiquitous business processes by extending BPMN. Finally, we propose a concrete case study about time-banking to corroborate our theory. A comparative study of the same process, in ubiquitous and non-ubiquitous versions, is established. The results clearly illustrate that ubiquitous computing impacts positively the business process performance metrics. Still, the case study corroborates that ubiquitous computing not only improves a business process but also enables it to get improved with the least of human interventions.

Tactile maps are useful, but not commonly available, tools for providing visually impaired individuals with knowledge about their environment. We have developed a web tool to allow visually impaired users to specify locations and customize 3D map models for production with 3D printers. Our tool uses available online map data and encodes features such as roads and waterways into 3D-printable tactile features. We present a preliminary overview of our web interface and tactile map-generating software with a focus on the design choices that were informed by our pilot studies. We will also discuss additional findings from our focus group interviews and future plans for this work.

We study eye gaze movement behavior during paper reading and generate a series of annotations from a user's reading features: gray shading to indicate reading speed, borders to indicate frequency of re-reading, and lines to indicate transitions between sections of a document. Through a user study, we validate that our SocialReading system that shares teachers' gaze data for an academic paper can improve students' reading comprehension of that paper.

Interruptions while driving can be quite dangerous, whether these are self-interruptions or external interruptions. They increase driver workload and reduce performance on the primary driving task. Being able to identify when a driver is interruptible is critical for building systems that can mediate these interruptions. In this paper, we collect sensor and human-annotated data from 15 drivers, including vehicle motion, traffic states, physiological responses and driver motion. We demonstrate that this data can be used to build a machine learning classifier that can determine interruptibility every second with a 94% accuracy. We present both population and individual models and discuss the features that contribute to the high performance of this system. Such a classifier can be used to build systems that mediate when drivers use technology to self-interrupt and when drivers are interrupted by technology.

Eye tracking is a compelling tool for revealing people's spatial-temporal distribution of visual attention. But quality eye tracking hardware is expensive and can only be used with one person at a time. Further, webcam eye tracking systems have significant limitations on head movement and lighting conditions that result in significant data loss and inaccuracies. To address these drawbacks, we introduce a new approach that harnesses the crowd to understand allocation of visual attention. In our approach, crowdsourcing participants use mouse clicks to self-report the positions and trajectory for the following valuable eye tracking measures: first gaze, last gaze and all gazes. We validate our crowdsourcing approach with a user study, which demonstrated good accuracy when compared to a real eye tracker. We then deployed our prototype, GazeCrowd, in a crowdsourcing setting, and showed that it accurately generated gaze heatmaps and trajectory maps. Such an approach will allow designers to evaluate and refine their visual design without requiring the use of limited/expensive eye trackers.

In this paper, we present the Group Context Framework (GCF), a general-purpose toolkit that allows mobile devices to opportunistically share contextual information. GCF provides a standardized way for developers to request contextual data for their applications. The framework then intelligently groups with other devices to satisfy these requirements. Through two prototypes, we demonstrate how GCF can be used to support a broad range of collaborative and cooperative tasks. We then show how our framework's architecture allows devices to opportunistically detect and collaborate with one another, even when running different applications. Finally, we present two real-world domains that show how GCF's ability to form groups increases users' access to relevant and timely information, and discuss possible incentives and safeguards to context sharing from a user standpoint.

We present a new technique that allows mobile devices to opportunistically group with one another, thus improving their ability to facilitate one-time or spontaneous exchanges of information. In our approach, devices share context with each other, and form groups when these readings are found to be similar to one another. Through a formative study, we examine the limitations of using a single type of context to form groups, and show how leveraging multiple contexts improves our ability to detect and form relevant groupings. We then present DIDJA, a robust software toolkit that automatically collects and analyzes contextual information in order to find and form groups. Through two prototypes, we demonstrate how DIDJA enhances existing user experiences, and show how developers can use our toolkit to easily facilitate frictionless collaborations between users and their environment. We then perform an extended experiment and show how DIDJA is able to accurately form groups under realistic conditions.

Occupancy count in rooms is valuable for applications such as room utilization, opportunistic meeting support, and efficient heating-cooling operations. Few buildings, however, have the means of knowing occupancy beyond simple binary presence-absence. In this paper we present the PerCCS algorithm that explores the possibility of estimating person count from CO2 sensors already integrated in everyday room air-conditioning infrastructure. PerCSS uses task-driven Sparse Non-negative Matrix Factorization (SNMF) to learn a nonnegative low-dimensional representation of the CO2 data in the preprocessing stage. This denoised CO2 acts as the predictor variable for estimating occupancy count using Ensemble Least Square Regression. We tested the algorithm to estimate 15 minutes average occupancy count from a classroom of capacity 42 and compared its performance against existing methods from the literature. PerCSS estimates occupancy with a normalized mean squared error (NMSE) of 0.075 and outperformed our comparative methods in predicting occupancy count with 91 % and 15 % for exact occupancy estimation, when the room was unoccupied and occupied respectively, whereas the competing methods failed mostly.

In today's ubiquitous computing environment where users carry, manipulate, and interact with an increasing number of networked devices, applications and web services, human attention is the new bottleneck in computing. It is therefore important to minimize a user's mental effort due to notifications, especially in situations where users are mobile and using multiple wearable and mobile devices. To this end, we propose Attelia II, a novel middleware that identifies breakpoints in users' lives while using those devices, and delivers notifications at these moments. Attelia II works in real-time and uses only the mobile and wearable devices that users naturally use and wear, without any modifications to applications, and without any dedicated psycho-physiological sensors. Our in-the-wild evaluation in users' multi-device environment (smart phones and smart watches) with 41 participants for 1 month validated the effectiveness of Attelia. Our new physical activity-based breakpoint detection, in addition to the UI Event-based breakpoint detection, resulted in a 71.8% greater reduction of users' perception of workload, compared with our previous system that used UI events only. Adding this functionality to a smart watch reduced workload perception by 19.4% compared to random timing of notification deliveries. Our multi-device breakpoint detection across smart phones and watches resulted in about 3 times greater reduction in workload perception than our previous system.

In mobile computing, varying access to resources makes it difficult for developers to ensure that satisfactory system response times will be maintained at all times. Wearable physiological sensors offer a way to dynamically detect user frustration in response to increased system delays. However, most prior efforts have focused on binary classifiers designed to detect the presence or absence of a task-specific stimulus. In this paper, we make two contributions. Our first contribution is in identifying the use of variable length system response delays, a universal and task-independent feature of computing, as a stimulus for driving different levels of frustration. By doing so, we are able to make our second and primary contribution, which is the development of models that predict multiple levels of user frustration from psycho-physiological responses caused by system response delays. We investigate how incorporating different sensor features, application settings, and timing constraints impact the performance of our models. We demonstrate that our models of physiological responses can be used to classify five levels of frustration in near real-time with over 80% accuracy, which is comparable to the accuracy of binary classifiers.

In today's ubiquitous computing environment where the number of devices, applications and web services are ever increasing, human attention is the new bottleneck in computing. To minimize user cognitive load, we propose Attelia, a novel middleware that identifies breakpoints in user interaction and delivers notifications at these moments. Attelia works in realtime and uses only the mobile devices that users naturally use and wear, without any modifications to applications, and without any dedicated psycho-physiological sensors. Our evaluation proved the effectiveness of Attelia. A controlled user study showed that notifications at detected breakpoint timing resulted in 46% lower cognitive load compared to randomly-timed notifications. Furthermore, our “in-the-wild” user study with 30 participants for 16 days further validated Attelia's value, with a 33% decrease in cognitive load compared to randomly-timed notifications.

The widespread presence of motion sensors on users' personal mobile devices has spawned a growing research interest in human activity recognition (HAR). However, when deployed at a large-scale, e.g., on multiple devices, the performance of a HAR system is often significantly lower than in reported research results. This is due to variations in training and test device hardware and their operating system characteristics among others. In this paper, we systematically investigate sensor-, device- and workload-specific heterogeneities using 36 smartphones and smartwatches, consisting of 13 different device models from four manufacturers. Furthermore, we conduct experiments with nine users and investigate popular feature representation and classification techniques in HAR research. Our results indicate that on-device sensor and sensor handling heterogeneities impair HAR performances significantly. Moreover, the impairments vary significantly across devices and depends on the type of recognition technique used. We systematically evaluate the effect of mobile sensing heterogeneities on HAR and propose a novel clustering-based mitigation technique suitable for large-scale deployment of HAR, where heterogeneity of devices and their usage scenarios are intrinsic.

Smartphone users do not fully know what their apps do. For example, an applications' network usage and underlying security configuration is invisible to users. In this paper we introduce Securacy, a mobile app that explores users' privacy and security concerns with Android apps. Securacy takes a reactive, personalized approach, highlighting app permission settings that the user has previously stated are concerning, and provides feedback on the use of secure and insecure network communication for each app. We began our design of Securacy by conducting a literature review and in-depth interviews with 30 participants to understand their concerns. We used this knowledge to build Securacy and evaluated its use by another set of 218 anonymous participants who installed the application from the Google Play store. Our results show that access to address book information is by far the biggest privacy concern. Over half (56.4%) of the connections made by apps are insecure, and the destination of the majority of network traffic is North America, regardless of the location of the user. Our app provides unprecedented insight into Android applications' communications behavior globally, indicating that the majority of apps currently use insecure network connections.

Using the physiological system to perform affect modeling has great potential but also introduces many challenging issues in pervasive and interactive computing. With the advances in low-power mobile sensors, it is now possible to create a good quality of affect models based on physiological responses, which are useful in understanding how people express affect in real-world environments. In this paper, we have investigated an affect modeling technique that analyzes physiological changes and models user affect with data gathered in the field. In particular, we have identified a number of sensor channels and features that are discriminable in recognizing stress with Support Vector Machines. We have empirically investigated the value of creating an affect model by using a subset of informative features for an individual on physiological data collected in real-world environments (i.e. outside the lab), and we provide a discussion of the remaining challenging issues in performing field-based physiological analysis.

Driving aggressively increases the risk of accidents. Assessing a person's driving style is a useful way to guide aggressive drivers toward having safer driving behaviors. A number of studies have investigated driving style, but they often rely on the use of self-reports or simulators, which are not suitable for the real-time, continuous, automated assessment and feedback on the road. In order to understand and model aggressive driving style, we construct an in-vehicle sensing platform that uses a smartphone instead of using heavyweight, expensive systems. Utilizing additional cheap sensors, our sensing platform can collect useful information about vehicle movement, maneuvering and steering wheel movement. We use this data and apply machine learning to build a driver model that evaluates drivers' driving styles based on a number of driving-related features. From a naturalistic data collection from 22 drivers for 3 weeks, we analyzed the characteristics of drivers who have an aggressive driving style. Our model classified those drivers with an accuracy of 90.5% (violation-class) and 81% (questionnaire-class). We describe how, in future work, our model can be used to provide real-time feedback to drivers using only their current smartphone.

Medication taking is a self-regulatory process that requires individuals to self-monitor their medication taking behaviors, but this can be difficult because medication taking is such a mundane, unremarkable behavior. Ubiquitous sensing systems have the potential to sense everyday behaviors and provide the objective feedback necessary for self-regulation of medication taking. We describe an unobtrusive sensing system consisting of a sensor-augmented pillbox and an ambient display that provides near real-time visual feedback about how well medications are being taken. In contrast to other systems that focus on reminding before medication taking, our approach uses feedback after medication taking to allow the individual to develop their own routines through self-regulation. We evaluated this system in the homes of older adults in a 10-month deployment. Feedback helped improve the consistency of medication-taking behaviors as well as increased ratings of self-efficacy. However, the improved performance did not persist after the feedback display was removed, because individuals had integrated the feedback display into their routines to support their self-awareness, identify mistakes, guide the timing of medication taking, and provide a sense of security that they are taking their medications well. Finally, we reflect on design considerations for feedback systems to support the process of self-regulation of everyday behaviors.

When a user learns to use a new device, her understanding of it evolves. A progressive comparison of the evolving user models towards the device target model, for analysing learning, involves determining the behavioral proximity between them. To quantify the gap between a user model and a target model, we introduce an edit distance metric for measuring their behavioral proximity using a bisimulation-based equivalence relation. We define edit distance to be the minimum number of edges and states with incident edges required to be deleted from and/or added to a user model to make it bisimilar to the target model. We propose an algorithm to compute edit distance between two models and employ the heuristic procedure on experimental data for computing edit distance between target and user models. The data is organised into two experiments depending on the device the user interacted with: (a) a simple device resembling a vending machine and (b) a close to real-world vehicle transmission model. The results validate our proposed metric as edit distance converges with progressive user learning, increases for erroneous learning, and remains unchanged indicating no learning.

Location prediction enables us to use a person's mobility history to realize various applications such as efficient temperature control, opportunistic meeting support, and automated receptionists. Indoor location prediction is a challenging problem, particularly due to a high density of possible locations and short transition distances between these locations. In this paper we present Indoor-ALPS, an Adaptive Indoor Location Prediction System that uses temporal-spatial features to create individual daily models for the prediction of when a user will leave their current location (transition time) and the next location she will transition to. We tested Indoor-ALPS on the Augsburg Indoor Location Tracking Benchmark and compared our approach to the best performing temporal-spatial mobility prediction algorithm, Prediction by Partial Match (PPM). Our results show that Indoor-ALPS improves the temporal-spatial prediction accuracy over PPM for look-aheads up to 90 minutes by 6.2%, and for up to 30 minute look-aheads by 10.7%. These results demonstrate that Indoor-ALPS can be used to support a wide variety of indoor mobility prediction-based applications.

Mobile devices have become powerful ultra-portable personal computers supporting not only communication but also running a variety of complex, interactive applications. Because of the unique characteristics of mobile interaction, a better understanding of the time duration and context of mobile device uses could help to improve and streamline the user experience. In this paper, we first explore the anatomy of mobile device use and propose a classification of use based on duration and interaction type: glance, review, and engage. We then focus our investigation on short review interactions and identify opportunities for streamlining these mobile device uses through proactively suggesting short tasks to the user that go beyond simple application notifications. We evaluate the concept through a user evaluation of an interactive lock screen prototype, called ProactiveTasks. We use the findings from our study to create and explore the design space for proactively presenting tasks to the users. Our findings underline the need for a more nuanced set of interactions that support short mobile device uses, in particular review sessions.

Research suggests smartphone users face 'application overload', but literature lacks an in-depth investigation of how users manage their time on smartphones. In a 3-week study we collected smartphone application usage patterns from 21 participants to study how they manage their time interacting with the device. We identified events we term application micro-usage: brief bursts of interaction with applications. While this practice has been reported before, it has not been investigated in terms of the context in which it occurs (e.g., location, time, trigger and social context). In a 2-week follow-up study with 15 participants, we captured participants? context while micro-using, with a mobile experience sampling method (ESM) and weekly interviews. Our results show that about approximately 40% of application launches last less than 15 seconds and happen most frequently when the user is at home and alone. We further discuss the context, taxonomy and implications of application micro-usage in our field. We conclude with a brief reflection on the relevance of short-term interaction observations for other domains beyond mobile phones.

As mobile context-aware services gain mainstream popularity, there is increased interest in developing techniques that can detect anomalous activities for applications such as user authentication, adaptive assist technologies and remote elder-care monitoring. Existing approaches have limited applicability as they regularly poll power-hungry sensors (e.g., accelerometer, GPS) reducing the availability of devices to perform anomaly detection. This paper present SALCS (Soft Authentication with Low-Cost Signatures), an approach for anomaly detection on a user's routine comprised of a collection of anomaly detection techniques utilizing soft-sensor data (e.g., call-logs, messages) and radio channel information (e.g., GSM cell IDs), all of which are available as part of a phone's routine usage. Using these information sources we model aspects of a person's routine, such as movement, messaging and conversation patterns. We present extensive evaluations of the individual anomaly detection techniques, compare the collection SALCS to an existing power-hungry approach showing SALCS has a 7.6% higher detection rate and gives 5x better coverage throughout the day.

With ever more everyday objects becoming “smart” due to embedded processors and communication capabilities, the provisioning of intuitive user interfaces to control smart things is quickly gaining importance. We present a model-based interface description scheme that enables automatic, modality-independent user interface generation. User interface description languages based on our approach carry enough information to suggest intuitive interfaces while still being easily producible for developers. This is enabled by describing the atomic interactive components of a device and capturing the semantics of interactions with the device. We propose a taxonomy of abstract sensing and actuation primitives and present a smartphone application that can act as a ubiquitous device controller. An evaluation of the mobile application in a laboratory setup, home environments, and an educational setting as well as the results of a user study highlight the accessibility of the proposed scheme for application developers and its suitability for controlling smart devices.

Sighted individuals often develop significant knowledge about their environment through what they can visually observe. In contrast, individuals who are visually impaired mostly acquire such knowledge about their environment through information that is explicitly related to them. This paper examines the practices that visually impaired individuals use to learn about their environments and the associated challenges. In the first of our two studies, we uncover four types of information needed to master and navigate the environment. We detail how individuals' context impacts their ability to learn this information, and outline requirements for independent spatial learning. In a second study, we explore how individuals learn about places and activities in their environment. Our findings show that users not only learn information to satisfy their immediate needs, but also to enable future opportunities -- something existing technologies do not fully support. From these findings, we discuss future research and design opportunities to assist the visually impaired in independent spatial learning.

Autism education programs for children collect and use large amounts of behavioral data on each student. Staff use paper almost exclusively to collect these data, despite significant problems they face in tracking student data in situ, filling out data sheets and graphs on a daily basis, and using the sheets in collaborative decision making. We conducted fieldwork to understand data collection and use in the domain of autism education to explain why current technology had not met staff needs. We found that data needs are complex and unstandardized, immediate demands of the job interfere with staff ability to collect in situ data, and existing technology for data collection is inadequate. We also identified opportunities for technology to improve sharing and use of data. We found that data sheets are idiosyncratic and not useful without human mediation; improved communication with parents could benefit children's development; and staff are willing, and even eager, to incorporate technology. These factors explain the continued dependence on paper for data collection in this environment, and reveal opportunities for technology to support data collection and improve use of collected data.

Mobile phone user interfaces typically show an icon to indicate remaining battery, but not the amount of time the device can be used for, often forcing users to make faulty estimates and predictions about battery life. Here we report on two studies that capture users' experiences with a user-centered battery interface design. In Study 1, we analyze 12 participants' use of mobile phones, demonstrating that mobile phone users do not know how or what to do to extend their mobile's battery life. We further identify the information they rely on to assess battery life. In Study 2, we use this information to design, prototype and evaluate an interactive battery interface (IBI) with another 22 participants. Our findings describe how users perceive battery life and how we used their mental models of mobile phone batteries to create IBI. Lastly, we report on the users' experiences and IBI's effect on battery lifetime, showing gains of approximately 27% over the course of a day.

Reducing the large energy consumption of temperature regulation systems is a challenge for researchers and practitioners alike. In this paper, we explore and compare two common types of solutions: A manual systems that encourages reduced energy use, and an intelligent automatic control system. We deployed an eco-feedback system with the ability to remotely control one's thermostat to ten participants for three months. Participants appreciated the ability to remotely control the thermostat, and controlled their heating system with 78.8% accuracy, a 6.3% improvement over not having this system. However, despite having feedback and remote control, they still wasted a lot of energy heating when away from home for the day. Using data from our deployment, we developed TherML, an occupancy prediction algorithm that uses GPS data from a user's smartphone to automatically control the indoor temperature of a home with 92.1% accuracy. We compare TherML to other state-of-the-art techniques, and show that the higher accuracy of our approach optimizes both energy usage and user comfort. We end with recommendations for a mixed initiative system that leverages aspects of both the manual and automated approaches that can better match heating control to users' routines and preferences.

Smartphone adoption has increased significantly and, with the increase in smartphone capabilities, this means that users can access the Internet, communicate, and entertain themselves anywhere and anytime. However, there is growing evidence of problematic use of smartphones that impacts both social and heath aspects of users' lives. Currently, assessment of overuse or problematic use depends on one-time, self-reported behavioral information about phone use. Due to the known issues with self-reports in such types of assessments, we explore an automated, objective and repeatable approach for assessing problematic usage. We collect a wide range of phone usage data from smartphones, identify a number of usage features that are relevant to this assessment, and build detection models based on Adaboost with machine learning algorithms automatically detecting problematic use. We found that the number of apps used per day, the ratio of SMSs to calls, the number of event-initiated sessions, the number of apps used per event initiated session, and the length of non-event-initiated sessions are useful for detecting problematic usage. With these, a detection model can identify users with problematic usage with 89.6% accuracy (F-score of .707).

This poster presents the homeset algorithm, a lightweight approach to estimate occupancy schedules of private households. The algorithm relies on the mobile phones of households' occupants to collect Wi-Fi scans. The scans are then used to determine if occupants are at home or not. The algorithm operates in an autonomous fashion using only information available locally on the mobile phones. We validate our approach using a data set from the Nokia Lausanne Data Collection Campaign.

The principle of maximum entropy provides a powerful framework for estimating joint, conditional, and marginal probability distributions. However, there are many important distributions with elements of interaction and feedback where its applicability has not been established. This paper presents the principle of maximum causal entropy-an approach based on directed information theory for estimating an unknown process based on its interactions with a known process. We demonstrate the breadth of the approach using two applications: a predictive solution for inverse optimal control in decision processes and computing equilibrium strategies in sequential games.

Designing radically new technology systems that people will want to use is complex. Design teams must draw on knowledge related to people's current values and desires to envision a preferred yet plausible future. However, the introduction of new technology can shape people's values and practices, and what-we-know-now about them does not always translate to an effective guess of what the future could, or should, be. New products and systems typically exist outside of current understandings of technology and use paradigms; they often have few interaction and social conventions to guide the design process, making efforts to pursue them complex and risky. User Enactments (UEs) have been developed as a design approach that aids design teams in more successfully investigate radical alterations to technologies' roles, forms, and behaviors in uncharted design spaces. In this paper, we reflect on our repeated use of UE over the past five years to unpack lessons learned and further specify how and when to use it. We conclude with a reflection on how UE can function as a boundary object and implications for future work.

Barriers to physical activity prevent older adults from meeting recommended physical activity levels necessary for maintaining quality of life. As technology becomes more advanced, we have the opportunity and the responsibility to address concerns faced by the aging population. We seek opportunities for technology to empower older adults to overcome barriers on their own by interviewing and learning from older adults who have successfully overcome these barriers. In this paper, we present a set of needs that technology can address, and considerations for designing technology interventions that support physical activity by older adults.

In this note, we describe Spark, an informative art display that visualizes physical activity using abstract art. We present results from five deployments, lasting two to three weeks, that suggest that while graph visualizations are useful for information seeking, abstract visualizations are preferred for display purposes. Our results show that informative art is an appropriate way to visualize physical activity, and can be used in addition to graphs to increase enjoyment and engagement with physical activity displays.

Ubiquitous applications and smart environment technologies are complex to deploy, manage and use. Intelligibility, in ubiquitous computing applications, explains to users what a system did (outputs) and why it did it (inputs or contextual information). Making software more intelligible can reduce the complexity of a system for users. This paper presents our work on an intelligibility strategy for fuzzy logic systems, applied to a context-aware software organization and service provision (SOSP) middleware for smart environments. This fuzzy logic intelligibility strategy has been evaluated and tested with two groups of real users (technical and less technical users), and two versions of our prototype (with and without intelligibility).

With advances in physiological sensors, we are able to understand people's physiological status and recognize stress to provide beneficial services. Despite the great potential in physiological stress recognition, there are some critical issues that need to be addressed such as the sensitivity and variability of physiology to many factors other than stress (e.g., physical activity). To resolve these issues, in this paper, we focus on the understanding of physiological responses to both stressor and physical activity and perform stress recognition, particularly in situations having multiple stimuli: physical activity and stressors. We construct stress models that correspond to individual situations, and we validate our stress modeling in the presence of physical activity. Analysis of our experiments provides an understanding on how physiological responses change with different stressors and how physical activity confounds stress recognition with physiological responses. In both objective and subjective settings, the accuracy of stress recognition drops by more than 14% when physical activity is performed. However, by modularizing stress models with respect to physical activity, we can recognize stress with accuracies of 82% (objective stress) and 87% (subjective stress), achieving more than a 5-10% improvement from approaches that do not take physical activity into account.

Smart environments are improving their performance and services by increasingly using ubiquitous sensing and complex inference mechanisms. However, this comes at a cost of reduced intelligibility, user trust and control. The Intelligibility Toolkit was developed to support the automatic generation and provision of explanations to help users understand context-aware inference. We have extended the toolkit to generate explanations for a wider range of inference models and to provide two styles of explanations --- rule traces and weights of evidence. We describe explanations generated from several inference models for a smart home dataset for activity recognition. This demonstrates the versatility of using the Intelligibility Toolkit to retain explanatory capabilities across different inference models.

Recorded images of children's activities can be useful to caregivers and clinicians who need behavioral evidence to support children with autism. However, image capture systems for autism are typically complex and provide only a top-down, outsider's view. In this work, we assessed the use of cameras worn by children to record the context of their activities and interactions from their perspective. We used a technology probe to explore how this simple, parent-driven system could be designed for families to adopt in their homes. We present the results of a five-week field study with five families. The system helped parents to (1) see the world from their child's eyes, (2) increase their understanding of their child's needs when their child is uncommunicative, and (3) help them encourage their child's social engagement. We discuss how these systems can be designed and used to their full potential.

It is becoming harder to find an app on one's smart phone due to the increasing number of apps available and installed on smart phones today. We collect sensory data including app use from smart phones, to perform a comprehensive analysis of the context related to mobile app use, and build prediction models that calculate the probability of an app in the current context. Based on these models, we developed a dynamic home screen application that presents icons for the most probable apps on the main screen of the phone and highlights the most probable one. Our models outperformed other strategies, and, in particular, improved prediction accuracy by 8% over Most Frequently Used from 79.8% to 87.8% (for 9 candidate apps). Also, we found that the dynamic home screen improved accessibility to apps on the phone, compared to the conventional static home screen in terms of accuracy, required touch input and app selection time.

Numerous interaction techniques have been developed that make "virtual" pointing at targets in graphical user interfaces easier than analogous physical pointing tasks by invoking target-based interface modifications. These pointing facilitation techniques crucially depend on methods for estimating the relevance of potential targets. Unfortunately, many of the simple methods employed to date are inaccurate in common settings with many selectable targets in close proximity. In this paper, we bring recent advances in statistical machine learning to bear on this underlying target relevance estimation problem. By framing past target-driven pointing trajectories as approximate solutions to well-studied control problems, we learn the probabilistic dynamics of pointing trajectories that enable more accurate predictions of intended targets.

The combination of sensors, perception algorithms, and mobile computing enables situationally aware systems that provide proactive assistance. This paper outlines the basic components of a virtual coach with illustrations in five applications ranging from reminders to advice to opportunities for personal reflection.

There are many physical activity awareness systems available in today's market. These systems show physical activity information (e.g., step counts, energy expenditure, heart rate) which is sufficient for many self-knowledge needs, but information about the factors that affect physical activity may be needed for deeper self-reflection and increased self-knowledge. We explored the use of contextual information, such as events, places, and people, to support reflection on the factors that affect physical activity. We present three findings from our studies. First, users make associations between physical activity and contextual information that help them become aware of factors that affect their physical activity. Second, reflecting on physical activity and context can increase people's awareness of opportunities for physical activity. Lastly, automated tracking of physical activity and contextual information benefits long-term reflection, but may have detrimental effects on immediate awareness.

When mobile robots perform tasks in environments with humans, it seems appropriate for the robots to rely on such humans for help instead of dedicated human oracles or supervisors. However, these humans are not always available nor always accurate. In this work, we consider human help to a robot as concretely providing observations about the robot's state to reduce state uncertainty as it executes its policy autonomously. We model the probability of receiving an observation from a human in terms of their availability and accuracy by introducing Human Observation Providers POMDPs (HOP-POMDPs). We contribute an algorithm to learn human availability and accuracy online while the robot is executing its current task policy. We demonstrate that our algorithmis effective in approximating the true availability and accuracy of humans without depending on oracles to learn, thus increasing the tractability of deploying a robot that can occasionally ask for help.

Part of being a parent is taking responsibility for arranging and supplying transportation of children between various events. Dual-income parents frequently develop routines to help manage transportation with a minimal amount of attention. On days when families deviate from their routines, effective logistics can often depend on knowledge of the routine location, availability and intentions of other family members. Since most families rarely document their routine activities, making that needed information unavailable, coordination breakdowns are much more likely to occur. To address this problem we demonstrate the feasibility of learning family routines using mobile phone GPS. We describe how we (1) detect pick-ups and drop-offs; (2) predict which parent will perform a future pick-up or drop-off; and (3) infer if a child will be left at an activity. We discuss how these routine models give digital calendars, reminder and location systems new capabilities to help prevent breakdowns, and improve family life.

The elder population is rising worldwide; in the US, no longer being able to drive is a significant marker of loss of independence. One of the approaches to helping elders drive more safely is to investigate the use of automotive user interface technology, and specifically, to explore the instrument panel (IP) display design to help attract and manage attention and make information easier to interpret.

Older adults often struggle with maintaining self-aware of their ability to carry out everyday activities important for independence. Unobtrusive sensors embedded in the home can monitor how older adults interact with objects around the home and can provide objective accounts of behaviors to support self-awareness. In this paper, we describe the design and four month deployment of a prototype sensing system that tracks medication taking and phone use in the homes of two older adults. We describe two case studies on 1) how they engaged with the data by looking for and explaining their own anomalous behaviors and 2) how they used the sensor data to reflect on their actions and their own self-awareness of their abilities to remain independent. Finally, we propose recommendations for the design of home sensing systems that support awareness of functional abilities for older adults using reflection.

Much research in ubiquitous computing assumes that a user's phone will be always on and at-hand, for collecting user context and for communicating with a user. Previous work with the previous generation of mobile phones has shown that such an assumption is false. Here, we investigate whether this assumption about users' proximity to their mobile phones holds for a new generation of mobile phones, smart phones. We conduct a data collection field study of 28 smart phone owners over a period of 4 weeks. We show that in fact this assumption is still false, with the within arm's reach proximity being true close to 50% of the time, similar to the earlier work. However, we also show that smart phone proximity within the same room (arm+room) as the user is true almost 90% of the time. We discuss the reasons for these phone proximities and the implications of this on the development of mobile phone applications, particularly those that collect user and environmental context, and delivering notification to users. We also show that we can accurately predict the proximity at the arm level and arm+room level with 75 and 83% accuracy, respectively, with features simple to collect and model on a mobile phone. Further we show that for several individuals who are almost always within the arm+room level, we can predict this level with over 90% accuracy.

We live in a world where many kinds of data about us can be collected and more will be collected as Ubicomp technologies mature. People reflect on this data using different tools for personal informatics. However, current tools do not have sufficient understanding of users' self-reflection needs to appropriately leverage Ubicomp technologies. To design tools that effectively assist self-reflection, we need to comprehensively understand what kinds of questions people have about their data, why they ask these questions, how they answer them with current tools, and what kinds of problems they encounter. To explore this, we conducted interviews with people who use various kinds of tools for personal informatics. We found six kinds of questions that people asked about their data. We also found that certain kinds of questions are more important at certain times, which we call phases. We identified two phases of reflection: Discovery and Maintenance. We discuss the kinds of questions and the phases in detail and identify features that should be supported in personal informatics tools for which Ubicomp technologies can play an important role.

Context-aware applications use sensing and inference to attempt to determine users' contexts, and take appropriate action. However, they are prone to uncertainty, and this may compromise the trust users have in them. Providing intelligibility has been proposed to help explain to users how context-aware applications work in order to improve user impressions of them. However, we hypothesize that intelligibility may actually be harmful for applications that are very uncertain of their actions. We conducted a large controlled study of a location-aware and a sound-aware application, investigating the impact of intelligibility on understanding, and user impression of applications with varying certainty. We found that intelligibility impacts user impressions, depending on the application's certainty and behavior appropriateness. Intelligibility is helpful for applications with high certainty, but it is harmful if applications behave appropriately, yet display low certainty.

Context-aware applications are increasingly complex and autonomous, and research has indicated that explanations can help users better understand and ultimately trust their autonomous behavior. However, it is still unclear how to effectively present and provide these explanations. This work builds on previous work to make context-aware applications intelligible by supporting a suite of explanations using eight question types (e.g., Why, Why Not, What If). We present a formative study on design and usability issues for making an intelligible real-world, mobile context-aware application, focusing on the use of intelligibility for the mobile contexts of availability, place, motion, and sound activity. We discuss design strategies that we considered, findings of explanation use, and design recommendations to make intelligibility more usable.

Researchers have detailed the importance of routines in how people live and work, while also cautioning system designers about the importance of people's idiosyncratic behavior patterns and the challenges they would present to learning systems. We wish to take up their challenge, and offer a vision of how simple sensing technology could capture and model idiosyncratic routines, enabling applications to solve many real world problems.

Although network bandwidth has increased dramatically, high-resolution images often take several seconds to load, and considerably longer on mobile devices over wireless connections. Progressive image loading techniques allow for some visual content to be displayed prior to the whole file being downloaded. In this note, we present an empirical evaluation of popular progressive image loading methods, and derive one novel technique from our findings. Results suggest a spiral variation of bilinear interlacing can yield an improvement in content recognition time.

People strive to obtain self-knowledge. A class of systems called personal informatics is appearing that help people collect and reflect on personal information. However, there is no comprehensive list of problems that users experience using these systems, and no guidance for making these systems more effective. To address this, we conducted surveys and interviews with people who collect and reflect on personal information. We derived a stage-based model of personal informatics systems composed of five stages (preparation, collection, integration, reflection, and action) and identified barriers in each of the stages. These stages have four essential properties: barriers cascade to later stages; they are iterative; they are user-driven and/or system-driven; and they are uni-faceted or multi-faceted. From these properties, we recommend that personal informatics systems should 1) be designed in a holistic manner across the stages; 2) allow iteration between stages; 3) apply an appropriate balance of automated technology and user control within each stage to facilitate the user experience; and 4) explore support for associating multiple facets of people's lives to enrich the value of systems.

With a focus on presenting information at the right time, the ubicomp community can benefit greatly from learning the most salient human measures of cognitive load. Cognitive load can be used as a metric to determine when or whether to interrupt a user. In this paper, we collected data from multiple sensors and compared their ability to assess cognitive load. Our focus is on visual perception and cognitive speed-focused tasks that leverage cognitive abilities common in ubicomp applications. We found that across all participants, the electrocardiogram median absolute deviation and median heat flux measurements were the most accurate at distinguishing between low and high levels of cognitive load, providing a classification accuracy of over 80% when used together. Our contribution is a real-time, objective, and generalizable method for assessing cognitive load in cognitive tasks commonly found in ubicomp systems and situations of divided attention.

Context-aware applications should be intelligible so users can better understand how they work and improve their trust in them. However, providing intelligibility is non-trivial and requires the developer to understand how to generate explanations from application decision models. Furthermore, users need different types of explanations and this complicates the implementation of intelligibility. We have developed the Intelligibility Toolkit that makes it easy for application developers to obtain eight types of explanations from the most popular decision models of context-aware applications. We describe its extensible architecture, and the explanation generation algorithms we developed. We validate the usefulness of the toolkit with three canonical applications that use the toolkit to generate explanations for end-users.

As computing devices become more pervasive, our daily activities start generating a vast amount of information that could be exploited for helping us better understand ourselves. In this paper we present a system that uses easily available data correlated into a story-based representation aimed at providing users with a better understanding of their lifestyles. While this is still work in progress, we believe that it provides valuable insights into the design of such systems. Our initial findings show that user data generated through a person's daily activities can reveal a wealth of valuable information which they can use to adjust and improve their lifestyles.

We present two studies that evaluate the accuracy of human responses to an intelligent agent's data classification questions. Prior work has shown that agents can elicit accurate human responses, but the applications vary widely in the data features and prediction information they provide to the labelers when asking for help. In an initial analysis of this work, we found the five most popular features, namely uncertainty, amount and level of context, prediction of an answer, and request for user feedback. We propose that there is a set of these data features and prediction information that maximizes the accuracy of labeler responses. In our first study, we compare accuracy of users of an activity recognizer labeling their own data across the dimensions. In the second study, participants were asked to classify a stranger's emails into folders and strangers' work activities by interruptibility. We compared the accuracy of the responses to the users' self-reports across the same five dimensions. We found very similar combinations of information (for users and strangers) that led to very accurate responses as well as more feedback that the agents could use to refine their predictions. We use these results for insight into the information that help labelers the most.

In spite of intensive effort to resolve conflicts between multiple users of context-aware applications in a smart space, there has been no practical solution for flexibly resolving them based on the situation of the users. In this paper, we propose a mixed resolution method to combine automatic resolution with social participation for resolving multiuser conflicts. For combining the two resolution approaches, various contexts such as preferences, priority, and types of applications are used to select an appropriate resolution method for the encountered conflict. Through an evaluation, we found that the performance of selection algorithms mainly depended on the number of users and the similarity between their preferences, and we derived an appropriate threshold for determining whether users were similar or not. With a user study of 3 applications in a smart-space test-bed, we observed that the combination of automatic resolution when preferences are similar and social mediation when preferences are different effectively resolved multiuser conflict even though social pressure played an important role, and the 3 different applications had different thresholds.

Intelligibility and control are important user concerns in context-aware applications. They allow a user to understand why an application is behaving a certain way, and to change its behavior. Because of their importance to end users, they must be addressed at an interface level. However, often the sensors or machine learning systems that users need to understand and control are created long before a specific application is built, or created separately from the application interface. Thus, supporting interface designers in building intelligibility and control into interfaces requires application logic and underlying infrastructure to be exposed in some structured fashion. As context-aware infrastructures do not provide generalized support for this, we extended one such infrastructure with Situations, components that appropriately exposes application logic, and supports debugging and simple intelligibility and control interfaces, while making it easier for an application developer to build context-aware applications and facilitating designer access to application state and behavior. We developed support for interface designers in Visual Basic and Flash. We demonstrate the usefulness of this support through an evaluation of programmers, an evaluation of the usability of the new infrastructure with interface designers, and the augmentation of three common context-aware applications.

A common effect of aging is decline in spatial cognition. This is an issue for all elders, but particularly for elder drivers. To address this driving issue, we propose a novel concept of an in-vehicle navigation display system that displays navigation information directly onto the vehicle's windshield, superimposing it on the driver's view of the actual road. An evaluation of our simulated version of this display shows that it results in a significant reduction in navigation errors and distraction-related measures compared to a typical in-car navigation display for elder drivers. These results help us understand how context-sensitive information and a simulated augmented reality representation can be combined to minimize the cognitive load in translating between virtual/information spaces and the real world.

Context-aware intelligent systems employ implicit inputs, and make decisions based on complex rules and machine learning models that are rarely clear to users. Such lack of system intelligibility can lead to loss of user trust, satisfaction and acceptance of these systems. However, automatically providing explanations about a system's decision process can help mitigate this problem. In this paper we present results from a controlled study with over 200 participants in which the effectiveness of different types of explanations was examined. Participants were shown examples of a system's operation along with various automatically generated explanations, and then tested on their understanding of the system. We show, for example, that explanations describing why the system behaved a certain way resulted in better understanding and stronger feelings of trust. Explanations describing why the system did not behave a certain way, resulted in lower understanding yet adequate performance. We discuss implications for the use of our findings in real-world context-aware applications.

Intelligibility can help expose the inner workings and inputs of context-aware applications that tend to be opaque to users due to their implicit sensing and actions. However, users may not be interested in all the information that the applications can produce. Using scenarios of four real-world applications that span the design space of context-aware computing, we conducted two experiments to discover what information users are interested in. In the first experiment, we elicit types of information demands that users have and under what moderating circumstances they have them. In the second experiment, we verify the findings by soliciting users about which types they would want to know and establish whether receiving such information would satisfy them. We discuss why users demand certain types of information, and provide design implications on how to provide different intelligibility types to make context-aware applications intelligible and acceptable to users.

Almost all technology supported activities we perform today involve a device with computational power able to collect data that can give information about our activities as well as, with the appropriate sensors, about our physiological status. While recording user data is not particularly novel, in our work, we concern ourselves with methods that people can use to gather and analyze data about themselves as they go about everyday work and leisure activities in order to better support self-monitoring and self-understanding. Our aim is to enable people to detect causality relationships in their behaviours either for serving their curiosity or, as we hope, for really empowering them with a tool for self-changes. In this ‘work-in-progress’ paper we describe ongoing work towards these ends. We start out by a motivation and short description of our work, followed by an exemplar scenario on ‘preventive healthcare’ for the system we envision. Then we describe some of the most relevant and influential related work before describing the system design and the main challenges, followed by an overview of the current status and our future vision.

We present a novel approach for determining robot movements that efficiently accomplish the robot's tasks while not hindering the movements of people within the environment. Our approach models the goal-directed trajectories of pedestrians using maximum entropy inverse optimal control. The advantage of this modeling approach is the generality of its learned cost function to changes in the environment and to entirely different environments. We employ the predictions of this model of pedestrian trajectories in a novel incremental planner and quantitatively show the improvement in hindrance-sensitive robot trajectory planning provided by our approach.

Context-awareness is one of the rising trends of future mobile technology, and due to advances in technology development, new application and service concepts are being developed and demonstrated in an ever-increasing manner. This workshop brings together researchers and practitioners working on humancomputer interaction (HCI) aspects of context-aware mobile technology and communities to present their insights and research on new concepts, interaction design for mobile context-awareness, usability challenges, collaborative context-aware services and applications for supporting communities, and other topics related to HCI with mobile context-aware technology.

Asking questions is an inevitable part of collaborative interactions between humans and robots. However, robotics novices may have difficulty answering the robots' questions if they do not understand what the robot is asking. We are particularly interested in whether robots can supplement their questions with information about their state in a manner that increases the accuracy of human responses. In this work, we design and carefully analyze a human-robot collaborative task experiment to measure humans' responses and accuracies to different amounts of supplemental information. We vary the content of the questions along four dimensions of the robot state, namely uncertainty, context, predictions, and feature selection. Based on our results, we contribute guidelines on the effective combination of the four dimensions, under the assumption that the robot has no limitations on generating question context. Finally, we validate our guidelines against educated recommendations from the HRI community.

The size and resolution of computer displays has increased dramatically, allowing more information than ever to be rendered on-screen. However, items can now be so small or screens so cluttered that users need to lean forward to properly examine them. This behavior may be detrimental to a user's posture and eyesight. Our Lean and Zoom system detects a user's proximity to the display using a camera and magnifies the on-screen content proportionally. This alleviates dramatic leaning and makes items more readable. Results from a user study indicate people find the technique natural and intuitive. Most participants found on-screen content easier to read, and believed the technique would improve both their performance and comfort.

Experience sampling method (or ESM) is a common data collection method to understand user behavior and to evaluate ubiquitous computing technologies. However, ESM studies often demand too much time and commitment from participants, which leads to attrition and low compliance among participants. We introduce a new approach called experience sampling with feedback or ES+feedback that improves compliance by giving feedback to participants through various visualizations. Providing feedback to users makes the information personally relevant and increases the value of the study to participants, which increases their compliance. Our exploratory study shows that ES+feedback increases the compliance rate by 23%.

Lifelogging technologies have the potential to provide memory cues for people who struggle with episodic memory impairment (EMI). These memory cues enable the recollection of significant experiences, which is important for people with EMI to regain a sense of normalcy in their lives. However, lifelogging technologies often collect an overwhelmingly large amount of data to review. The best memory cues need to be extracted and presented in a way that best supports episodic recollection. We describe the design of a new lifelogging system that captures photos, ambient audio, and location information and leverages both automated content/context analysis and the expertise of family caregivers to facilitate the extraction and annotation of a salient summary consisting of good cues from the lifelog. The system presents the selected cues for review in a way that maximizes the opportunities for the person with EMI to think deeply about these cues to trigger memory recollection on his own without burdening the caregiver. We compare our system with another review system that requires the caregiver to repeatedly guide the review process. Our self-guided system resulted in better memory retention and imposed a smaller burden on the caregiver whereas the caregiver-guided approach provided more opportunities for caregiver interaction.

A number of technologies have contributed to automatically resolving resource conflicts between multiple users in a smart space. However, such systems eliminate the users' ability to perform this conflict resolution by themselves, which they actually prefer to do in certain circumstances. Since both resolution approaches have their merits, we propose a mixed-initiative conflict resolution system, which combines automatic conflict resolution with mediated, or user-driven, resolution by exploiting contextual information in context-aware applications. An evaluation of our system found that users prefer to use a mediated resolution approach when their preferences about outcome are very different from others', but have no preferred method when their preferences about outcome are similar to others'.

We present PROCAB, an efficient method for Probabilistically Reasoning from Observed Context-Aware Behavior. It models the context-dependent utilities and underlying reasons that people take different actions. The model generalizes to unseen situations and scales to incorporate rich contextual information. We train our model using the route preferences of 25 taxi drivers demonstrated in over 100,000 miles of collected data, and demonstrate the performance of our model by inferring: (1) decision at next intersection, (2) route to known destination, and (3) destination given partially traveled route.

Alzheimer's disease impairs episodic memory and subtly and progressively robs people of their ability to remember their recent experiences. In this paper, we describe two studies that lead to a better understanding of how caregivers use cues to support episodic memory impairment and what types of cues are best for supporting recollection. We also show how good memory cues differ between people with and without episodic memory impairment. We discuss how this improved understanding impacts the design of lifelogging technologies for automatically capturing and extracting the best memory cues to assist overburdened caregivers and people with episodic memory impairment in supporting recollection of episodic memory.

In order to develop intelligent systems that attain the trust of their users, it is important to understand how users perceive such systems and develop those perceptions over time. We present an investigation into how users come to understand an intelligent system as they use it in their daily work. During a six-week field study, we interviewed eight office workers regarding the operation of a system that predicted their managers' interruptibility, comparing their mental models to the actual system model. Our results show that by the end of the study, participants were able to discount some of their initial misconceptions about what information the system used for reasoning about interruptibility. However, the overarching structures of their mental models stayed relatively stable over the course of the study. Lastly, we found that participants were able to give lay descriptions attributing simple machine learning concepts to the system despite their lack of technical knowledge. Our findings suggest an appropriate level of feedback for user interfaces of intelligent systems, provide a baseline level of complexity for user understanding, and highlight the challenges of making users aware of sensed inputs for such systems.

A common task in graphical user interfaces is controlling onscreen elements using a pointer. Current adaptive pointing techniques require applications to be built using accessibility libraries that reveal information about interactive targets, and most do not handle path/menu navigation. We present a pseudo-haptic technique that is OS and application independent, and can handle both dragging and clicking. We do this by associating a small force with each past click or drag. When a user frequently clicks in the same general area (e.g., on a button), the patina of past clicks naturally creates a pseudo-haptic magnetic field with an effect similar to that ofsnapping or sticky icons. Our contribution is a bottom-up approach to make targets easier to select without requiring prior knowledge of them.

Computer displays can be helpful for making users aware of the remote presence of friends and family. In many of the research projects that have explored the use of novel displays, the real goal is to improve a user's sense of connectedness to those remote loved ones. However, very few have leveraged a user-centered design process or empirically studied the effects of using a display on users' sense of awareness and connectedness. In this paper, we present our multi-phase, user-centered design process for building displays that support awareness and connectedness: Presence Displays, which are physical, peripheral awareness displays of online presence of close friends or family. We present evidence, from a 5-week long field study, that these displays provide significantly better awareness of and connectedness to a loved one, than a traditional graphical display of online presence.

In interspecies households, it is common for the non homo sapien members to be isolated and ignored for many hours each day when humans are out of the house or working. For pack animals, such as canines, information about a pack member's extended pack interactions (outside of the nuclear household) could help to mitigate this social isolation. We have developed a Pack Activity Watch System: Allowing Broad Interspecies Love In Telecommunication with Internet-Enabled Sociability (PAWSABILITIES) for helping to support remote awareness of social activities. Our work focuses on canine companions, and includes, pawticipatory design, labradory tests, and canid camera monitoring.

Many context-aware services make the assumption that the context they use is completely accurate. However, in reality, both sensed and interpreted context is often ambiguous. A challenge facing the development of realistic and deployable context-aware services, therefore, is the ability to handle ambiguous context. Although some of this ambiguity may be resolved using automatic techniques, we argue that correct handling of ambiguous context will often need to involve the user. We use the term mediation to refer to the dialogue that ensues between the user and the system. In this article, we describe an architecture that supports the building of context-aware services that assume context is ambiguous and allows for mediation of ambiguity by mobile users in aware environments. We present design guidelines that arise from supporting mediation over space and time, issues not present in the graphical user interface domain where mediation has typically been used in the past. We illustrate the use of our architecture and evaluate it through an example context-aware application, a word predictor system.

Context-aware applications are applications that implicitly take their context of use into account by adapting to changes in a user's activities and environments. No one has more intimate knowledge about these activities and environments than end-users themselves. Currently there is no support for end-users to build context-aware applications for these dynamic settings. To address this issue, we present a CAPpella, a programming by demonstration Context-Aware Prototyping environment intended for end-users. Users "program" their desired context-aware behavior (situation and associated action) in situ, without writing any code, by demonstrating it to a CAPpella and by annotating the relevant portions of the demonstration. Using a meeting and medicine-taking scenario, we illustrate how a user can demonstrate different behaviors to a CAPpella. We describe a CAPpella's underlying system to explain how it supports users in building behaviors and present a study of 14 end-users to illustrate its feasibility and usability.

Traditionally, computer interfaces have been confined to conventional displays and focused activities. However, as displays become embedded throughout our environment and daily lives, increasing numbers of them must operate on the periphery of our attention. Peripheral displays can allow a person to be aware of information while she is attending to some other primary task or activity. We present the Peripheral Displays Toolkit (PTK), a toolkit that provides structured support for managing user attention in the development of peripheral displays. Our goal is to enable designers to explore different approaches to managing user attention. The PTK supports three issues specific to conveying information on the periphery of human attention. These issues are abstraction of raw input, rules for assigning notification levels to input, and transitions for updating a display when input arrives. Our contribution is the investigation of issues specific to attention in peripheral display design and a toolkit that encapsulates support for these issues. We describe our toolkit architecture and present five sample peripheral displays demonstrating our toolkit's capabilities.

Infrastructure software comprises code libraries or runtime processes that support the development or operation of application software. A particular infrastructure system may support certain styles of application, and may even determine the features of applications built using it. This poses a challenge: although we have good techniques for designing and evaluating interactive applications, our techniques for designing and evaluating infrastructure intended to support these applications are much less well formed. In this paper, we reflect on case studies of two infrastructure systems for interactive applications. We look at how traditional user-centered techniques, while appropriate for application design and evaluation, fail to properly support infrastructure design and evaluation. We present a set of lessons from our experience, and conclude with suggestions for better user-centered design and evaluation of infrastructure software.

We present a technique for evaluating the usability and effectiveness of ambient displays. Ambient displays are abstract and aesthetic peripheral displays portraying non-critical information on the periphery of a user's attention. Although many innovative displays have been published, little existing work has focused on their evaluation, in part because evaluation of ambient displays is difficult and costly. We adapted a low-cost evaluation technique, heuristic evaluation, for use with ambient displays. With the help of ambient display designers, we defined a modified set of heuristics. We compared the performance of Nielsen's heuristics and our heuristics on two ambient displays. Evaluators using our heuristics found more, severe problems than evaluators using Nielsen's heuristics. Additionally, when using our heuristics, 3-5 evaluators were able to identify 40--60% of known usability issues. This implies that heuristic evaluation is an effective technique for identifying usability issues with ambient displays.

One of the first, most common, and most useful applications that today's computer users access is the World Wide Web (web). One population of users for whom the web is especially important is those with motor disabilities, because it may enable them to do things that they might not otherwise be able to do: shopping; getting an education; running a business. This is particularly important for low bandwidth users: users with such limited motor and speech that they can only produce one or two signals when communicating with a computer. We present requirements for low bandwidth web accessibility, and two tools that address these requirements. The first is a modified web browser, the second a proxy that modifies HTML. Both work without requiring web page authors to modify their pages.

Many context-aware services make the assumption that the context they use is completely accurate. However, in reality, both sensed and interpreted context is often ambiguous. A challenge facing the development of realistic and deployable context-aware services, therefore, is the ability to handle ambiguous context. In this paper, we describe an architecture that supports the building of context-aware services that assume context is ambiguous and allows for mediation of ambiguity by mobile users in aware environments. We illustrate the use of our architecture and evaluate it through three example context-aware services, a word predictor system, an In/Out Board, and a reminder tool.

In the future, a largely invisible and ubiquitous computing infrastructure \nwill assist people with a variety of activities in the home and at work. \nThe applications that will be deployed in such systems will create and \nmanipulate private information and will provide access to a variety of other \nresources. Securing such applications is challenging for a number of \nreasons. Unlike traditional systems where access control has been explored, \naccess decisions may depend on the context in which requests are made. We \nshow how the well-developed notion of roles can be used to capture \nsecurity-relevant context of the environment in which access requests are \nmade. By introducing environment roles, we create a uniform access control \nframework that can be used to secure context-aware applications. We also \npresent a security architecture that supports security policies that make \nuse of environment roles to control access to resources.

Computing devices and applications are now used beyond the desktop, in diverse environments, and this trend toward ubiquitous computing is accelerating. One challenge that remains in this emerging research field is the ability to enhance the behavior of any application by informing it of the context of its use. By context, we refer to any information that characterizes a situation related to the interaction between humans, applications, and the surrounding environment. Context-aware applications promise richer and easier interaction, but the current state of research in this field is still far removed from that vision. This is due to 3 main problems: (a) the notion of context is still ill defined, (b) there is a lack of conceptual models and methods to help drive the design of context-aware applications, and (c) no tools are available to jump-start the development of context-aware applications. In this anchor article, we address these 3 problems in turn. We first define context, identify categories of contextual information, and characterize context-aware application behavior. Though the full impact of context-aware computing requires understanding very subtle and high-level notions of context, we are focusing our efforts on the pieces of context that can be inferred automatically from sensors in a physical environment. We then present a conceptual framework that separates the acquisition and representation of context from the delivery and reaction to context by a context-aware application. We have built a toolkit, the Context Toolkit, that instantiates this conceptual framework and supports the rapid development of a rich space of context-aware applications. We illustrate the usefulness of the conceptual framework by describing a number of context-aware applications that have been prototyped using the Context Toolkit. We also demonstrate how such a framework can support the investigation of important research challenges in the area of context-aware computing.

Context-enabled applications are just emerging and promise richer interaction by taking environmental context into account. However, they are difficult to build due to their distributed nature and the use of unconventional sensors. The concepts of toolkits and widget libraries in graphical user interfaces has been tremendously successtil, allowing programmers to leverage off existing building blocks to build interactive systems more easily. We introduce the concept of context widgets that mediate between the environment and the application in the same way graphical widgets mediate between the user and the application. We illustrate the concept of context widgets with the beginnings of a widget library we have developed for sensing presence, identity and activity of people and things. We assess the success of our approach with two example context-enabled applications we have built and an existing application to which we have added context-sensing capabilities.

Applications are often designed to take advantage of the potential for integration with each other via shared information. Current approaches for integration are limited, affecting both the programmer and end-user. In this paper, we present CyberDesk, a framework for self-integrating software in which integration is driven by user context. It relieves the burden on programmers by removing the necessity to predict how software should be integrated. It also relieves the burden from users by removing the need to understand how to make different software components work together. q 1998 Elsevier Science B.V. All rights reserved.

No abstract available.

No abstract available.