Application developers frequently augment their code to produce event logs of specific operations performed by their users. Subsequent analysis of these event logs can help provide insight about the users' behavior relative to its intended use. The analysis process typically includes both event organization and pattern discovery activities. However, most existing visual analytics systems for interaction log analysis excel at supporting pattern discovery and overlook the importance of flexible event organization. This omission limits the practical application of these systems. Therefore, we developed a novel visual analytics system called IntiVisor that implements the entire end-to-end interaction analysis approach. An evaluation of the system with interaction data from four visualization applications showed the value and importance of supporting event organization in interaction log analysis.

Algorithmic estimations of worker behavior are gaining popularity. Passive Sensing–enabled AI (PSAI) systems leverage behavioral traces from workers’ digital tools to infer their experience. Despite their conceptual promise, the practical designs of these systems elicit tensions that lead to workers resisting adoption. This paper teases apart the monolithic representation of PSAI by investigating system components that maximize value and mitigate concerns. We conducted an interactive online survey using the Experimental Vignette Method. Using Linear Mixed-effects Models we found that PSAI systems were more acceptable when sensing digital time use or physical activity, instead of visual modes. Inferences using language were only acceptable in work-restricted contexts. Compared to insights into performance, workers preferred insights into mental wellbeing. However, they resisted systems that automatically forwarded these insights to others. Our findings provide a template to reflect on existing systems and plan future implementations of PSAI to be more worker-centered.

Early detection and intervention for relapse is important in the treatment of schizophrenia spectrum disorders. Researchers have developed AI models to predict relapse from patient-contributed data like social media. However, these models face challenges, including misalignment with practice and ethical issues related to transparency, accountability, and potential harm. Furthermore, how patients who have recovered from schizophrenia view these AI models has been underexplored. To address this gap, we first conducted semi-structured interviews with 28 patients and reflexive thematic analysis, which revealed a disconnect between AI predictions and patient experience, and the importance of the social aspect of relapse detection. In response, we developed a prototype that used patients' Facebook data to predict relapse. Feedback from seven patients highlighted the potential for AI to foster collaboration between patients and their support systems, and to encourage self-reflection. Our work provides insights into human-AI interaction and suggests ways to empower people with schizophrenia.

In the mental health domain, patient engagement is key to designing human-centered technologies. CSCW and HCI researchers have delved into various facets of collaboration in AI research; however, previous research neglects the individuals who both produce the data and will be most impacted by the resulting technologies, such as patients. This study examines how interdisciplinary researchers and mental health patients who donate their data for AI research collaborate and how we can improve human-centeredness in mental health AI research. We interviewed patient participants, AI researchers, and clinical researchers in a federally funded mental health AI research project. We used the concept of boundary objects to understand stakeholder collaboration. Our findings reveal that the social media data provided by patient participants functioned as boundary objects that facilitated stakeholder collaboration. Although the collaboration appeared to be successful, we argue that building consensus, or understanding each other's perspectives, can improve the human-centeredness of mental health AI research. Based on the findings, we provide suggestions for human-centered mental health AI research, working with data donors as domain experts, making invisible work visible, and privacy implications.

Today’s electronics are manufactured to provide stable functionality and fixed physical forms optimized for reliable operation over long periods and repeated use. However, even when applications don’t call for such robustness, the permanency of these electronics comes with environmental consequences. In this paper, we describe an alternative approach that utilizes sustainable transient electronics whose method of destruction is also key to their functionality. We create these electronics through three different methods: 1) by inkjet printing conductive silver traces on poly(vinyl alcohol) (PVA) substrates to create water-soluble sensors; 2) by mixing a conductive beeswax material configured as a meltable sensor; and 3) by fabricating edible electronics with 3D printed chocolate and culinary gold leaf. To enable practical applications of these devices, we implement a fully transient and sustainable chipless RF detection system.

In this paper, we propose SwellSense, a fabrication technique to screen print stretchable circuits onto a special micro-capsule paper, creating localized swelling patterns with sensing capabilities. This simple technique will allow users to create a wide range of paper-based tactile interactive devices, which are mostly maintaining 2D planar form factor but can also be curved or folded into 3D interactive artifacts. We frst present the design guidelines to support various tactile interaction design including basic tactile graphic geometries, patterns with directional density, or fner interactive textures with embedded sensing such as touch sensor, pressure sensor, and mechanical switch. We then provide a design editor to enable users to design more creatively using the SwellSense technique. We provide a technical evaluation and user evaluation to validate the basic performance of SwellSense. Lastly, we demonstrate several application examples and conclude with a discussion on current limitations and future work.

We are witnessing an emergence in Passive Sensing enabled AI (PSAI) to provide dynamic insights for performance and wellbeing of information workers. Hybrid work paradigms have simultaneously created new opportunities for PSAI, but have also fostered anxieties of misuse and privacy intrusions within a power asymmetry. At this juncture, it is unclear if those who are sensed can find these systems acceptable. We conducted scenario-based interviews of 28 information workers to highlight their perspectives as data subjects in PSAI. We unpack their expectations using the Contextual Integrity framework of privacy and information gathering. Participants described appropriateness of PSAI based on its impact on job consequences, work-life boundaries, and preservation of flexibility. They perceived that PSAI inferences could be shared with selected stakeholders if they could negotiate the algorithmic inferences. Our findings help envision worker-centric approaches to implementing PSAI as an empowering tool in the future of work.

The growing use of computing and proliferation of computing devices requires a holistic focus on sustainability as the environmental impacts of computing technologies go beyond their energy consumption.Environmental impacts span all stages of the lifecycle -manufacturing, operation, and disposal.A sustainability mindset must permeate all organizations involved in design, manufacturing, operation, and disposal/recycling of computational devices.This panel will discuss current and new research on sustainability in computing that spans the full lifecycle, all layers of the computing stack and across the computing spectrum from edge to cloud.The panel will also discuss potential crossdisciplinary approaches to sustainable computing and new notions and metrics to quantify sustainability.

The growing use of computing and proliferation of computing devices requires a holistic focus on sustainability as the environmental impacts of computing technologies go beyond their energy consumption. Environmental impacts span all stages of the lifecycle - manufacturing, operation, and disposal. A sustainability mindset must permeate all organizations involved in design, manufacturing, operation, and disposal/recycling of computational devices. This panel will discuss current and new research on sustainability in computing that spans the full lifecycle, all layers of the computing stack and across the computing spectrum from edge to cloud. The panel will also discuss potential cross-disciplinary approaches to sustainable computing and new notions and metrics to quantify sustainability.

Transition planning is a collaborative process to promote agency in students with disabilities by encouraging them to participate in setting their own goals with team members and learn ways to assess their progress towards the goals. For autistic young adults who experience a lower employment rate, less stability in employment, and lower community connections than those with other disabilities, successful transition planning is an important opportunity to develop agency towards preparing and attaining success in employment and other areas meaningful to them. However, a failure of consistent information sharing among team members and opportunities for agency in students has prevented successful transition planning for autistic students. Therefore, this work brings causal agency theory and the collaborative reflection framework together to uncover ways transition teams can develop students' agency by collaboratively reflecting on students' inputs related to transition goals and progress. By interviewing autistic students, parents of autistic students, and professionals who were involved in transition planning, we uncovered that teams can better support student agency by accommodating their needs and encouraging their input in annual meetings, building relationships through transparent and frequent communication about day-to-day activities, centering goals on student's interests, and supporting student's skill-building in areas related to their transition goals. However, we found that many teams were not enacting these practices, leading to frustration and negative outcomes for young adults. Based on our findings, we propose a role for autistic students in the collaborative reflection framework that encouraged participation and builds causal agency. We also make design recommendations to encourage autistic students' participation in collaborative reflection around long-term and short-term needs in ways that promote their causal agency.

Social media platforms are being utilized by individuals with mental illness for engaging in self-disclosure, finding support, or navigating treatment journeys. Individuals also increasingly bring their social media data to psychotherapy consultations. This emerging practice during psychotherapy can help us to better understand how patients appropriate social media technologies to develop and iterate patient narratives -- the stories of patients' own experiences that are vital in mental health treatment. In this paper, we seek to understand patients' perspectives regarding why and how they bring up their social media activities during psychotherapy consultations as well as related concerns. Through interviews with 18 mood disorder patients, we found that social media helps augment narratives around interpersonal conflicts, digital detox, and self-expression. We also found that discussion of social media activities shines a light on the power imbalance and privacy concerns regarding use of patient-generated health information. Based on the findings, we discuss that social media data are different from other types of patient-generated health data in terms of supporting patient narratives because of the social interactions and curation social media inherently engenders. We also discuss privacy concerns and trust between a patient and a therapist when patient narratives are supported by patients' social media data. Finally, we suggest design implications for social computing technologies that can foster patient narratives rooted in social media activities.

This work presents PITAS, a thin-sheet robotic material composed of a reversible phase transition actuating layer and a heating/sensing layer. The synthetic sheet material enables non-expert makers to create shape-changing devices that can locally or remotely convey physical information such as shape, color, texture and temperature changes. PITAS sheets can be manipulated into various 2D shapes or 3D geometries using subtractive fabrication methods such as laser, vinyl, or manual cutting or an optional additive 3D printing method for creating 3D objects. After describing the design of PITAS, this paper also describes a study conducted with thirteen makers to gauge the accessibility, design space, and limitations encountered when PITAS is used as a soft robotic material while designing physical information communication devices. Lastly, this work reports on the results of a mechanical and electrical evaluation of PITAS and presents application examples to demonstrate its utility.

Contact tracers assist in containing the spread of highly infectious diseases such as COVID-19 by engaging community members who receive a positive test result in order to identify close contacts. Many contact tracers rely on community member’s recall for those identifications, and face limitations such as unreliable memory. To investigate how technology can alleviate this challenge, we developed a visualization tool using de-identified location data sensed from campus WiFi and provided it to contact tracers during mock contact tracing calls. While the visualization allowed contact tracers to find and address inconsistencies due to gaps in community member’s memory, it also introduced inconsistencies such as false-positive and false-negative reports due to imperfect data, and information sharing hesitancy. We suggest design implications for technologies that can better highlight and inform contact tracers of potential areas of inconsistencies, and further present discussion on using imperfect data in decision making.

When modeling passive data to infer individual mental wellbeing, a common source of ground truth is self-reports. But these tend to represent the psychological facet of mental states, which might not align with the physiological facet of that state. Our paper demonstrates that when what people “feel” differs from what people “say they feel”, we witness a semantic gap that limits predictions. We show that predicting mental wellbeing with passive data (offline sensors or online social media) is related to how the ground-truth is measured (objective arousal or self-report). Features with psycho-social signals (e.g., language) were better at predicting self-reported anxiety and stress. Conversely, features with behavioral signals (e.g., sleep), were better at predicting stressful arousal. Regardless of the source of ground truth, integrating both signals boosted prediction. To reduce the semantic gap, we provide recommendations to evaluate ground truth measures and adopt parsimonious sensing.

We have witnessed explosive growth in computing devices at all scales, in particular with small wireless devices that can permeate most of our physical world. The IoT industry is helping to fuel this insatiable desire for more and more data. We have to balance this growth with an understanding of its environmental impact. Indeed, the ENSsys community must take leadership in putting sustainability up front as a primary design principle for the future of IoT and related areas, expanding the research mandate beyond the intricacies of the computing systems in isolation to encompass and integrate the materials, new applications, and circular lifecycle of electronics in the IoT. Our call to action is seeded with a circularity-focused computing agenda that demands a cross-stack research program for energy-harvesting computational things.

Deviant eating behavior such as skipping meals and consuming unhealthy meals has a significant association with mental well-being in college students. However, there is more to what an individual eats. While eating patterns form a critical component of their mental well-being, insights and assessments related to the interplay of eating patterns and mental well-being remain under-explored in theory and practice. To bridge this gap, we use an existing real-time eating detection system that captures context during meals to examine how college students’ eating context associates with their mental well-being, particularly their affect, anxiety, depression, and stress. Our findings suggest that students’ irregularity or skipping meals negatively correlates with their mental well-being, whereas eating with family and friends positively correlates with improved mental well-being. We discuss the implications of our study in designing dietary intervention technologies and guiding student-centric well-being technologies.

Improving end-users’ awareness of cybersecurity warnings (e.g., phishing and malware alerts) remains a longstanding problem in usable security. Prior work suggests two key weaknesses with existing warnings: they are primarily communicated via saturated communication channels (e.g., visual, auditory, and vibrotactile); and, they are communicated rationally, not viscerally. We hypothesized that wrist-based affective haptics should address both of these weaknesses in a form-factor that is practically deployable: i.e., as a replaceable wristband compatible with modern smartwatches like the Apple Watch. To that end, we designed and implemented Spidey Sense, a wristband that produces customizable squeezing sensations to alert users to urgent cybersecurity warnings. To evaluate Spidey Sense, we applied a three-phased ‘Gen-Rank-Verify’ study methodology with 48 participants. We found evidence that, relative to vibrotactile alerts, Spidey Sense was considered more appropriate for the task of alerting people to cybersecurity warnings.

Airflow energy harvesting has attracted much attention in the community of energy harvesting, but the efforts to make it accessible for individual users have been limited. In this paper, we address this issue and demonstrate a comprehensive toolkit, Exergy, which can help and support novice users to simulate, design, and manufacture an airflow energy harvester for vehicles.

Augmenting everyday surfaces with interaction sensing capability that is maintenance-free, low-cost (∼ $1), and in an appropriate form factor is a challenge with current technologies. MARS (Multi-channel Ambiently-powered Realtime Sensing) enables battery-free sensing and wireless communication of touch, swipe, and speech interactions by combining a nanowatt programmable oscillator with frequency-shifted analog backscatter communication. A zero-threshold voltage field-effect transistor (FET) is used to create an oscillator with a low startup voltage (∼ 500 mV) and current (< 2uA), whose frequency can be affected through changes in inductance or capacitance from the user interactions. Multiple MARS systems can operate in the same environment by tuning each oscillator circuit to a different frequency range. The nanowatt power budget allows the system to be powered directly through ambient energy sources like photodiodes or thermoelectric generators. We differentiate MARS from previous systems based on power requirements, cost, and part count and explore different interaction and activity sensing scenarios suitable for indoor environments.

Bluetooth requires device pairing to ensure security in data transmission, encumbering a number of ad-hoc, transactional interactions that require both ease-of-use and “good enough” security: e.g., sharing contact information or secure links to people nearby. We introduce Bit Whisperer, an ad-hoc short-range wireless communication system that enables “walk up and share” data transmissions with “good enough” security. Bit Whisperer transmits data to proximate devices co-located on a solid surface through high frequency, inaudible acoustic signals. The physical surface has two benefits: it enhances acoustic signal transmission by reflecting sound waves as they propagate; and, it makes the domain of communication visible, helping users identify exactly with whom they are sharing data without prior pairing. Through a series of technical evaluations, we demonstrate that Bit Whisperer is robust for common use-cases and secure against likely threats. We also implement three example applications to demonstrate the utility of Whisperer: 1-to-1 local contact sharing, 1-to-N private link sharing to open a secure group chat, and 1-to-N local device authentication.

A number of computing departments have adapted studio-based pedagogy for teaching Human-Computer Interaction (HCI) to help students develop creative design skills. These efforts have shown promise, however, some challenges that emerge when integrating studio-based pedagogy into computing programs have been neglected by researchers. This research focuses on these challenges in the implementation and evaluation of a studio-based approach integrated into three HCI-related courses. This study is based on 6 classes, 44 interviews with students, observations, and interviews with members of the teaching team. Findings supported many positive outcomes from previous studio-based implementations. Nevertheless, students felt the change in pedagogy frustrating and it may have hindered their creativity, motivation, and engagement. This paper highlights these conflicting narratives, comparing computing culture with studio-based pedagogy. We conclude with considerations for future implementations.

Organizational culture (OC) encompasses the underlying beliefs, values, and practices that are unique to an organization. However, OC is inherently subjective and a coarse construct, and therefore challenging to quantify. Alternatively, self-initiated workplace reviews on online platforms like Glassdoor provide the opportunity to leverage the richness of language to understand OC. In as much, first, we use multiple job descriptors to operationalize OC as a word vector representation. We validate this construct with language used in 650k different Glassdoor reviews. Next, we propose a methodology to apply our construct on Glassdoor reviews to quantify the OC of employees by sector. We validate our measure of OC on a dataset of 341 employees by providing empirical evidence that it helps explain job performance. We discuss the implications of our work in guiding tailored interventions and designing tools for improving employee functioning.

We present Self-powered Paper INterfaces (SPIN) combining folding paper creases with triboelectric nanogenerator (TENG). Embedding TENG into paper creases, we developed a design editor and set of fabrication techniques to create paper-based interfaces that power sensors and actuators. Our SPIN design editor enables users to design their own crease pattern by changing parameters, embed power generating modules into the design, estimate total power generation, and export the files. Then following the fabrication instructions, users can cut and crease materials, and assemble them to build their own interfaces. We employ repetitive push-and-pull based embodied interactions with the mechanism of paper creases and demonstrate four application examples that show new expressive possibilities applying different scales of embodied interactions.

We introduce Serpentine, a self-powered sensor that is a reversibly deformable cord capable of sensing a variety of human input. The material properties and structural design of Serpentine allow it to be flexible, twistable, stretchable and squeezable, enabling a broad variety of expressive input modalities. The sensor operates using the principle of Triboelectric Nanogenerators (TENG), which allows it to sense mechanical deformation without an external power source. The affordances of the cord include six interactions---Pluck, Twirl, Stretch, Pinch, Wiggle and Twist. Serpentine demonstrates the ability to simultaneously recognize these inputs through a single physical interface. A 12-participant user study illustrates 95.7% accuracy for a user-dependent recognition model using a realtime system and 92.17% for user-independent offline detection. We conclude by demonstrating how Serpentine can be employed in everyday ubiquitous computing applications.

Developing systems for Human Activity Recognition (HAR) using wearables typically relies on datasets that were manually annotated by human experts with regards to precise timings of instances of relevant activities. However, obtaining such data annotations is often very challenging in the predominantly mobile scenarios of Human Activity Recognition. As a result, labels often carry a degree of uncertainty-label jitter-with regards to: i) correct temporal alignments of activity boundaries; and ii) correctness of the actual label provided by the human annotator. In this work, we present a scheme that explicitly incorporates label jitter into the model training process. We demonstrate the effectiveness of the proposed method through a systematic experimental evaluation on standard recognition tasks for which our method leads to significant increases of mean F1 scores.

We present Surface++, which leverages our previous work SATURN, a self-powered flexible acoustic sensor, and ZEUSSS, a passive wireless sound communication technique using analog backscatter, to create a scalable and self-sustainable wireless sound sensing surface. Our new prototype allows for large area acoustic sensing using modular fabrication techniques with the promise of being fully printable. A single small Surface++ patch can be used to extend voice and gesture input for everyday surfaces, while our more sensitive Surface++ modular array allows for large-area context sensing and localization.

Organizations often strive to recruit and retain individuals who would be a "good fit" with their core values, beliefs and practices. Person-Organization (P-O congruence is known to explain employee satisfaction, commitment and absenteeism. This paper proposes a new measure of P-O fit by empirically investigating the similarity of routine within an organization. This measure of routine fit is motivated by the theory of entrainment, which refers to the synchrony of individual and community behaviors. We use unobtrusive bluetooth sensing to examine how the concurrence of latent activity patterns is related to job performance and wellbeing. Routine fit echoes traditional constructs of congruence as it is significantly related to higher task performance and lower workplace deviance. Additionally however, it is also related to greater stress and higher arousal. Prior work in organizational psychology have used single-occasion survey instruments to infer uni-dimensional models of fit. These methods are limited by subjective perceptions of employees. In contrast, we demonstrate a data-driven and multidimensional approach to study normative routines in an organization as a measure of P-O fit. We discuss the potential of our approach in designing technologies that understand the congruence of employee routines and positively impact employee functioning at the workplace.

FingerPing is a novel sensing technique that can recognize various fine-grained hand poses by analyzing acoustic resonance features. A surface-transducer mounted on a thumb ring injects acoustic chirps (20Hz to 6,000Hz) to the body. Four receivers distributed on the wrist and thumb collect the chirps. Different hand poses of the hand create distinct paths for the acoustic chirps to travel, creating unique frequency responses at the four receivers. We demonstrate how FingerPing can differentiate up to 22 hand poses, including the thumb touching each of the 12 phalanges on the hand as well as 10 American sign language poses. A user study with 16 participants showed that our system can recognize these two sets of poses with an accuracy of 93.77% and 95.64%, respectively. We discuss the opportunities and remaining challenges for the widespread use of this input technique.

Feature extraction is a critical step in sliding-window based standard activity recognition chains. Recently, distribution based features have been introduced that showed excellent generalization capabilities across a wide range of application domains in human activity recognition scenarios based on body-worn sensors. These features capture the data distribution of individual analysis frames, yet they ignore temporal structure inherent to the signal of a frame. We explore four variants of adding temporal structure to distribution based features and demonstrate their potential for statistically significant improvements of activity recognition in general. The addition of temporal structure comes with a moderate increase in computational complexity rendering the proposed methods applicable to mobile and embedded scenarios.

Sliding window based activity recognition chains represent the state-of-the-art for many mobile and embedded scenarios as they are common in wearable computing. The length of the analysis frames is a crucial system parameter that directly influences the effectiveness of the overall approach. In this paper we present a method that optimizes the window length - individually for each target activity. Instead of employing a single, multi-class recognition system that is based on a generic window length, we combine individually optimized activity detectors into an Ensemble based recognition approach. We demonstrate the effectiveness of the approach through an experimental evaluation on eight benchmark datasets. The proposed method leads to significant improvements across a range of activity recognition application domains.

Washing hands is one of the easiest yet most effective ways to prevent spreading illnesses and diseases. However, not adhering to thorough handwashing routines is a substantial problem worldwide. For example, in hospital operations lack of hygiene leads to healthcare associated infections. We present WristWash, a wrist-worn sensing platform that integrates an inertial measurement unit and a Hidden Markov Model-based analysis method that enables automated assessments of handwashing routines according to recommendations provided by the World Health Organization (WHO). We evaluated Wrist-Wash in a case study with 12 participants. WristWash is able to successfully recognize the 13 steps of the WHO handwashing procedure with an average accuracy of 92% with user-dependent models, and with 85% for user-independent modeling. We further explored the system's robustness by conducting another case study with six participants, this time in an unconstrained environment, to test variations in the hand-washing routine and to show the potential for real-world deployments.

The proliferation of high resolution and affordable virtual reality (VR) headsets is quickly making room-scale VR experiences available in our homes. Most VR experiences strive to achieve complete immersion by creating a disconnect from the real world. However, due to the lack of a standardized notification management system and minimal context awareness in VR, an immersed user may face certain situations such as missing an important phone call (digital scenario), tripping over wandering pets (physical scenario), or losing track of time (temporal scenario). In this paper, we present the results of 1) a survey across 61 VR users to understand common interruptions and scenarios that would benefit from some form of notifications; 2) a design exercise with VR professionals to explore possible notification methods; and 3) an empirical study on the noticeability and perception of 5 different VR interruption scenarios across 6 modality combinations (e.g., audio, visual, haptic, audio + haptic, visual + haptic, and audio + visual) implemented in Unity and presented using the HTC Vive headset. Finally, we combine key learnings from each of these steps along with participant feedback to present a set of observations and recommendations for notification design in VR.

Driving is the second highest expense for the average American household. Yet few people know the total cost of owning and operating their vehicles, and most cannot estimate accurately how much a common driving trip (like a daily commute) costs. There are an increasing number of viable alternatives for personal transportation, such as car services (e.g. Uber, Lyft), in addition to ridesharing, transit, biking, and walking. Cost is one factor in transportation mode choice, and awareness of the cost of driving is useful in making better informed decisions. To bridge this awareness gap, we built and deployed a system that makes the total cost of each driving trip (including depreciation, maintenance, insurance, and fuel) visible to the user. After this intervention, participants were able to more accurately and confidently estimate costs of their driving commutes, and transfer this knowledge to other trips for which they had not seen a cost.

In this paper we develop a model capable of classifying drivers from their driving behaviors sensed by only low level sensors. The sensing platform consists of data available from the diagnostic outlet (OBD) of the car and smartphone sensors. We develop a window based support vector machine model to classify drivers. We test our model with two datasets collected under both controlled and naturalistic conditions. Furthermore, we evaluate the model using each sensor source (car and phone) independently and combining both the sensors. The average classification accuracies attained with data collected from three different cars shared between couples in a naturalistic environment were 75.83%, 85.83% and 86.67% using only phone sensors, only cars sensors and combined car and phone sensors respectively.

The touchscreen has been the dominant input surface for smartphones and smartwatches. However, its small size compared to a phone limits the richness of the input gestures that can be supported. We present TapSkin, an interaction technique that recognizes up to 11 distinct tap gestures on the skin around the watch using only the inertial sensors and microphone on a commodity smartwatch. An evaluation with 12 participants shows our system can provide classification accuracies from 90.69% to 97.32% in three gesture families -- number pad, d-pad, and corner taps. We discuss the opportunities and remaining challenges for widespread use of this technique to increase input richness on a smartwatch without requiring further on-body instrumentation.

Most smartphones today have a rich set of sensors that could be used to infer input (e.g., accelerometer, gyroscope, microphone); however, the primary mode of interaction is still limited to the front-facing touchscreen and several physical buttons on the case. To investigate the potential opportunities for interactions supported by built-in sensors, we present the implementation and evaluation of BeyondTouch, a family of interactions to extend and enrich the input experience of a smartphone. Using only existing sensing capabilities on a commodity smartphone, we offer the user a wide variety of additional inputs on the case and the surface adjacent to the smartphone. Although most of these interactions are implemented with machine learning methods, compact and robust rule-based detection methods can also be applied for recognizing some interactions by analyzing physical characteristics of tapping events on the phone. This article is an extended version of Zhang et al. [2015], which solely covered gestures implemented by machine learning methods. We extended our previous work by adding gestures implemented with rule-based methods, which works well with different users across devices without collecting any training data. We outline the implementation of both machine learning and rule-based methods for these interaction techniques and demonstrate empirical evidence of their effectiveness and usability. We also discuss the practicality of BeyondTouch for a variety of application scenarios and compare the two different implementation methods.

We report on a nine-month-long observational study with teachers and students with autism in a classroom setting. We explore the impact of motion-based activities on students' behavior. In particular, we examine how the playful gaming activity impacted students' engagement, peer-directed social behaviors, and motor skills. We document the effectiveness of a collaborative game in supporting initiation of social activities between peers, and in eliciting novel body movements that students were not observed to produce outside of game play. We further identify the positive impact of game play on overall classroom engagement. This includes an "audience effect" whereby non-playing peers direct initiations to those playing the game and vice versa, and a positive "spillover" effect of the activity on students' social behavior outside of game play. We identify key considerations for designing and deploying motion-based activities for children with autism in a classroom setting.

Although food journaling is understood to be both important and difficult, little work has empirically documented the specific challenges people experience with food journals. We identify key challenges in a qualitative study combining a survey of 141 current and lapsed food journalers with analysis of 5,526 posts in community forums for three mobile food journals. Analyzing themes in this data, we find and discuss barriers to reliable food entry, negative nudges caused by current techniques, and challenges with social features. Our results motivate research exploring a wider range of approaches to food journal design and technology.

Difficulty in navigating daily life can lead to frustration and decrease independence for people with autism. While they turn to online autism communities for information and advice for coping with everyday challenges, these communities may present only a limited perspective because of their in-group nature. Obtaining support from out-group sources beyond the in-group community may prove valuable in dealing with challenging situations such as public anxiety and workplace conflicts. In this paper, we explore the value of supplementary out-group support from crowdsourced responders added to in-group support from a community of members. We find that out-group sources provide relatively rapid, concise responses with direct and structured information, socially appropriate coping strategies without compromising emotional value. Using an autism community as a motivating example, we conclude by providing design implications for combining in-group and out-group resources that may enhance the question-and-answer experience.

Recognizing when eating activities take place is one of the key challenges in automated food intake monitoring. Despite progress over the years, most proposed approaches have been largely impractical for everyday usage, requiring multiple on-body sensors or specialized devices such as neck collars for swallow detection. In this paper, we describe the implementation and evaluation of an approach for inferring eating moments based on 3-axis accelerometry collected with a popular off-the-shelf smartwatch. Trained with data collected in a semi-controlled laboratory setting with 20 subjects, our system recognized eating moments in two free-living condition studies (7 participants, 1 day; 1 participant, 31 days), with F-scores of 76.1% (66.7% Precision, 88.8% Recall), and 71.3% (65.2% Precision, 78.6% Recall). This work represents a contribution towards the implementation of a practical, automated system for everyday food intake monitoring, with applicability in areas ranging from health research and food journaling.

Dietary self-monitoring has been shown to be an effective method for weight-loss, but it remains an onerous task despite recent advances in food journaling systems. Semi-automated food journaling can reduce the effort of logging, but often requires that eating activities be detected automatically. In this work we describe results from a feasibility study conducted in-the-wild where eating activities were inferred from ambient sounds captured with a wrist-mounted device; twenty participants wore the device during one day for an average of 5 hours while performing normal everyday activities. Our system was able to identify meal eating with an F-score of 79.8% in a person-dependent evaluation, and with 86.6% accuracy in a person-independent evaluation. Our approach is intended to be practical, leveraging off-the-shelf devices with audio sensing capabilities in contrast to systems for automated dietary assessment based on specialized sensors.

While most smartphones today have a rich set of sensors that could be used to infer input (.e.g., accelerometer, gyroscope, microphone), the primary mode of interaction is still limited to the front-facing touchscreen and several physical buttons on the case. To investigate the potential opportunities for interactions supported by built-in sensors, we present the implementation and evaluation of BeyondTouch, a family of interactions to extend and enrich the input experience of a smartphone. Using only existing sensing capabilities on a commodity smartphone, we offer the user a wide variety of additional tapping and sliding inputs on the case of and the surface adjacent to the smartphone. We outline the implementation of these interaction techniques and demonstrate empirical evidence of their effectiveness and usability. We also discuss the practicality of BeyondTouch for a variety of application scenarios.

The pervasiveness of mobile cameras has resulted in a dramatic increase in food photos, which are pictures reflecting what people eat. In this paper, we study how taking pictures of what we eat in restaurants can be used for the purpose of automating food journaling. We propose to leverage the context of where the picture was taken, with additional information about the restaurant, available online, coupled with state-of-the-art computer vision techniques to recognize the food being consumed. To this end, we demonstrate image-based recognition of foods eaten in restaurants by training a classifier with images from restaurant's online menu databases. We evaluate the performance of our system in unconstrained, real-world settings with food images taken in 10 restaurants across 5 different types of food (American, Indian, Italian, Mexican and Thai).

Abstract Developmental psychology researchers examine the temporal relationships of social and communicative behaviors, such as how a child responds to a name call, to understand early typical and atypical development and to discover early signs of autism and developmental delay. These related behaviors occur together or within close temporal proximity, forming unique patterns and relationships of interest. However, the task of finding these early signs, which are in the form of atypical behavioral patterns, becomes more challenging when behaviors of multiple children at different ages need to be compared with each other in search of generalizable patterns. The ability to visually explore the temporal relationships of behaviors, including flexible redefinition of closeness, over multiple social interaction sessions with children of different ages, can make such knowledge extraction easier. We have designed a visualization tool called TipoVis that helps psychology researchers visually explore the temporal patterns of social and communicative behaviors. We present two case studies to show how TipoVis helped two researchers derive new understandings of their data.

The last fifty years have seen an exponential increase in the amount of technology embedded in and passing though home environments, creating spaces that are now technologically as well as socially, psychologically and environmentally complex. It is only in the last fifteen years that the study of Human-Computer Interaction in the home has entered the mainstream, with essential challenges and themes (Edwards & Grinter 2001, Blythe & Monk 2002) defined, and groundbreaking environments for research created and analyzed (Kidd et al., 1999). Diverse aspects within the domestic use of technologies, from childcare to health and wellbeing, to food, media, and energy, have now become the focus of detailed bodies of research in their own right. This trend has brought with it unique methodological concerns which need to be considered by those researching and designing interactions with technologies in the home (Coughlan et al., 2013). This special issue has emerged from an ongoing interest within the Human-Computer Interaction community in the practice of performing research and design activities in and about the home. While home based technology research has been performed and widely reported previously we focus on the methods themselves, exploring the development of novel methods, and the implication of applying existing methods in home environments.

Mobile sensor data-to-knowledge (MD2K) was chosen as one of 11 Big Data Centers of Excellence by the National Institutes of Health, as part of its Big Data-to-Knowledge initiative. MD2K is developing innovative tools to streamline the collection, integration, management, visualization, analysis, and interpretation of health data generated by mobile and wearable sensors. The goal of the big data solutions being developed by MD2K is to reliably quantify physical, biological, behavioral, social, and environmental factors that contribute to health and disease risk. The research conducted by MD2K is targeted at improving health through early detection of adverse health events and by facilitating prevention. MD2K will make its tools, software, and training materials widely available and will also organize workshops and seminars to encourage their use by researchers and clinicians.

This editorial introduction describes the aims and scope of the ACM Transactions on Interactive Intelligent Systems special issue on Activity Recognition for Interaction. It explains why activity recognition is becoming crucial as part of the cycle of interaction between users and computing systems, and it shows how the five articles selected for this special issue reflect this theme.

As CSCW creates and investigates technologies for social communication, it is important to understand the emotional benefits and costs of these systems. We propose the Affec-tive Benefits and Costs of Communication Technologies (ABCCT) questionnaire to supplement traditional qualita-tive methods of understanding communication media. We discuss the pilots of this survey with 45 children and 110 adults measuring the inter-item reliability of this instru-ment. We present the results of interviews with 14 children and 14 adults, which help confirm that the ABCCT measures the same constructs that may emerge through interview investigations. Finally, we demonstrate that the ABCCT is sensitive enough to discriminate between differ-ent communication technologies and has shown promise in some of its early adoption. Though the ABCCT is not with-out limitations, it may provide a way to compare technolo-gies in field deployments, draw findings across investiga-tions, and quantify the impact of specific design decisions.

The recent emergence of comfortable wearable sensors has focused almost entirely on monitoring physical activity, ignoring opportunities to monitor more subtle phenomena, such as the quality of social interactions. We argue that it is compelling to address whether physiological sensors can shed light on quality of social interactive behavior. This work leverages the use of a wearable electrodermal activity (EDA) sensor to recognize ease of engagement of children during a social interaction with an adult. In particular, we monitored 51 child-adult dyads in a semi-structured play interaction and used Support Vector Machines to automatically identify children who had been rated by the adult as more or less difficult to engage. We report on the classification value of several features extracted from the child's EDA responses, as well as several other features capturing the physiological synchrony between the child and the adult.

A child's failure to respond to his or her name being called is an early warning sign for autism and response to name is currently assessed as a part of standard autism screening and diagnostic tools. In this paper, we explore markerless child head tracking as an unobtrusive approach for automatically predicting child response to name. Head turns are used as a proxy for visual attention. We analyzed 50 recorded response to name sessions with the goal of predicting if children, ages 15 to 30 months, responded to name calls by turning to look at an examiner within a defined time interval. The child's head turn angles and hand annotated child name call intervals were extracted from each session. Human assisted tracking was employed using an overhead Kinect camera, and automated tracking was later employed using an additional forward facing camera as a proof-of-concept. We explore two distinct analytical approaches for predicting child responses, one relying on rule-based approached and another on random forest classification. In addition, we derive child response latency as a new measurement that could provide researchers and clinicians with finer grain quantitative information currently unavailable in the field due to human limitations. Finally we reflect on steps for adapting our system to work in less constrained natural settings.

Interaction design methodologies have become increasingly popular in the design and development of technologies for children with special needs. However, designing within this area remains fraught with difficulties. This workshop aims to explore the issues that occur when working with children with special needs and seeks to establish a set of guidelines for interaction design researchers who are working with such a diverse group. This will be achieved through the discussion of a number of themes that have highlighted as important when designing within this complicated area.

Autism is characterized by language delays and difficulties in social interactions. Remote synchronous communication technologies may provide children with autism with new opportunities to practice social interaction when in-person interaction may not be possible. We conducted an empirical investigation, asking children with autism to interact with a teacher using two different videochat prototypes. We found that the teachers were able to engage the children in social interactions using videochat, using certain techniques to draw the children away from distractions or fixations. We describe the effective strategies used by the teachers and discuss opportunities and challenges in using a traditional video-conferencing layout compared to an alternative side-by-side layout.

Building social support networks is crucial both for less-independent individuals with autism and for their primary caregivers. In this paper, we describe a four-week exploratory study of a social network service (SNS) that allows young adults with autism to garner support from their family and friends. We explore the unique benefits and challenges of using SNSs to mediate requests for help or advice. In particular, we examine the extent to which specialized features of an SNS can engage users in communicating with their network members to get advice in varied situations. Our findings indicate that technology-supported communication particularly strengthened the relationship between the individual and extended network members, mitigating concerns about over-reliance on primary caregivers. Our work identifies implications for the design of social networking services tailored to meet the needs of this special needs population.

We deployed the ShareTable - a system that provides easy-to-initiate videochat and a shared tabletop task space - in four divorced households. Throughout the month of its use, the families employed the ShareTable to participate in shared activities, share emotional moments, and communicate closeness through metaphorical touch. The ShareTable provided a number of advantages over the phone and was easier to use than standard videoconferencing. However, it did also introduce concerns over privacy and new sources of conflict about appropriate calling practices. We relate our findings to the larger research landscape and present implications for future work.

We introduce a new problem domain for activity recognition: the analysis of children's social and communicative behaviors based on video and audio data. We specifically target interactions between children aged 1-2 years and an adult. Such interactions arise naturally in the diagnosis and treatment of developmental disorders such as autism. We introduce a new publicly-available dataset containing over 160 sessions of a 3-5 minute child-adult interaction. In each session, the adult examiner followed a semi-structured play interaction protocol which was designed to elicit a broad range of social behaviors. We identify the key technical challenges in analyzing these behaviors, and describe methods for decoding the interactions. We present experimental results that demonstrate the potential of the dataset to drive interesting research questions, and show preliminary results for multi-modal activity recognition.

This paper introduces a low cost, fast and accessible technology to support the rapid prototyping of functional electronic devices. Central to this approach of 'instant inkjet circuits' is the ability to print highly conductive traces and patterns onto flexible substrates such as paper and plastic films cheaply and quickly. In addition to providing an alternative to breadboarding and conventional printed circuits, we demonstrate how this technique readily supports large area sensors and high frequency applications such as antennas. Unlike existing methods for printing conductive patterns, conductivity emerges within a few seconds without the need for special equipment. We demonstrate that this technique is feasible using commodity inkjet printers and commercially available ink, for an initial investment of around US$300. Having presented this exciting new technology, we explain the tools and techniques we have found useful for the first time. Our main research contribution is to characterize the performance of instant inkjet circuits and illustrate a range of possibilities that are enabled by way of several example applications which we have built. We believe that this technology will be of immediate appeal to researchers in the ubiquitous computing domain, since it supports the fabrication of a variety of functional electronic device prototypes.

First-person point-of-view (FPPOV) images taken by wearable cameras can be used to better understand people's eating habits. Human computation is a way to provide effective analysis of FPPOV images in cases where algorithmic approaches currently fail. However, privacy is a serious concern. We provide a framework, the privacy-saliency matrix, for understanding the balance between the eating information in an image and its potential privacy concerns. Using data gathered by 5 participants wearing a lanyard-mounted smartphone, we show how the framework can be used to quantitatively assess the effectiveness of four automated techniques (face detection, image cropping, location filtering and motion filtering) at reducing the privacy-infringing content of images while still maintaining evidence of eating behaviors throughout the day.

The social world that most of us navigate effortlessly can prove to be a perplexing and disconcerting place for individuals with autism. Interactive tools to teach social skills that are personalized to the individual's needs show promise, but it is challenging to author them. We describe the design, development, and preliminary evaluation of an approach to using human computation that enables the creation of models of complex and interesting social scenarios, possible obstacles that may arise in those scenarios, and potential solutions to those obstacles. Our preliminary evaluation of the models confirms that these models have the potential to help an author create a social skills instructional module.

Independence is key to a successful transition to adulthood for individuals with autism. Social support is a crucial factor for achieving adaptive self-help life skills. In this paper we describe the results of a formative design exercise with young adults with autism and their caregivers to uncover opportunities for social networks to promote independence and facilitate coordination. We propose the concept of SocialMirror, a device connected to an online social network that allows the young adult to seek advice from a trusted and responsive network of family, friends and professionals. Focus group discussions reveal the potential for SocialMirror to increase motivation to learn everyday life skills among young adults with autism and to foster collaboration among a distributed care network. We present design considerations to leverage a small trusted network that balances quick response with safeguards for privacy and security of young adults with autism.

Weiser's landmark Scientific American article inspired many researchers to explore an exciting socio-technical vision of a third generation of computing. At the 21st anniversary of that published vision, I want to assess ubicomp's maturity and explore the identity challenge it faces. Today, ubicomp as a niche research topic no longer makes sense; we must celebrate its "disappearance" as a well-scoped research agenda because it has become a profound agenda across most of computing, and beyond. This should not be surprising; the 2nd generation of computing, the personal computer revolution, experienced the same profound disappearance. In celebration of this imminent disappearance, I will highlight the unique contributions of the ubicomp community, express some remaining intellectual challenges, and speculate on how to formulate new visions of computing that might succeed this third generation.

Severe behavior problems of children with developmental disabilities often require intervention by specialists. These specialists rely on direct observation of the behavior, usually in a controlled clinical environment. In this paper, we present a technique for using on-body accelerometers to assist in automated classification of problem behavior during such direct observation. Using simulated data of episodes of severe behavior acted out by trained specialists, we demonstrate how machine learning techniques can be used to segment relevant behavioral episodes from a continuous sensor stream and to classify them into distinct categories of severe behavior (aggression, disruption, and self-injury). We further validate our approach by demonstrating it produces no false positives when applied to a publicly accessible dataset of activities of daily living. Finally, we show promising classification results when our sensing and analysis system is applied to data from a real assessment session conducted with a child exhibiting problem behaviors.

Activity recognition in the home has been long recognized as the foundation for many desirable applications in fields such as home automation, sustainability, and healthcare. However, building a practical home activity monitoring system remains a challenge. Striking a balance between cost, privacy, ease of installation and scalability continues to be an elusive goal. In this paper, we explore infrastructure-mediated sensing combined with a vector space model learning approach as the basis of an activity recognition system for the home. We examine the performance of our single-sensor water-based system in recognizing eleven high-level activities in the kitchen and bathroom, such as cooking and shaving. Results from two studies show that our system can estimate activities with overall accuracy of 82.69% for one individual and 70.11% for a group of 23 participants. As far as we know, our work is the first to employ infrastructure-mediated sensing for inferring high-level human activities in a home setting.

We describe a system for detecting moments of eye contact between an adult and a child, based on a single pair of gaze-tracking glasses which are worn by the adult. Our method utilizes commercial gaze tracking technology to determine the adult's point of gaze, and combines this with computer vision analysis of video of the child's face to determine their gaze direction. Eye contact is then detected as the event of simultaneous, mutual looking at faces by the dyad. We report encouraging findings from an initial implementation and evaluation of this approach.

We present the evaluation of BrailleTouch, an accessible keyboard for blind users on touchscreen smartphones. Based on the standard Perkins Brailler, BrailleTouch implements a six-key chorded braille soft keyboard. Eleven blind participants typed for 165 twenty-minute sessions on three mobile devices: 1) BrailleTouch on a smartphone; 2) a soft braille keyboard on a touchscreen tablet; and 3) a commercial braille keyboard with physical keys. Expert blind users averaged 23.2 words per minute (wpm) on the BrailleTouch smartphone. The fastest participant, a touchscreen novice, achieved 32.1 wpm during his first session. Overall, participants were able to transfer their existing braille typing skills to a touchscreen device within an hour of practice. We report the speed for braille text entry on three mobile devices, an in depth error analysis, and the lessons learned for the design and evaluation of accessible and eyes-free soft keyboards.

We present the findings from a cross-cultural study of the expectations and perceptions of individuals with autism and other intellectual disabilities (AOID) in Kuwait, Pakistan, South Korea, and the United States. Our findings exposed cultural nuances that have implications for the design of assistive technologies. We develop a framework, based on three themes; 1) lifestyle; 2) socio-technical infrastructure; and 3) monetary and informational resources within which the cultural implications and opportunities for assistive technology were explored. The three key contributions of this work are: 1) the development of a framework that outlines how culture impacts perceptions and expectations of individuals with social and intellectual disabilities; 2) a mapping of how this framework leads to implications and opportunities for assistive technology design; 3) the presentation of concrete examples of how these implications impact the design of three emerging assistive technologies.

Previously, we presented Viz-A-Vis, a VIsualiZation of Activity through computer VISion [17]. Viz-A-Vis visualizes behavior as aggregate motion over observation space. In this paper, we present two complementary user studies of Viz-A-Vis measuring its performance and discovery affordances. First, we present a controlled user study aimed at comparatively measuring behavioral analysis preference and performance for observation and search tasks. Second, we describe a study with architects measuring discovery affordances and potential impacts on their work practices. We conclude: 1) Viz-A-Vis significantly reduced search time; and 2) it increased the number and quality of insightful discoveries.

Parents and children in families living with regular separation due to work develop strategies to manage being apart. We interviewed 14 pairs of parents and children (ages 7 -- 13) from work-separated families to understand their experiences and the strategies that they use to keep their family together. Parents focus on combining scheduled synchronous and spontaneous asynchronous communication to maintain a constant presence in the life of the child. Children, on the other hand, focus on other sources of support, on other activities, and on the eventual reunion. Both the remote parent and the child rely heavily on a collocated adult to maintain aware-ness and contact. We compare work-separated families with other types of separation and highlight opportunities for new designs.

Texting is the essence of mobile communication and connectivity, as evidenced by today's teenagers, tomorrow's workforce. Fifty-four percent of American teens contact each other daily by texting, as compared to face-to-face (33%) and talking on the phone (30%) according to the Pew Research Center's Internet & American Life Project, 2010. Arguably, today's technologies support mobile text input poorly, primarily due to the size constraints of mobile devices. This is the case for everyone, but it is particularly relevant to the visually impaired. According to the World Health Organization, 284 million people are visually impaired worldwide. In order to connect these users to the global mobile community, we need to design effective and efficient methods for eyes-free text input on mobile devices. Furthermore, everyone would benefit from effective mobile texting for safety and speed. This design brief presents BrailleTouch, our working prototype solution for eyes-free mobile text input.

We present an empirical study of the long-term practicality of using human motion to generate operating power for body-mounted consumer electronics and health sensors. We have collected a large continuous acceleration data set from eight experimental subjects going about their normal daily routine for three days each. Each subject is instrumented with a data collection apparatus that simultaneously logs 3-axis, 80 Hz acceleration data from six body locations. We use this data set to optimize a first-principles physical model of the commonly used velocity damped resonant generator (VDRG) by selecting physical parameters such as resonant frequency and damping coefficient to maximize the harvested power. Our results show that with reasonable assumptions on size, mass, placement, and efficiency of VDRG harvesters, most body-mounted wireless sensors and even some consumer electronics devices can be powered continuously and indefinitely from everyday motion. We have optimized the power harvesters for each individual and for each body location. In addition, we present the potential of designing a damping- and frequency-tunable power harvester that could mitigate the power reduction of a generator generalized for "average” subjects. We present the full details on the collection of the acceleration data sets, the development of the VDRG model, and a numerical simulator, and discuss some of the future challenges that remain in this promising field of research.

Individuals with Autism Spectrum Disorder (ASD) often engage in stereotypical behaviors. In some individuals these behaviors occur with very high frequency and can be disruptive and at times self-injurious. We propose a system that can tacitly collect contextual data related to the individual's physiological state and their external environment, and map it to occurrences of stereotypies. A user study was conducted with children with ASD, parents, and caregivers to explore and validate this concept. A prototype of the system, developed through participatory design, was used in the study as a probe to elicit the information needs of these stakeholders, and provide a better understanding of the nuances involved in supporting those needs. Here we present the findings of this study, and four design recommendations; promoting ecological integration, addressing privacy concerns, supporting inference, and enabling customization.

We present a novel approach to the causal temporal analysis of event data from video content. Our key observation is that the sequence of visual words produced by a space-time dictionary representation of a video sequence can be interpreted as a multivariate point-process. By using a spectral version of the pairwise test for Granger causality, we can identify patterns of interactions between words and group them into independent causal sets. We demonstrate qualitatively that this produces semantically-meaningful groupings, and we demonstrate quantitatively that these groupings lead to improved performance in retrieving and classifying social games from unstructured videos.

A persistent concern of wireless sensors is the power consumption required for communication, which presents a significant adoption hurdle for practical ubiquitous computing applications. This work explores the use of the home powerline as a large distributed antenna capable of receiving signals from ultra-low-power wireless sensor nodes and thus allowing nodes to be detected at ranges that are otherwise impractical with traditional over-the-air reception. We present the design and implementation of small ultra-low-power 27 MHz sensor nodes that transmit their data by coupling over the powerline to a single receiver attached to the powerline in the home. We demonstrate the ability of our general purpose wireless sensor nodes to provide whole-home coverage while consuming less than 1 mW of power when transmitting (65 ¼W consumed in our custom CMOS transmitter). This is the lowest power transmitter to date compared to those found in traditional whole-home wireless systems.

While supporting family communication has traditionally been a domain of interest for interaction designers, few research initiatives have explicitly investigated remote synchronous communication between children and parents. We discuss the design of the ShareTable, a media space that supports synchronous interaction with children by augmenting videoconferencing with a camera-projector system to allow for shared viewing of physical artifacts. We present an exploratory evaluation of this system, highlighting how such a media space may be used by families for learning and play activities. The ShareTable was positively received by our participants and preferred over standard videoconferencing. Informed by the results of our exploratory evaluation, we discuss the next design iteration of the ShareTable and directions for future investigations in this area.

Parents of young children often want to keep a variety of records on their children's early years, for the purposes of preservation of memories or at the request of their pediatrician. However, time constraints, motivation, and forgetfulness may hinder their ability to keep consistent records. We developed a system, Baby Steps, which is designed to improve the record-keeping process. In this paper, we present the results of a 3-month deployment study of this technology with 8 families and their pediatricians. The study showed that when compared to a control condition, experimental design features of Baby Steps encouraged parents to more frequently collect and review records, provided higher confidence in reporting, and improved parent-pediatrician communication.

A new problem of retrieving social games from unstructured videos is proposed. Social games are characterized by repetitions (with variations) of alternating turns between two players. We define games as quasi-periodic motion patterns in video based on their repetitiveness property. We have developed an algorithm to extract such patterns from video. The patterns extracted by our method, from video clips of social games taken from YouTube, are shown to correspond to meaningful stages of the games. We demonstrate promising results in retrieving social games from unstructured, lab-recorded footage of children's play, and identifying social interactions in a dataset of approximately 3.75 hours of home movies.

Summary form only given. Many technologists have been wooed by the vision of pervasive or ubiquitous computing. We must recall that much of the initial visions of this field urged a human-centric view of the challenges and opportunities. Much of the initial efforts in our field have addressed important technical challenges, but for the overall health and longevity of pervasive computing, we need to think beyond our own relatively small communities. In this talk, we try to show how our field must turn outward and think beyond the formulation of problems that we alone understand. We show some examples that use health, education and the home as case studies.

Years ago I was trained as a software engineering researcher, well versed in the mathematical abstractions of programming. However, influenced by the writings and research of Mark Weiser and his Xerox PARC Ubiquitous Computing group, I started to think about a different way of doing research in computing. Computing technology, as Weiser predicted, has become so prevalent in our lives that we almost don't recognize everything that is computationally-based in our everyday world.

In this paper, we describe the design considerations and implementation of the Child'sPlay system, a technology for supporting the automatic recording, recognition, and quantification of a child's object play behaviors for retrospective analysis. Our prototype system consists of six varieties of toys augmented with wireless sensing capabilities and a mobile computing platform which uses statistical pattern recognition techniques to automatically classify sensed play behaviors. This paper discusses our choices in toy design both in form factor as well as sensing capabilities. In addition, we also describe the play activities the system supports and provide an overview of our initial recognition algorithms.

Identifying the function of problem behavior can lead to the development of more effective interventions. One way to identify the function is through functional behavior assessment (FBA). Teachers conduct FBA in schools. However, the task load of recording the data manually is high, and the challenge of accurately identifying antecedents and consequences is significant while interacting with students. These issues often result in imperfect information capture. CareLog allows teachers more easily to conduct FBAs and enhances the capture of relevant information. In this paper, we describe the design process that led to five design principles that governed the development of CareLog. We present results from a five-month, quasi-controlled study aimed at validating those design principles. We reflect on how various constraints imposed by special education settings impact the design and evaluation process for HCI practitioners and researchers.

Within a group of peers, it is often useful or interesting to know whether someone in the group has gone to bed or whether they have awakened in the morning. This information, naturally integrated as a peripheral augmentation of an alarm clock, allows people to know whether it is appropriate to make a call or feel more connected with someone living remotely. In this paper, we present the design and evaluation of such an alarm clock, the BuddyClock, and describe how it enables users in a small social network to automatically share information about their sleeping behaviors with one another. Through 3-6 week deployment studies of this technology with five different social networks, we found that the alarm clock affected participant behaviors and allowed them to feel more connected to those with whom they shared their sleeping behaviors.

Fingerprinting techniques for indoor localization have been widely explored. A particular approach by Patel et al. suggested leveraging of the residential powerline as the signaling mechanism for a domestic location capability. In this paper, we critically examine that initial work, called powerline positioning (PLP). We find the proposed technique lacking in temporal stability, requiring frequent and undesired recalibration in some environments. We also determine that there is no a priori method to determine a pair of signaling frequencies that will reliably work in any space. We propose a wideband approach to PLP (WPLP) that injects up to 44 different frequencies into the powerline. We show that this WPLP approach improves upon overall positioning accuracy, demonstrates greatly improved temporal stability and has the added advantage of working in commercial indoor spaces.

We present an empirical study of the long-term practicality of using human motion to generate operating power for body-mounted consumer electronics and health sensors. We have collected a large continuous acceleration dataset from eight experimental subjects going about their normal daily routine for 3 days each. Each subject is instrumented with a data collection apparatus that simultaneously logs 3-axis, 80Hz acceleration data from six body locations. We use this dataset to optimize a first-principles physical model of the commonly used velocity damped resonant generator (VDRG) by selecting physical parameters such as resonant frequency and damping coefficient to maximize harvested power. Our results show that with reasonable assumptions on size, mass, placement, and efficiency of VDRG harvesters, most body-mounted wireless sensors and even some consumer electronics devices, may be powered continuously and indefinitely from everyday motion.

A number of prototype meeting capture applications have been created in the past decade, yet relatively little research has focused on the review and long-term use of real captured meeting information. To that end, we have implemented a system called TeamSpace for capturing and reviewing general meetings. In this article, we describe the long-term deployment of TeamSpace to a university research group, along with a pseudo-controlled study involving the same group of users and their meetings. We gained a detailed understanding of the behavioral patterns involved in reviewing meeting content and how to improve on the experience. Our evaluations also demonstrate several of the barriers and challenges in realizing the potential benefits of meeting capture.

In this paper, we propose the UbiFloorII, a novel floor-based user identification system to recognize humans based on their stepping pattern, the arrays of the transitional footprints from heel-strike to toe-off. To obtain users' stepping pattern from their gait, we deployed photo interrupter sensors instead of switch sensors used in the UbiFloorI. We developed a software module to extract stepping pattern from users' gait. For user identification, we employed neural network trained with users' stepping samples. We achieved about 92% recognition accuracy using this floor-based approach. The UbiFloorII system may be used to automatically and transparently identify users in a home environment.

We describe the two-year-long development and evaluation of the Proportionality Method, a design method intended to aid HCI practitioners in designing advanced IT applications with complex privacy implications. The method is inspired by Data Protection Authorities' (DPA) and Courts' practice and proposes to balance the impact on privacy of IT applications with their usefulness. We discuss the results of an evaluation of the design method to verify its usability, usefulness and effectiveness vis-à-vis other design methods proposed in the HCI literature to address similar issues. Results suggest that different design methods for privacy highlight different sets of issues and a combination of methods should be employed in a comprehensive design process. We propose to judge design methods based on their overall quantitative and qualitative merits, including the type of application and technology for which they are most fit and their methodological approach. We finally propose to develop a privacy toolbox, that is, a set of heuristic methods that designers can choose from with knowledge and understanding of their relative advantages and limitations.

In the established procedural model of information visualization, the first operation is to transform raw data into data tables [1]. The transforms typically include abstractions that aggregate and segment relevant data and are usually defined by a human, user or programmer. The theme of this paper is that for video, data transforms should be supported by low level computer vision. High level reasoning still resides in the human analyst, while part of the low level perception is handled by the computer. To illustrate this approach, we present Viz-A-Vis, an overhead video capture and access system for activity analysis in natural settings over variable periods of time. Overhead video provides rich opportunities for long-term behavioral and occupancy analysis, but it poses considerable challenges. We present initial steps addressing two challenges. First, overhead video generates overwhelmingly large volumes of video impractical to analyze manually. Second, automatic video analysis remains an open problem for computer vision.

From birth through age five, children undergo rapid development and learn skills that will influence them their entire lives. Regular visits to the pediatrician and detailed record-keeping can ensure that children are progressing and can identify early warning signs of developmental delay or disability. However, new parents are often overwhelmed with new responsibilities, and we believe there is an opportunity for computing technology to assist in this process. In this paper, we present a qualitative study aimed at uncovering some specific needs for record-keeping and analysis for new parents and their network of caregivers. Through interviews and focus groups, we have confirmed assumptions about the rationales parents have and the functions required for using technology for record-keeping. We also identify new themes, potential prototypes, and design guidelines for this domain.

We describe a unique form of hands-free interaction that can be implemented on most commodity computing platforms. Our approach supports blowing at a laptop or computer screen to directly control certain interactive applications. Localization estimates are produced in real-time to determine where on the screen the person is blowing. Our approach relies solely on a single microphone, such as those already embedded in a standard laptop or one placed near a computer monitor, which makes our approach very cost-effective and easy-to-deploy. We show example interaction techniques that leverage this approach.

Recent research suggests design pre-patterns, structured collections of evidence-based research and design knowledge, provide a useful resource for design activities in emerging application domains. This paper extends previous research by exploring the impact of pre-patterns and tools to support pre-pattern exploration for the domain of ubiquitous computing in the home. We conducted an empirical study of 44 designers engaged in a two hour concentrated brainstorming and design task for the home of the future. Our results show pre-patterns are an easily adopted resource for designers that can impact even the earliest of design activities. We also provide insights for future development of pre-patterns based on designer feedback.

Storyboarding is a common technique in HCI and design for demonstrating system interfaces and contexts of use. Despite its recognized benefits, novice designers still encounter challenges in the creation of storyboards. Furthermore, as computing becomes increasingly integrated into the environment, blurring the distinction between the system and its surrounding context, it is imperative to depict context explicitly in storyboards. In this paper, we present two formative studies designed to uncover the important elements of storyboards. These elements include the use of text, inclusion of people, level of detail, number of panels, and representation of the passage of time. We further present an empirical study to assess the effects of these elements on the understanding and enjoyment of storyboard consumers. Finally, we demonstrate how these guidelines were successfully used in an undergraduate HCI class.

Finding lost items is a common problem for the visually impaired and is something that computing technology can help alleviate. In this paper, we present the design and evaluation of a mobile solution, called FETCH, for allowing the visually impaired to track and locate objects they lose frequently but for which they do not have a specific strategy for tracking. FETCH uses devices the user already owns, such as their cell phone or laptop, to locate objects around their house. Results from a focus group with visually impaired users informed the design of the system. We then studied the usability of a laptop solution in a laboratory study and studied the usability and usefulness of the system through a one-month deployment and diary study. These studies demonstrate that FETCH is usable and useful, but there is still room for improvement.

Evidence-based care is an increasingly popular process for long term diagnosis and monitoring of education and healthcare disabilities. Because this evidence must also be collected in everyday life, it is a technique that can greatly benefit from automated capture technologies. These solutions, however, can raise significant concerns about privacy, control, and surveillance. In this paper, we present an analysis of these concerns with regard to evidence-based care. This analysis underscores the need to consider community-based risk and reward analyses in addition to the traditionally used analyses for individual users when designing socially appropriate technologies.

We developed an inquiry technique, which we called "paratype," based on experience prototyping and event-contingent experience sampling, to survey people in real-life situations about ubiquitous computing (ubicomp) technology. We used this tool to probe the opinions of the conversation partners of users of the Personal Audio Loop, a memory aid that can have a strong impact on their privacy. We present the findings of this study and their implications, specifically the need to broaden public awareness of ubicomp applications and the unfitness of traditional data protection guidelines for tackling the privacy issues of many ubicomp applications. We also point out benefits and methodological issues of paratypes and discuss why they are particularly fit for studying certain classes of mobile and ubicomp applications.

Teams of therapists often provide targeted interventions for children with developmental disabilities. A common practice in these cases is one-on-one interaction between a therapist and the child together with occasional group meetings of the therapists to discuss progress and make informed decisions to modify the intervention plan. We designed a system called Abaris to support this form of collaborative decision-making for a particular intervention popular in the treatment of children with autism. Our system allows for the simultaneous use of trending data across therapy sessions and detailed session data that is automatically integrated with highly indexed video. We discuss the impact this system had on the team dynamics, the amount of collaboration, and the effect it had on the team using evidence and videos to make decisions about the care of the child.

Navigation through large multimedia collections that include videos and images still remains cumbersome. In this paper, we introduce a novel method to visualize and navigate through the collection by creating a mosaic image that visually represents the compilation. This image is generated by a labeling-based layout algorithm using various sizes of sample tile images from the collection. Each tile represents both the photographs and video files representing scenes selected by matching algorithms. This generated mosaic image provides a new way for thematic video and visually summarizes the videos. Users can generate these mosaics with some predefined themes and layouts, or base it on the results of their queries. Our approach supports automatic generation of these layouts by using meta-information such as color, time-line and existence of faces or manually generated annotated information from existing systems (e.g., the Family Video Archive).

Front projection allows large displays to be deployed relatively easily. However, it is sometimes difficult to find a location to place a projector, especially for ad-hoc installations. Additionally, front projection suffers from shadows and occlusions, making it ill-suited for interactive displays. The GVU-PROCAMS system allows programmers to deploy projectors and displays easily in arbitrary locations by enabling enhanced keystone correction via warping on 3D hardware. In addition, it handles the calibration of multiple projectors using computer vision to produce a redundantly illuminated surface. Redundant illumination offers robustness in the face of occlusions, providing a user with the experience of a rear-projected surface. This paper presents a stand-alone application (WinPVRP) and a programming system (GVU-PROCAMS) that easily allows others to create projected displays with enhanced warping and redundant illumination.

We argue that an analytic proportionality assessment balancing usefulness and burden on individual or group privacy must be conducted throughout the design process to create acceptable ubiquitous computing (ubicomp) applications and services. We introduce the principle of proportionality, which originates within the legal and data protection communities. Inspired by this principle, we develop a design method for ubicomp applications, based on our own experience, and aimed at HCI practitioners and designers. We discuss the method in relation to real-world examples, user inquiry techniques and requirements engineering models. Finally, we report a sample application of the method, involving a ubiquitous, personal memory aid tool.

In this article, we describe the design process of Reno, a location-enhanced, mobile coordination tool and person finder. The design process included three field experiments: a formative Experience Sampling Method (ESM) study, a pilot deployment and an extended user study. These studies were targeted at the significant personal security, privacy and data protection concerns caused by this application. We distil this experience into a small set of guidelines for designers of social mobile applications and show how these guidelines can be applied to a different application, called Boise. These guidelines cover issues pertaining to personal boundary definition, control, deception and denial, and group vs. individual communication. We also report on lessons learned from our evaluation experience, which might help practitioners in designing novel mobile applications, including the choice and characterization of users for testing security and privacy features of designs, the length of learning curves and their effect on evaluation and the impact of peculiar deployment circumstances on the results of these finely tuned user studies.

A number of studies have explored issues of interruption and availability in workplace environments, but few have examined how attitudes toward availability play out in home life. In this paper, we begin to explore factors in the home that might be useful for signaling availability to close friends and family. In particular, we use the Experience Sampling Method (ESM) to measure subjects' current activities and self-reported availability to interruption. Based on follow-up interviews, we develop a number of hypotheses that we test through a hierarchical linear regression analysis. Results indicate that individual differences, certain home locations, and leisure activities play an important role in determining patterns of availability. This study has implications for the development of CSCW systems with automatic sensing of activity to deal with interruption and activity recognition both inside and out of the home.

Powerful mobile devices with minimal I/O capabilities increase the likelihood that we will want to annex these devices to I/O resources we encounter in the local environment. This opportunistic annexing will require authentication. We present a sensor-based authentication mechanism for mobile devices that relies on physical possession instead of knowledge to setup the initial connection to a public terminal. Our solution provides a simple mechanism for shaking a device to authenticate with the public infrastructure, making few assumptions about the surrounding infrastructure while also maintaining a reasonable level of security.

Knowledge Acquisition (KA) is important throughout systems development for gathering expert domain knowledge that is incorporated into the requirements and design of a system. There are problems ensuring that accurate and useful knowledge is captured initially, refined as needed, and transferred to later development efforts in a usable format. We present a method, called tagging, for addressing these problems without undue burden on the KA practitioners, along with initial studies to examine the feasibility of real-time tagging and to inform the design of a tool called TAGGER. TAGGER operates by permitting KA discussions to be "tagged" as they happen with concepts and groupings relevant to software development.

This article presents results from a study of an automated capture and access system, eClass, which was designed to capture the materials presented in college lectures for later review by students. In this article, we highlight the lessons learned from our three-year study focusing on the effect of capture and access on grades, attendance, and use of the captured notes and media. We then present suggestions for building future systems discussing improvements from our system in the capture, integration, and access of college lectures.

Meetings contain a large amount of rich project information that is often not documented. Capturing audio and video of a meeting can provide a comprehensive meeting record. However, finding detailed information in that record can be challenging because there is no structural information other than time to help the user navigate. This paper surveys various ways of creating indices into meeting records and introduces the notion of creating indices based upon user interaction with domain-specific artifacts. As an example, we present a prototype system that uses general team artifacts to provide meaningful pointers into the meeting record.

We present a multi-camera vision-based eye tracking method to robustly locate and track user's eyes as they interact with an application. We propose enhancements to various vision-based eye-tracking approaches, which include (a) the use of multiple cameras to estimate head pose and increase coverage of the sensors and (b) the use of probabilistic measures incorporating Fisher's linear discriminant to robustly track the eyes under varying lighting conditions in real-time. We present experiments and quantitative results to demonstrate the robustness of our eye tracking in two application prototypes.

Many context-aware services make the assumption that the context they use is completely accurate. However, in reality, both sensed and interpreted context is often ambiguous. A challenge facing the development of realistic and deployable context-aware services, therefore, is the ability to handle ambiguous context. In this paper, we describe an architecture that supports the building of context-aware services that assume context is ambiguous and allows for mediation of ambiguity by mobile users in aware environments. We illustrate the use of our architecture and evaluate it through three example context-aware services, a word predictor system, an In/Out Board, and a reminder tool.

In the future, a largely invisible and ubiquitous computing infrastructure \nwill assist people with a variety of activities in the home and at work. \nThe applications that will be deployed in such systems will create and \nmanipulate private information and will provide access to a variety of other \nresources. Securing such applications is challenging for a number of \nreasons. Unlike traditional systems where access control has been explored, \naccess decisions may depend on the context in which requests are made. We \nshow how the well-developed notion of roles can be used to capture \nsecurity-relevant context of the environment in which access requests are \nmade. By introducing environment roles, we create a uniform access control \nframework that can be used to secure context-aware applications. We also \npresent a security architecture that supports security policies that make \nuse of environment roles to control access to resources.

Computing devices and applications are now used beyond the desktop, in diverse environments, and this trend toward ubiquitous computing is accelerating. One challenge that remains in this emerging research field is the ability to enhance the behavior of any application by informing it of the context of its use. By context, we refer to any information that characterizes a situation related to the interaction between humans, applications, and the surrounding environment. Context-aware applications promise richer and easier interaction, but the current state of research in this field is still far removed from that vision. This is due to 3 main problems: (a) the notion of context is still ill defined, (b) there is a lack of conceptual models and methods to help drive the design of context-aware applications, and (c) no tools are available to jump-start the development of context-aware applications. In this anchor article, we address these 3 problems in turn. We first define context, identify categories of contextual information, and characterize context-aware application behavior. Though the full impact of context-aware computing requires understanding very subtle and high-level notions of context, we are focusing our efforts on the pieces of context that can be inferred automatically from sensors in a physical environment. We then present a conceptual framework that separates the acquisition and representation of context from the delivery and reaction to context by a context-aware application. We have built a toolkit, the Context Toolkit, that instantiates this conceptual framework and supports the rapid development of a rich space of context-aware applications. We illustrate the usefulness of the conceptual framework by describing a number of context-aware applications that have been prototyped using the Context Toolkit. We also demonstrate how such a framework can support the investigation of important research challenges in the area of context-aware computing.

The Web is used for many purposes in education, such as the publication of course management information, centralized distribution of course material, and supporting on-line discussions between instructors and students or among the students themselves. Leveraging off the Web for educational activities both inside and outside the classroom produces a dynamic educational repository. In this paper, we present work that explicitly attempts to connect in-class activity, in the form of multimedia, Web-accessible captured lectures, with collaborative discussion spaces. Flexible and dynamic interfaces for the captured lectures and the discussion spaces are presented, as well as specialized interfaces that connect the two. We discuss our experience in a recent course taught using this integrated and dynamic educational repository and explain how our experience has lead to some solutions for visualizing the changes that occur over this rich space.

Interfaces based on recognition technologies are used extensively in both the commercial and research worlds. But recognizers are still error-prone, and this results in human performance problems, brittle dialogues, and other barriers to acceptance and utility of recognition systems. Interface techniques specialized to recognition systems can help reduce the burden of recognition errors, but building these interfaces depends on knowledge about the ambiguity inherent in recognition. We have extended a user interface toolkit in order to model and to provide structured support for ambiguity at the input event level. This makes it possible to build re-usable interface components for resolving ambiguity and dealing with recognition errors. These interfaces can help to reduce the negative effects of recognition errors. By providing these components at a toolkit level, we make it easier for application writers to provide good support for error handling. Further, with this robust support, we a...

Because of its promise of natural interaction, recognition is coming into its own as a mainstream technology for use with computers. Both commercial and research applications are beginning to use it extensively. However the errors made by recognizers can be quite costly, and this is increasingly becoming a focus for researchers. We present a survey of existing error correction techniques in the user interface. These mediation techniques most commonly fall into one of two strategies, repetition and choice. Based on the needs uncovered by this survey, we have developed OOPS, a toolkit that supports resolution of input ambiguity through mediation. This paper describes four new interaction techniques built using OOPS, and the toolkit mechanisms required to build them. These interaction techniques each address problems not directly handled by standard approaches to mediation, and can all be re-used in a variety of settings. INTRODUCTION Because of its promise of natural interaction, recog...

The proliferation of computing into the physical world promises more than the ubiquitous availability of computing infrastructure; it suggest new paradigms of interaction inspired by constant access to information and computational capabilities. For the past decade, application-driven research on abiquitous computing (ubicomp) has pushed three interaction themes: natural interfaces, context-aware applications, and automated capture and access . To chart a course for future research in ubiquitous computing, we review the accomplishments of these efforts and point to remaining research challenges. Research in ubiquitious computing implicitly requires addressing some notion of scale, whether in the number and type of devices, the physical space of distributed computing, or the number of people using a system. We posit a new area of applications research, everyday computing, focussed on scaling interaction with respect to time. Just as pushing the availiability of computing away from the traditional desktop fundamentally changes the relationship between humans and computers, providing continuous interaction moves computing from a localized tool to a constant companion. Designing for continous interaction requires addressing interruption and reumption of intreaction, representing passages of time and providing associative storage models. Inherent in all of these interaction themes are difficult issues in the social implications of ubiquitous computing and the challenges of evaluating> ubiquitious computing research. Although cumulative experience points to lessons in privacy, security, visibility, and control, there are no simple guidelines for steering research efforts. Akin to any efforts involving new technologies, evaluation strategies form a spectrum from technology feasibility efforts to long-term use studies—but a user-centric perspective is always possible and necessary

Context-enabled applications are just emerging and promise richer interaction by taking environmental context into account. However, they are difficult to build due to their distributed nature and the use of unconventional sensors. The concepts of toolkits and widget libraries in graphical user interfaces has been tremendously successtil, allowing programmers to leverage off existing building blocks to build interactive systems more easily. We introduce the concept of context widgets that mediate between the environment and the application in the same way graphical widgets mediate between the user and the application. We illustrate the concept of context widgets with the beginnings of a widget library we have developed for sensing presence, identity and activity of people and things. We assess the success of our approach with two example context-enabled applications we have built and an existing application to which we have added context-sensing capabilities.

In the last decade, we have experienced the advent of the paradigm of ubiquitous computing, with the goal of making computational services so pervasive throughout an environment that they become transparent to the human user.Research in ubiquitous computing raises many challenging issues for computer science in general, but successful research in ubiquitous computing requires the deployment of applications that can survive everyday use, and this in itself presents a great software engineering challenge.In our experience, we have found three features common across many ubiquitous computing applications -transparent interfaces that provide appropriate alternatives to the desktopbound traditional graphical user interface, the ability to modify behavior of a application based on knowledge of its context of use, and the ability to capture live experiences for later recall.Building ubiquitous computing applications with these features raises software engineering problems in toolkit design, software structuring for separation of concerns and component integration.We will clarify these problems and discuss our approaches towards their solution.

A large portion of software development effort is focused on modification and evolution of existing software systems. To feed forward-engineering and design activities, analysts must first recover and synthesize a complete and consistent set of architectural representations. Architectural Synthesis is one method to build this representation. During the Architectural Synthesis of a software system, an analyst must combine information derived from a variety of sources (which we call perspectives). This combination process requires the analyst to make decisions about which elements in the perspectives denote the same underlying parts of the software system. We present an automated technique for matching these elements based upon a mathematical technique called concept analysis. This technique constructs a spectrum of matching relations using a lattice of concepts drawn from the perspectives and descriptive information about the system's application domain. The results show the promise of using concept analysis to match elements and aid in synthesizing a large number of perspectives.

In this paper, we describe our work on developing a system to support the personalization of a captured public experience. Specifically, we are interested in providing students with the ability to personalize the capture of the lecture experiences as part of the Classroom 2000 project. We discuss the issues and challenges involved in designing a system that performs live integration of personal streams of information with multiple other streams of information made available to it through an environment designed to capture public information.

Article Investigating the capture, integration and access problem of ubiquitous computing in an educational setting Share on Authors: Gregory D. Abowd GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GA GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile , Christopher G. Atkeson GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GA GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile , Jason Brotherton GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GA GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile , Tommy Enqvist GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GA GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile , Paul Gulley GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GA GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile , Johan LeMon GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GA GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile Authors Info & Claims CHI '98: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsJanuary 1998 Pages 440–447https://doi.org/10.1145/274644.274704Online:01 January 1998Publication History 73citation1,473DownloadsMetricsTotal Citations73Total Downloads1,473Last 12 Months12Last 6 weeks2 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Applications are often designed to take advantage of the potential for integration with each other via shared information. Current approaches for integration are limited, affecting both the programmer and end-user. In this paper, we present CyberDesk, a framework for self-integrating software in which integration is driven by user context. It relieves the burden on programmers by removing the necessity to predict how software should be integrated. It also relieves the burden from users by removing the need to understand how to make different software components work together. q 1998 Elsevier Science B.V. All rights reserved.

No abstract available.

No abstract available.

Building a system at the architectural level can be thought of as decomposition into components followed by a series of exercises in matching. Components must be composed with each other via matching mechanisms; matching signatures within those mechanisms ensures that data and control flows through the system; and matching semantics among the components ensures that the system meets its behavioral requirements. The standard concepts of software architecture (components, connectors, styles) have been widely used with little more than intuitive understanding of their meaning. Mechanism matching is currently an ad hoc exercise that relies on the peculiarities of programming language facilities. This paper presents a set of well-known but informally described software architectural elements used in system composition, and taxonomizes them under a basic set of characteristic features. This classification allows us to describe legal combinations of architectural elements by performing a simple matching exercise on the relevant features of the member elements. This classification also allows us to identify architectural elements that can be substituted for each other and satisfy the same mechanism matching requirements. This leads to delayed binding of architectural mechanisms, which in turns provides increased flexibility and greater opportunities for reuse of units of computation.

Software evolution is the most costly and time-consuming software development activity, yet software engineering research is predominantly concerned with initial development. MORALE is a development method specifically designed for evolving software. It features an inquiry-based approach to eliciting change requirements, a reverse engineering technique for extracting architectural information from existing code, an approach to impact assessment that determines the extent to which the existing system's architectural components can be reused in the evolved version, a reflective approach to actually perform the evolution, and a specific technique for dealing with the difficulties that arise when evolving user interfaces. MORALE is described in the context of making a specific change to an existing system: adding user-configurable viewers to Version 2.4 of the Mosaic Web browser. Issues that arise are discussed, and the Esprit de Corps tool-suite is described

No abstract available.

Understanding the high level structure of a software system is useful for determining the impact of proposed changes. We investigate techniques for recovering the high level structure, or software architecture, for cases in which no trusted architectural description exists. We then demonstrate how a scenario based approach to architectural evaluation can be used to document the impact of proposed changes on a software system. Architectural recovery and impact analysis are demonstrated using a case study comparing two graphical debuggers, xdbx and ups.

We view college classroom teaching and learning as a multimedia authoring activity. The classroom provides a rich setting in which a number of different forms of communication co-exist, such as speech, writing and projected images. Much of the information in a lecture is poorly recorded or lost currently. Our hypothesis is that tools to aid in the capture and subsequent access of classroom information will enhance both the learning and teaching experience. To test that hypothesis, we initiated the Classroom 2000 project at Georgia Tech. The purpose of the project is to apply ubiquitous computing technology to facilitate automatic capture, integration and access of multimedia information in the educational setting of the university classroom. In this paper, we discuss various prototype tools we have created and used in a variety of courses and provide an initial evaluation of the acceptance and effectiveness of the technology. We also share some lessons learned in applying ubiquitous computing technology in a real setting.

Article Free Access Share on Rapid prototyping of mobile context-aware applications: the Cyberguide case study Authors: Sue Long College of Computing, Georgia Institute of Technology, Atlanta, GA College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile , Rob Kooper College of Computing, Georgia Institute of Technology, Atlanta, GA College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile , Gregory D. Abowd College of Computing, Georgia Institute of Technology, Atlanta, GA College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile , Christopher G. Atkeson College of Computing, Georgia Institute of Technology, Atlanta, GA College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile Authors Info & Claims MobiCom '96: Proceedings of the 2nd annual international conference on Mobile computing and networkingNovember 1996 Pages 97–107https://doi.org/10.1145/236387.236412Published:01 November 1996Publication History 166citation3,011DownloadsMetricsTotal Citations166Total Downloads3,011Last 12 Months139Last 6 weeks16 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

The software architecture of most systems is usually described informally and diagrammatically by means of boxes and lines. In order for these descriptions to be meaningful, the diagrams are understood by interpreting the boxes and lines in specific, conventionalized ways. The informal, imprecise nature of these interpretations has a number of limitations. In this article we consider these conventionalized interpretations as architectural styles and provide a formal framework for their uniform definition. In addition to providing a template for precisely defining new architectural styles, this framework allows for analysis within and between different architectural styles.

In this paper we investigate the appropriateness of formal specification languages for the description of user interface phenomena. Specifically, we are concerned with the distinction between continuously available information, which we call status, and atomic, non-persistent information, which we call events. We propose a hybrid model and notation to address status and event phenomena symmetrically. We demonstrate the effectiveness of this model for designing and understanding mixed control interaction, an especially important topic in the design of multi-user systems.

The software architecture of most systems is described informally and diagrammatically. In order for these descriptions to be meaningful at all, figures are understood by interpreting the boxes and lines in specific, conventionalized ways[5]. The imprecision of these interpretations has a number of limitations. In this paper we consider these conventionalized interpretations as architectural styles and provide a formal framework for their uniform definition. In addition to providing a template for precisely defining new architectural styles, this framework allows for the proof that the notational constraints on a style are sufficient to guarantee the meanings of all described systems and provides a unified semantic base through which different stylistic interpretations can be compared.

The problems associated with the provision of an undo support facility in the context of a synchronous shared or group editor are investigated. Previous work on the development of formal models of ‘undo’ has been restricted to single user systems and has focused on the functionality of undo, as opposed to discussing the support that users require from any error recovery facility. Motivated by new issues that arise in the context of computer supported co-operative work, the authors aim to integrate formal modelling of undo with an analysis of how users understand undo facilities. Together, these combined perspectives of the system and user lead to concrete design advice for implementing an undo facility. The special issues that arise in the context of shared undo also shed light on the emphasis that should be placed on single user undo. In particular, the authors regard undo not as a system command to be implemented, but as a user intention to be supported by the system.