2.5D effects, such as occlusion and perspective foreshortening, enhance visual dynamics and realism by introducing 3D depth cues into 2D designs. However, creating these effects remains challenging, as designers must manually infer and author depth relationships—such as relative ordering, occlusion boundaries, and perspective scaling—within 2D representations. We introduce DepthScape, a human–AI collaborative system that facilitates 2.5D effect creation by placing design elements directly into 3D reconstructions. Using monocular depth reconstruction, DepthScape transforms images into 3D scenes, enabling depth-based blending that produces realistic occlusion and perspective foreshortening. To simplify 3D placement, DepthScape leverages a vision-language model to analyze source images and extract key visual components as parametric anchors, which support direct manipulation editing. The system design was iteratively refined through a formative user study with an early prototype. We evaluate DepthScape through a technical study on 100 professional stock images to assess robustness, alongside an expert evaluation confirming design quality, usefulness, and broad application potential, further illustrated through five example scenarios.

Urban cycling benefits personal wellbeing, public health, and global sustainability. While current tools such as Google and Apple Maps provide bike route recommendations, they do not account for a person’s dynamic context (e.g., commuting, recreation). We introduce BikeButler, a personalized, context-sensitive bicycle route generation tool that enables users to generate, compare, virtually preview, and iteratively customize bike routes via custom profiles that encode seven bikeability features, including bike lane existence, slope, vegetation, and surface quality—fusing data from OpenStreetMap, open government data, and a custom VLM-based analysis of Street View images. To design BikeButler, we employed a human-centered, iterative approach starting with formative interviews and culminating in a user study (N=16). Our findings demonstrate that bike routing preferences change as a function of context, that BikeButler enables users to quickly create and iterate context-sensitive routes, and that generated routes differ significantly from Google Maps bike routing, reinforcing the importance of personalization.

Geovisualizations are powerful tools for communicating spatial information, but are inaccessible to screen-reader users. To address this limitation, we present GeoVisA11y, an LLM-based question-answering system that makes geovisualizations accessible through natural language interaction. The system supports map reading, analysis, interpretation and navigation by handling analytical, geospatial, visual, and contextual queries. Through user studies with six screen-reader users and six sighted participants, we demonstrate that GeoVisA11y effectively bridges accessibility gaps while revealing distinct interaction patterns between user groups. We contribute: (1) an open-source, accessible geovisualization system, (2) empirical findings on query and navigation differences, and (3) a dataset of geospatial queries to inform future research on accessible data visualization.

Accessible parking is critical for people with disabilities (PwDs), allowing equitable access to destinations, independent mobility, and community participation. Despite mandates, there has been no large-scale investigation of the quality or allocation of disability parking in the US nor significant research on PwD perspectives and uses of disability parking. In this paper, we first present a semi-structured interview study with 11 PwDs to advance understanding of disability parking uses, concerns, and relevant technology tools. We find that PwDs often adapt to disability parking challenges according to their personal mobility needs and value reliable, real-time accessibility information. Informed by these findings, we then introduce a new deep learning pipeline, called AccessParkCV, and parking dataset for automatically detecting disability parking and inferring quality characteristics (e.g., width) from orthorectified aerial imagery. We achieve a micro-F1=0.89 and demonstrate how our pipeline can support new urban analytics and end-user tools. Together, we contribute new qualitative understandings of disability parking, a novel detection pipeline and open dataset, and design guidelines for future tools.

Figure 1: We introduce GeoQA 3 , a novel accessible AI-based question-answering system for geovisualizations designed for screen-reader users.(A) Through a custom query pipeline, we combine geo-statistical analysis with an LLM to balance accuracy and performance.(B) Users can navigate the map through natural language commands or keyboard controls and (C) zoom in to view county-level data.The AI Chat system is context-aware, taking into account user interactions.See video for demonstration.

Record soundsTrain personalized model Iteratively test Figure 1: Overview of the SPECTRA pipeline.In an interactive machine learning training workfow, users collect audio data samples (left), flter their data into a training dataset (center), and assess their model's performance in a live environment (right).The design includes key elements to support the needs of DHH users during this process, including spectrogram and waveform audio visualizations of audio, data annotating to save useful contextual information, and an interactive clustering visualization of their dataset.

Despite diverse mobility needs worldwide, existing mapping tools fail to address the varied experiences of different mobility device users. This paper presents a large-scale online survey exploring how five mobility groups -- users of canes, walkers, mobility scooters, manual wheelchairs, and motorized wheelchairs -- perceive sidewalk barriers. Using 52 sidewalk barrier images, respondents evaluated their confidence in navigating each scenario. Our findings (N=190) reveal variations in barrier perceptions across groups, while also identifying shared concerns. To further demonstrate the value of this data, we showcase its use in two custom prototypes: a visual analytics tool and a personalized routing tool. Our survey findings and open dataset advance work in accessibility-focused maps, routing algorithms, and urban planning.

We introduce ArtInsight, a novel AI-powered system to facilitate deeper engagement with child-created artwork in mixed visual-ability families. ArtInsight leverages large language models (LLMs) to craft a respectful and thorough initial description of a child's artwork, and provides: creative AI-generated descriptions for a vivid overview, audio recording to capture the child's own description of their artwork, and a set of AI-generated questions to facilitate discussion between blind or low-vision (BLV) family members and their children. Alongside ArtInsight, we also contribute a new rubric to score AI-generated descriptions of child-created artwork and an assessment of state-of-the-art LLMs. We evaluated ArtInsight with five groups of BLV family members and their children, and as a case study with one BLV child therapist. Our findings highlight a preference for ArtInsight's longer, artistically-tailored descriptions over those generated by existing BLV AI tools. Participants highlighted the creative description and audio recording components as most beneficial, with the former helping ``bring a picture to life'' and the latter centering the child's narrative to generate context-aware AI responses. Our findings reveal different ways that AI can be used to support art engagement, including before, during, and after interaction with the child artist, as well as expectations that BLV adults and their sighted children have about AI-powered tools.

While augmented reality (AR) enables new ways to play, tell stories, and explore ideas rooted in the physical world, authoring personalized AR content remains difficult for non-experts, often requiring professional tools and time. Prior systems have explored AI-driven XR design but typically rely on manually defined VR environments and fixed asset libraries, limiting creative flexibility and real-world relevance. We introduce ImaginateAR, the first mobile tool for AI-assisted AR authoring to combine offline scene understanding, fast 3D asset generation, and LLMs -- enabling users to create outdoor scenes through natural language interaction. For example, saying "a dragon enjoying a campfire" (P7) prompts the system to generate and arrange relevant assets, which can then be refined manually. Our technical evaluation shows that our custom pipelines produce more accurate outdoor scene graphs and generate 3D meshes faster than prior methods. A three-part user study (N=20) revealed preferred roles for AI, how users create in freeform use, and design implications for future AR authoring tools. ImaginateAR takes a step toward empowering anyone to create AR experiences anywhere -- simply by speaking their imagination.

With use, Accessibility Scout becomes an increasingly capable "accessibility scout", tailoring accessibility scans to an individual's mobility level, preferences, and specific environmental interests through collaborative Human-AI assessments.We present findings from three studies: a formative study with six participants to inform the design of Accessibility Scout, a technical evaluation of 500 images of built environments, and a user study with 10 participants of varying mobility.Results from our technical evaluation and user study show that Accessibility Scout can generate personalized accessibility scans that extend beyond traditional ADA considerations.Finally, we conclude with a discussion on the implications of our work and future steps for building more scalable and personalized accessibility assessments of the physical world.

We present two studies exploring how blind or low-vision (BLV) family members engage with their sighted children’s artwork, strategies to support understanding and interpretation, and the potential role of technology, such as AI, therein. Our first study involved 14 BLV individuals, and the second included five groups of BLV individuals with their children. Through semi-structured interviews with AI descriptions of children’s artwork and multi-sensory design probes, we found that BLV family members value artwork engagement as a bonding opportunity, preferring the child’s storytelling and interpretation over other nonvisual representations. Additionally, despite some inaccuracies, BLV family members felt that AI-generated descriptions could facilitate dialogue with their children and aid self-guided art discovery. We close with specific design considerations for supporting artwork engagement in mixed visual-ability families, including enabling artwork access through various methods, supporting children’s corrections of AI output, and distinctions in context vs. content and interpretation vs. description of children’s artwork.

We have entered a new era of computing—one where AI permeates every aspect of society from education to healthcare. In this workshop, we examine the emerging role of AI in the design of equitable and accessible cities, transportation systems, and interactive tools for mapping and navigation. We will solicit short papers around key Urban AI + disability themes, including autonomous vehicles, intelligent wheelchairs, assistive human-robotic interaction, assessing and navigating pedestrian pathways, indoor accessibility, and overarching challenges related to ethics, bias, and data privacy and security. We invite both traditional HCI and accessibility researchers as well as scholars and practitioners from other disciplines relevant to this workshop, including disability studies, gerontology, social work, community psychology, and law. Our overarching goal is to identify open challenges, share current work across disciplines, and spur new collaborations related to AI and urban accessibility.

We examine the feasibility of using deep learning to infer 33 classes of sidewalk accessibility conditions in pre-cropped streetscape images, including bumpy, brick/cobblestone, cracks, height difference (uplifts), narrow, uneven/slanted, pole, and sign. We present two experiments: first, a comparison between two state-of-the-art computer vision models, Meta’s DINOv2 and OpenAI’s CLIP-ViT, on a cleaned dataset of ∼ 24k images; second, an examination of a larger but noisier crowdsourced dataset (∼ 87k images) on the best performing model from Experiment 1. Though preliminary, Experiment 1 shows that certain sidewalk conditions can be identified with high precision and recall, such as missing tactile warnings on curb ramps and grass grown on sidewalks, while Experiment 2 demonstrates that larger but noisier training data can have a detrimental effect on performance. We contribute an open dataset and classification benchmarks to advance this important area.

Mapping, assessing, and creating personalized routes of indoor spaces for people with disabilities remains a grand challenge in accessibility research. Drawing on recent work in robotics as well as emergent work in smartphone-based mapping, we introduce RAIS (Robotic Accessibility Indoor Scanner), a robotic-based indoor mapping and accessibility assessment system. As a rapid prototype, RAIS is constructed with off-the-shelf components including a vacuum robot, smartphone, and phone gimbal along with a modified version of our previous LiDAR-based accessibility scannar RASSAR. In a preliminary evaluation of three indoor spaces, we demonstrate RAIS’s ability to autonomously scan spaces, produce detailed 3D reconstructions, and find and highlight accessibility issues.

Tactile surface indicators (TSIs) provide ground-based tactile cues to help pedestrians who are blind or low-vision safely and independently navigate different environments. For example, TSIs can serve as warnings for hazards (e.g., edge of a subway platforms) and directional guides (e.g., a route through a mall). In this exploratory work, we examine how digital fabrication technologies such as 3D printing, CNC milling, vacuum forming, and heat transfer melting can enable the production of custom TSIs. To compare different fabrication approaches, we designed and evaluated a series of prototypes with varied surface materials and design features (e.g., bump height). We then solicited feedback on our ideas and fabricated TSIS via two initial qualitative evaluations: one with a blind cane user and another with an Orientation and Mobility (O&M) specialist. Our initial findings demonstrate that digital fabrication processes—primarily 3D printing and CNC milling—can produce salient and useful TSIs, and indicate interest in our approach and how highly customized, rapidly fabricable TSIs could support navigation in reconfigurable indoor spaces.

Voice assistants (VAs) like Siri and Alexa are transforming human-computer interaction; however, they lack awareness of users’ spatiotemporal context, resulting in limited performance and unnatural dialogue. We introduce GazePointAR, a fully-functional context-aware VA for wearable augmented reality that leverages eye gaze, pointing gestures, and conversation history to disambiguate speech queries. With GazePointAR, users can ask “what’s over there?” or “how do I solve this math problem?” simply by looking and/or pointing. We evaluated GazePointAR in a three-part lab study (N=12): (1) comparing GazePointAR to two commercial systems, (2) examining GazePointAR’s pronoun disambiguation across three tasks; (3) and an open-ended phase where participants could suggest and try their own context-sensitive queries. Participants appreciated the naturalness and human-like nature of pronoun-driven queries, although sometimes pronoun use was counter-intuitive. We then iterated on GazePointAR and conducted a first-person diary study examining how GazePointAR performs in-the-wild. We conclude by enumerating limitations and design considerations for future context-aware VAs.

Despite decades of effort, pedestrian infrastructure in cities continues to be unsafe or inaccessible to people with disabilities. In this paper, we examine the potential of community-driven digital civics to assess sidewalk accessibility through a deployment study of an open-source crowdsourcing tool called Project Sidewalk. We explore Project Sidewalk’s potential as a platform for civic learning and service. Specifically, we assess its effectiveness as a tool for community members to learn about human mobility, urban planning, and accessibility advocacy. Our findings demonstrate that community-driven digital civics can support accessibility advocacy and education, raise community awareness, and drive pro-social behavioral change. We also outline key considerations for deploying digital civic tools in future community-led accessibility initiatives.

Crowdsourcing platforms have transformed distributed problem-solving, yet quality control remains a persistent challenge. Traditional quality control measures, such as prescreening workers and refining instructions, often focus solely on optimizing economic output. This paper explores just-in-time AI interventions to enhance both labeling quality and domain-specific knowledge among crowdworkers. We introduce LabelAId, an advanced inference model combining Programmatic Weak Supervision (PWS) with FT-Transformers to infer label correctness based on user behavior and domain knowledge. Our technical evaluation shows that our LabelAId pipeline consistently outperforms state-of-the-art ML baselines, improving mistake inference accuracy by 36.7% with 50 downstream samples. We then implemented LabelAId into Project Sidewalk, an open-source crowdsourcing platform for urban accessibility. A between-subjects study with 34 participants demonstrates that LabelAId significantly enhances label precision without compromising efficiency while also increasing labeler confidence. We discuss LabelAId’s success factors, limitations, and its generalizability to other crowdsourced science domains.

Video games often pose accessibility barriers to gamers with disabilities, yet there is no standard method for identifying which games have barriers, what those barriers are, and whether and how they can be overcome. We propose and explore three phases of the “game adoption process”: Discovery, Evaluation, and Adaptation. To advance understanding of how gamers with disabilities experience this process, the resources and strategies they use, and the challenges experienced, we conducted an interview study with thirteen gamers with disabilities with differing backgrounds. We then engage with existing theories of consequence-based accessibility, of difficulty, and of identity-based gaming to better understand how these processes manifest “access difficulty” and to characterize the experience of “disabled gaming.” Finally, we present design recommendations for game developers and distributors to better support gamers with disabilities in the game adoption process by engaging with community-made resources, supporting socially-created access, and creating customizable experiences with opportunities for unconventional play.

As user-generated video dominates media landscapes, it poses an accessibility challenge. While disability advocacy groups globally have secured hard-won accessibility regulations for broadcast media, no such regulation of user-generated content exists. Yet, one major player in this shift, TikTok, has a culture of user-generated, creative captioning. We sought to understand how TikTok videos are captioned and the impact current practices have on those who need captions to access audio content. Therefore, we conducted a content analysis of 300 open-captioned TikToks and contextualized these findings by interviewing nine caption users. We found that the current state of TikTok captioning does facilitate access to the platform but that a user-generated, social video-specific standard for captioning could improve caption quality and expand access. We contribute an empirical account of the state of TikTok captioning and outline steps toward a standard for user-generated captioning.

The safety and accessibility of our homes is critical to quality of life and evolves as we age, become ill, host guests, or experience life events such as having children. Researchers and health professionals have created assessment instruments such as checklists that enable homeowners and trained experts to identify and mitigate safety and access issues. With advances in computer vision, augmented reality (AR), and mobile sensors, new approaches are now possible. We introduce RASSAR, a mobile AR application for semi-automatically identifying, localizing, and visualizing indoor accessibility and safety issues such as an inaccessible table height or unsafe loose rugs using LiDAR and real-time computer vision. We present findings from three studies: a formative study with 18 participants across five stakeholder groups to inform the design of RASSAR, a technical performance evaluation across ten homes demonstrating state-of-the-art performance, and a user study with six stakeholders. We close with a discussion of future AI-based indoor accessibility assessment tools, RASSAR’s extensibility, and key application scenarios.

Cooking is a central activity of daily living, supporting independence as well as mental and physical health. However, prior work has highlighted key barriers for people with low vision (LV) to cook, particularly around safely interacting with tools, such as sharp knives or hot pans. Drawing on recent advancements in computer vision (CV), we present CookAR, a head-mounted AR system with real-time object affordance augmentations to support safe and efficient interactions with kitchen tools. To design and implement CookAR, we collected and annotated the first egocentric dataset of kitchen tool affordances, fine-tuned an affordance segmentation model, and developed an AR system with a stereo camera to generate visual augmentations. To validate CookAR, we conducted a technical evaluation of our fine-tuned model as well as a qualitative lab study with 10 LV participants for suitable augmentation design. Our technical evaluation demonstrates that our model outperforms the baseline on our tool affordance dataset, while our user study indicates a preference for affordance augmentations over the traditional whole object augmentations.

Sound plays a crucial role in enhancing user experience and immersiveness in Augmented Reality (AR). However, current platforms lack support for AR sound authoring due to limited interaction types, challenges in collecting and specifying context information, and difficulty in acquiring matching sound assets. We present SonifyAR, an LLM-based AR sound authoring system that generates context-aware sound effects for AR experiences. SonifyAR expands the current design space of AR sound and implements a Programming by Demonstration (PbD) pipeline to automatically collect contextual information of AR events, including virtual-content-semantics and real-world context. This context information is then processed by a large language model to acquire sound effects with Recommendation, Retrieval, Generation, and Transfer methods. To evaluate the usability and performance of our system, we conducted a user study with eight participants and created five example applications, including an AR-based science experiment, and an assistive application for low-vision AR users.

3D printing is transforming how we customize and create physical objects in engineering, accessibility, and art. However, this technology is still primarily limited to confined working areas and dedicated print beds, thereby detaching design and fabrication from real-world environments and making measuring and scaling objects tedious and labor-intensive. In this paper, we present MobiPrint, a prototype mobile fabrication system that combines elements from robotics, architecture, and Human-Computer Interaction (HCI) to enable environment-scale design and fabrication in ad-hoc indoor environments. MobiPrint provides a multi-stage fabrication pipeline: first, the robotic 3D printer automatically scans and maps an indoor space; second, a custom design tool converts the map into an interactive CAD canvas for editing and placing models in the physical world; finally, the MobiPrint robot prints the object directly on the ground at the defined location. Through a “proof-by-demonstration” validation, we highlight our system’s potential across different applications, including accessibility, home furnishing, floor signage, and art. We also conduct a technical evaluation to assess MobiPrint’s localization accuracy, ground surface adhesion, payload capacity, and mapping speed. We close with a discussion of open challenges and opportunities for the future of contextualized mobile fabrication.

Geovisualizations are powerful tools for exploratory spatial analysis, enabling sighted users to discern patterns, trends, and relationships within geographic data. However, these visual tools have remained largely inaccessible to screen-reader users. We introduce AltGeoViz, a new interactive geovisualization approach that dynamically generates alt-text descriptions based on the user’s current map view, providing voiceover summaries of spatial patterns and descriptive statistics. In a remote user study with five screen-reader users, we found that participants were able to interact with spatial data in previously infeasible ways, demonstrated a clear understanding of data summaries and their location context, and could synthesize spatial understandings of their explorations. Moreover, we identified key areas for improvement, such as the addition of spatial navigation controls and comparative analysis features.

Public transportation provides vital connectivity to people with disabilities, facilitating access to work, education, and health services. While modern navigation applications provide a suite of information about transit options—including real-time updates about bus or train arrivals—they lack data about the accessibility of the transit stops themselves. Bus stop features such as seatings, shelters, and landing areas are critical, but few cities provide this information. In this demo paper, we introduce BusStopCV, a Human+AI web prototype for scalably collecting data on bus stop features using real-time computer vision and human labeling. We describe BusStopCV’s design, custom training with the YOLOv8 model, and an evaluation of 100 randomly selected bus stops in Seattle, WA. Our findings demonstrate the potential of BusStopCV and highlight opportunities for future work.

In this demo paper, we introduce RASSAR, a mobile AR application for semi-automatically identifying, localizing, and visualizing indoor accessibility and safety issues using LiDAR and real-time computer vision. Our prototype supports four classes of detection problems: inaccessible object dimensions (e.g., table height), inaccessible object positions (e.g., a light switch out of reach), the presence of unsafe items (e.g., scissors), and the lack of proper assistive devices (e.g., grab bars). RASSAR’s design was informed by a formative interview study with 18 participants from five key stakeholder groups, including wheelchair users, blind and low vision participants, families with young children, and caregivers. Our envisioned use cases include vacation rental hosts, new caregivers, or people with disabilities themselves documenting issues in their homes or rental spaces and planning renovations. We present key findings from our formative interviews, the design of RASSAR, and results from an initial performance evaluation.

Technologies for blind and low-vision (BLV) people have long been a focus of Human-Computer Interaction (HCI) and accessibility (ASSETS) research. To map and assess this cross-disciplinary field, prior literature reviews have focused on specific BLV research areas (e.g., navigation assistance) or study methodologies (e.g., qualitative methods). In this paper, we provide a more holistic examination, combining both quantitative bibliometric analyses with qualitative assessments. Using keyword queries of terms focused on the human (e.g., people) and their visual status (e.g., blind, low-vision), we first derived a dataset of 880 papers published between 2010-2022 from ACM and IEEE conferences and journals. We then apply a programmatic analysis of this dataset followed by a qualitative analysis of the 100 most-cited papers. Our findings highlight four major research areas: Accessibility at Home & on the Go, Non-Visual Interaction, Orientation & Mobility, and Education. We also capture the diversity of denominations used to refer to the BLV community and their co-occurrences, as well as computer systems targeting both blind and low-vision users with a focus on visual substitution. We close by suggesting areas for future work and hope to stimulate discussions in our field.

With improvements in automated speech recognition and increased use of videoconferencing, real-time captioning has changed significantly. This shift toward broadly available but less accurate captioning invites exploration of the role hearing conversation partners play in shaping the accessibility of a conversation to d/Deaf and hard of hearing (DHH) captioning users. While recent work has explored DHH individuals’ videoconferencing experiences with captioning, we focus on established groups’ current practices and priorities for future tools to support more accessible online conversations. Our study consists of three codesign sessions, conducted with four groups (17 participants total, 10 DHH, 7 hearing). We found that established groups crafted social accessibility norms that met their relational contexts. We also identify promising directions for future captioning design, including the need to standardize speaker identification and customization, opportunities to provide behavioral feedback during a conversation, and ways that videoconferencing platforms could enable groups to set and share norms.

Increasingly, crowds plus machine learning techniques are being used to semi-automatically analyze the accessibility of built environments; however, open questions remain about how to effectively combine the two. We present two experiments examining the effect of crowdsourced data in automatically classifying sidewalk accessibility features in streetscape images. In Experiment 1, we investigate the effect of validated data—which has been voted correct by the crowd but is more expensive to collect—compared with a larger but noisier aggregate dataset. In Experiment 2, we examine whether crowdsourced labeled data gathered in one city can be used as effective training data for another. Together, these experiments contribute to the growing literature in Crowd+AI approaches for semi-automatic sidewalk assessment and help identify pertinent challenges.

Inaccessible urban infrastructure creates and reinforces systemic exclusion of people with disabilities and impacts public health, physical activity, and quality of life for all. To improve the design of our cities and to enable more equitable policies and location-centric technology designs, we need new data collection techniques, data standards, and accessibility-infused analytic tools and interactive maps focused on the quality, safety, and accessibility of pathways, transit ecosystems, and buildings. In this workshop, we bring together leading experts in human mobility, urban design, disability, and accessible computing to discuss pressing urban access challenges across the world and brainstorm solutions. We invite contributions from practitioners, transit officials, disability advocates, and researchers.

Recent advances have enabled automatic sound recognition systems for deaf and hard of hearing (DHH) users on mobile devices. However, these tools use pre-trained, generic sound recognition models, which do not meet the diverse needs of DHH users. We introduce ProtoSound, an interactive system for customizing sound recognition models by recording a few examples, thereby enabling personalized and fine-grained categories. ProtoSound is motivated by prior work examining sound awareness needs of DHH people and by a survey we conducted with 472 DHH participants. To evaluate ProtoSound, we characterized performance on two real-world sound datasets, showing significant improvement over state-of-the-art (e.g., +9.7% accuracy on the first dataset). We then deployed ProtoSound's end-user training and real-time recognition through a mobile application and recruited 19 hearing participants who listened to the real-world sounds and rated the accuracy across 56 locations (e.g., homes, restaurants, parks). Results show that ProtoSound personalized the model on-device in real-time and accurately learned sounds across diverse acoustic contexts. We close by discussing open challenges in personalizable sound recognition, including the need for better recording interfaces and algorithmic improvements.

Urban accessibility assessments are challenging: they involve varied stakeholders across decision-making contexts while serving a diverse population of people with disabilities. To better support urban accessibility assessment using data visualizations, we conducted a three-part interview study with 25 participants across five stakeholder groups using map visualization probes. We present a multi-stakeholder analysis of visualization needs and sensemaking processes to explore how interactive visualizations can support stakeholder decision making. In particular, we elaborate how stakeholders’ varying levels of familiarity with accessibility, geospatial analysis, and specific geographic locations influences their sensemaking needs. We then contribute 10 design considerations for geovisual analytic tools for urban accessibility communication, planning, policymaking, and advocacy.

We present Kinergy—an interactive design tool for creating self-propelled motion by harnessing the energy stored in 3D printable springs. To produce controllable output motions, we introduce 3D printable kinetic units, a set of parameterizable designs that encapsulate 3D printable springs, compliant locks, and transmission mechanisms for three non-periodic motions—instant translation, instant rotation, continuous translation—and four periodic motions—continuous rotation, reciprocation, oscillation, intermittent rotation. Kinergy allows the user to create motion-enabled 3D models by embedding kinetic units, customize output motion characteristics by parameterizing embedded springs and kinematic elements, control energy by operating the specialized lock, and preview the resulting motion in an interactive environment. We demonstrate the potential of our techniques via example applications from spring-loaded cars to kinetic sculptures and close with a discussion of key challenges such as geometric constraints.

As HCI pedagogy research grows, so too does an emerging set of evidence-based teaching and curricular recommendations. Yet, few studies have implemented and examined these recommendations in the classroom. In this paper, we present a Research Through Design investigation of a studio-based HCI course, which was revised based on HCI education literature. Drawing on reflection surveys, video recordings of student-led user sessions, final project artifacts, and student interviews, we explore how students responded to key educational changes, the strategies that supported and hindered their reflective practices, and how reflection afforded new student insights. Our findings highlight the utility of video-based reflection exercises to support student learning in designing and running user sessions, the importance of multi-faceted reflection prompts, and how students noticed moments of inclusion and exclusion by attending to users’ non-verbal cues. Additionally, we empirically demonstrate the importance of implementing and studying HCI education research recommendations in the classroom.

What do sidewalk accessibility problems look like? How might these problems differ across cities? In this poster paper, we introduce Sidewalk Gallery, an interactive, filterable gallery of over 500,000 crowdsourced sidewalk accessibility images across seven cities in two countries (US and Mexico). Gallery allows users to explore and interactively filter sidewalk images based on five primary accessibility problem types, 35 tag categories, and a 5-point severity scale. When browsing images, users can also provide feedback about data correctness. We envision Gallery as a tool for teaching in urban design and accessibility and as a visualization aid for disability advocacy.

Android educational games are powerful learning tools but small, moving targets and game implementations pose accessibility challenges to people with upper-body motor impairments. In this poster, we present findings from a qualitative accessibility evaluation of 30 popular Android educational games, identify and reflect on accessibility barriers, and provide preliminary design recommendations.

How do sidewalks change over time? Are there geographic or socioeconomic patterns to this change? These questions are important but difficult to address with current GIS tools and techniques. In this demo paper, we introduce three preliminary crowd+AI (Artificial Intelligence) prototypes to track changes in street intersection accessibility over time—specifically, curb ramps—and report on results from a pilot usability study.

Accessibility research has grown substantially in the past few decades, yet there has been no literature review of the field. To understand current and historical trends, we created and analyzed a dataset of accessibility papers appearing at CHI and ASSETS since ASSETS' founding in 1994. We qualitatively coded areas of focus and methodological decisions for the past 10 years (2010-2019, N=506 papers), and analyzed paper counts and keywords over the full 26 years (N=836 papers). Our findings highlight areas that have received disproportionate attention and those that are underserved--for example, over 43% of papers in the past 10 years are on accessibility for blind and low vision people. We also capture common study characteristics, such as the roles of disabled and nondisabled participants as well as sample sizes (e.g., a median of 13 for participant groups with disabilities and older adults). We close by critically reflecting on gaps in the literature and offering guidance for future work in the field.

Visual semantics provide spatial information like size, shape, and position, which are necessary to understand and efficiently use interfaces and documents. Yet little is known about whether blind and low-vision (BLV) technology users want to interact with visual affordances, and, if so, for which task scenarios. In this work, through semi-structured and task-based interviews, we explore preferences, interest levels, and use of visual semantics among BLV technology users across two device platforms (smartphones and laptops), and information seeking and interactions common in apps and web browsing. Findings show that participants could benefit from access to visual semantics for collaboration, navigation, and design. To learn this information, our participants used trial and error, sighted assistance, and features in existing screen reading technology like touch exploration. Finally, we found that missing information and inconsistent screen reader representations of user interfaces hinder learning. We discuss potential applications and future work to equip BLV users with necessary information to engage with visual semantics.

Real-time captioning is a critical accessibility tool for many d/Deaf and hard of hearing (DHH) people. While the vast majority of captioning work has focused on formal settings and technical innovations, in contrast, we investigate captioning for informal, interactive small-group conversations, which have a high degree of spontaneity and foster dynamic social interactions. This paper reports on semi-structured interviews and design probe activities we conducted with 15 DHH participants to understand their use of existing real-time captioning services and future design preferences for both in-person and remote small-group communication. We found that our participants' experiences of captioned small-group conversations are shaped by social, environmental, and technical considerations (e.g., interlocutors' pre-established relationships, the type of captioning displays available, and how far captions lag behind speech). When considering future captioning tools, participants were interested in greater feedback on non-speech elements of conversation (e.g., speaker identity, speech rate, volume) both for their personal use and to guide hearing interlocutors toward more accessible communication. We contribute a qualitative account of DHH people's real-time captioning experiences during small-group conversation and future design considerations to better support the groups being captioned, both in person and online.?

Head-mounted displays can provide private and glanceable speech and sound feedback to deaf and hard of hearing people, yet prior systems have largely focused on speech transcription. We introduce HoloSound, a HoloLens-based augmented reality (AR) prototype that uses deep learning to classify and visualize sound identity and location in addition to providing speech transcription. This poster paper presents a working proof-of-concept prototype, and discusses future opportunities for advancing AR-based sound awareness.

Smartwatches have the potential to provide glanceable, always-available sound feedback to people who are deaf or hard of hearing. In this paper, we present a performance evaluation of four low-resource deep learning sound classification models: MobileNet, Inception, ResNet-lite, and VGG-lite across four device architectures: watch-only, watch+phone, watch+phone+cloud, and watch+cloud. While direct comparison with prior work is challenging, our results show that the best model, VGG-lite, performed similar to the state of the art for non-portable devices with an average accuracy of 81.2% (SD=5.8%) across 20 sound classes and 97.6% (SD=1.7%) across the three highest-priority sounds. For device architectures, we found that the watch+phone architecture provided the best balance between CPU, memory, network usage, and classification latency. Based on these experimental results, we built and conducted a qualitative lab evaluation of a smartwatch-based sound awareness app, called SoundWatch (Figure 1), with eight DHH participants. Qualitative findings show support for our sound awareness app but also uncover issues with misclassifications, latency, and privacy concerns. We close by offering design considerations for future wearable sound awareness technology.

We present a qualitative study with 16 deaf and hard of hearing (DHH) participants examining reactions to smartwatch-based visual + haptic sound feedback designs. In Part 1, we conducted a Wizard-of-Oz (WoZ) evaluation of three smartwatch feedback techniques (visual alone, visual + simple vibration, and visual + tacton) and investigated vibrational patterns (tactons) to portray sound loudness, direction, and identity. In Part 2, we visited three public or semi-public locations where we demonstrated sound feedback on the smartwatch in situ to examine contextual influences and explore sound filtering options. Our findings characterize uses for vibration in multimodal sound awareness, both for push notification and for immediately actionable sound information displayed through vibrational patterns (tactons). In situ experiences caused participants to request sound filtering - particularly to limit haptic feedback - as a method for managing soundscape complexity. Additional concerns arose related to learnability, possibility of distraction, and system trust. Our findings have implications for future portable sound awareness systems.

We introduce HomeSound, an in-home sound awareness system for Deaf and hard of hearing (DHH) users. Similar to the Echo Show or Nest Hub, HomeSound consists of a microphone and display, and uses multiple devices installed in each home. We iteratively developed two prototypes, both of which sense and visualize sound information in real-time. Prototype 1 provided a floorplan view of sound occurrences with waveform histories depicting loudness and pitch. A three-week deployment in four DHH homes showed an increase in participants' home- and self-awareness but also uncovered challenges due to lack of line of sight and sound classification. For Prototype 2, we added automatic sound classification and smartwatch support for wearable alerts. A second field deployment in four homes showed further increases in awareness but misclassifications and constant watch vibrations were not well received. We discuss findings related to awareness, privacy, and display placement and implications for future home sound awareness technology.

We introduce ARMath, a mobile Augmented Reality (AR) system that allows ch ildren to discover mathematical concepts in familiar, ord inary objects and engage with math problems in meaningful contexts. Leveraging advanced computer vision, ARMath recognizes everyday objects, visualizes their mathematical attributes, and turns them into tangible or virtual manipulatives. Using the manipulatives, children can solve problems that situate math operations or concepts in specific everyday contexts. Informed by four participatory design sessions with teachers and children, we developed five ARMath modules to support basic arithmetic and 2D geometry. We also conducted an exploratory evaluation of ARMath with 27 children (ages 5-8) at a local children's museum. Our findings demonstrate how ARMath engages children in math learning, how failures in AI can be used as learning opportunities, and challenges that children face when using ARMath.

Users are fundamental to HCI. However, little is known about how HCI education introduces students to working with users, particularly those different from themselves. To better understand design students' engagement, reactions, and reflections with users, we investigate a case study of a graduate-level 10-week prototyping studio course that partnered with a children's co-design team. HCI students participated in two co-design sessions with children to design a STEM learning experience for youth. We conducted participant observations, interviews with 14 students, and analyzed final artifacts. Our findings demonstrate the communication challenges and strategies students experienced, how students observed issues of power dynamics, and students' perceived value in engaging with users. We contribute empirical evidence of how HCI students directly interact with target users, principles for reflective HCI pedagogy, and highlight the need for more intentional investigation into HCI educational practice.

Traditionally, urban accessibility is defined as the ease of reaching destinations. Studies on urban accessibility for pedestrians with mobility disabilities (e.g., wheelchair users) have primarily focused on understanding the challenges that the built environment imposes and how they overcome them. In this paper, we move beyond physical barriers and focus on socio-political challenges in the civic ecosystem that impedes accessible infrastructure development. Using a multi-stakeholder approach, we interviewed five primary stakeholder groups (N=25): (1) people with mobility disabilities, (2) caregivers, (3) accessibility advocates, (4) department officials, and (5) policymakers. We discussed their current accessibility assessment and decision-making practices. We identified the key needs and desires of each group, how they differed, and how they interacted with each other in the civic ecosystem to bring about change. We found that people, politics, and money were intrinsically tied to underfunded accessibility improvement projects "without continued support from the public and the political leadership, existing funding may also disappear. Using the insights from these interviews, we explore how may technology enhance our stakeholders" decision-making processes and facilitate accessible infrastructure development.

We introduce PrototypAR, an Augmented Reality (AR) system that allows children to rapidly build complex systems using paper crafts and to test their designs in a digital environment. PrototypAR combines lo-fidelity prototyping to facilitate iterative design, real-time AR feedback to scaffold learning, and a virtual simulation environment to support personalized experiments. Informed by three participatory design sessions, we developed three PrototypAR applications: build-a-bike, build-a-camera, and build-an-aquarium---each highlights different aspects of our system. To evaluate PrototypAR, we conducted four single-session qualitative evaluations with 21 children working in teams. Our findings show how children build and explore complex systems models, how they use AR scaffolds, and the challenges they face when conducting experiments with their own prototypes.

Travel experiences offer a diverse view into an individual's interactions with different cultures, societies, and places. In this paper, we present a 2.5-year autoethnographic travel account of a hard of hearing individual-Jain. Through retrospective journals and field notes, we reveal the tensions and nuances in his travel, including the magnified difficulty of social conversations, issues with navigating unfamiliar environments and cultural contexts, and changes in the relationship to personal assistive technologies. By exploring the longitudinal travel experiences of a single individual, we uncover evocative and personal insights rarely available through participant-based research methods. Based on these lived experiences and post hoc reflections, we present two design explorations of personalized technology the autoethnographer created for aiding his travel. Finally, we offer reflections for customized travel technologies for deaf and hard of hearing users, and methodological guidelines for performing first-person research in the context of disability.

Blind and visually impaired (BVI) individuals are increasingly creating visual content online; however, there is a lack of tools that allow these individuals to modify the visual attributes of the content and verify the validity of those modifications. In this poster paper, we discuss the design and preliminary exploration of a multi-modal and accessible approach for BVI developers to edit visual layouts of webpages while maintaining visual aesthetics.

Recent work has applied machine learning methods to automatically find and/or assess pedestrian infrastructure in online map imagery (e.g., satellite photos, streetscape panoramas). While promising, these methods have been limited by two interrelated issues: small training sets and the choice of machine learning model. In this paper, aided by the recently released Project Sidewalk dataset of 300,000+ image-based sidewalk accessibility labels, we present the first examination of deep learning to automatically assess sidewalks in Google Street View (GSV) panoramas. Specifically, we investigate two application areas: automatically validating crowdsourced labels and automatically labeling sidewalk accessibility issues. For both tasks, we introduce and use a residual neural network (ResNet) modified to support both image and non-image (contextual) features (e.g., geography). We present an analysis of performance, the effect of our non-image features and training set size, and cross-city generalizability. Our results significantly improve on prior automated methods and, in some cases, meet or exceed human labeling performance.

To investigate preferences for mobile and wearable sound awareness systems, we conducted an online survey with 201 DHH participants. The survey explores how demographic factors affect perceptions of sound awareness technologies, gauges interest in specific sounds and sound characteristics, solicits reactions to three design scenarios (smartphone, smartwatch, head-mounted display) and two output modalities (visual, haptic), and probes issues related to social context of use. While most participants were highly interested in being aware of sounds, this interest was modulated by communication preference--that is, for sign or oral communication or both. Almost all participants wanted both visual and haptic feedback and 75% preferred to have that feedback on separate devices (e.g., haptic on smartwatch, visual on head-mounted display). Other findings related to sound type, full captions vs. keywords, sound filtering, notification styles, and social context provide direct guidance for the design of future mobile and wearable sound awareness systems.

Many traditional HCI methods, such as surveys and interviews, are of limited value when working with preschoolers. In this paper, we present anchored audio sampling (AAS), a remote data collection technique for extracting qualitative audio samples during field deployments with young children. AAS offers a developmentally sensitive way of understanding how children make sense of technology and situates their use in the larger context of daily life. AAS is defined by an anchor event, around which audio is collected. A sliding window surrounding this anchor captures both antecedent and ensuing recording, providing the researcher insight into the activities that led up to the event of interest as well as those that followed. We present themes from three deployments that leverage this technique. Based on our experiences using AAS, we have also developed a reusable open-source library for embedding AAS into any Android application.

The home is filled with a rich diversity of sounds from mundane beeps and whirs to dog barks and children's shouts. In this paper, we examine how deaf and hard of hearing (DHH) people think about and relate to sounds in the home, solicit feedback and reactions to initial domestic sound awareness systems, and explore potential concerns. We present findings from two qualitative studies: in Study 1, 12 DHH participants discussed their perceptions of and experiences with sound in the home and provided feedback on initial sound awareness mockups. Informed by Study 1, we designed three tablet-based sound awareness prototypes, which we evaluated with 10 DHH participants using a Wizard-of-Oz approach. Together, our findings suggest a general interest in smarthome-based sound awareness systems particularly for displaying contextually aware, personalized and glanceable visualizations but key concerns arose related to privacy, activity tracking, cognitive overload, and trust.

Underperforming, degraded, and missing insulation in US residential buildings is common. Detecting these issues, however, can be difficult. Using thermal cameras during energy audits can aid in locating potential insulation issues, but prior work indicates it is challenging to determine their severity using thermal imagery alone. In this work, we present an easy-to-deploy, temporal thermographic sensor system designed to support residential energy audits through quantitative analysis of building envelope performance. We then offer an evaluation of the system through two studies: (i) a one-week, in-home field study in five homes and (ii) a semi-structured interview study with five professional energy auditors. Our results show our system helps raise awareness, improves homeowners' ability to gauge the severity of issues, and provides opportunities for new interactions between homeowners, building data, and professional auditors.

We introduce Project Sidewalk, a new web-based tool that enables online crowdworkers to remotely label pedestrian-related accessibility problems by virtually walking through city streets in Google Street View. To train, engage, and sustain users, we apply basic game design principles such as interactive onboarding, mission-based tasks, and progress dashboards. In an 18-month deployment study, 797 online users contributed 205,385 labels and audited 2,941 miles of Washington DC streets. We compare behavioral and labeling quality differences between paid crowdworkers and volunteers, investigate the effects of label type, label severity, and majority vote on accuracy, and analyze common labeling errors. To complement these findings, we report on an interview study with three key stakeholder groups (N=14) soliciting reactions to our tool and methods. Our findings demonstrate the potential of virtually auditing urban accessibility and highlight tradeoffs between scalability and quality compared to traditional approaches.

We present Ondulé-an interactive design tool that allows novices to create parameterizable deformation behaviors in 3D-printable models using helical springs and embedded joints. Informed by spring theory and our empirical mechanical experiments, we introduce spring and joint-based design techniques that support a range of parameterizable deformation behaviors, including compress, extend, twist, bend, and various combinations. To enable users to design and add these deformations to their models, we introduce a custom design tool for Rhino. Here, users can convert selected geometries into springs, customize spring stiffness, and parameterize their design with mechanical constraints for desired behaviors. To demonstrate the feasibility of our approach and the breadth of new designs that it enables, we showcase a set of example 3D-printed applications from launching rocket toys to tangible storytelling props. We conclude with a discussion of key challenges and open research questions.

Prior work has explored communication challenges faced by people who are deaf and hard of hearing (DHH) and the potential role of new captioning and support technologies to address these challenges; however, the focus has been on stationary contexts such as group meetings and lectures. In this paper, we present two studies examining the needs of DHH people in moving contexts (e.g., walking) and the potential for mobile captions on head-mounted displays (HMDs) to support those needs. Our formative study with 12 DHH participants identifies social and environmental challenges unique to or exacerbated by moving contexts. Informed by these findings, we introduce and evaluate a proof-of-concept HMD prototype with 10 DHH participants. Results show that, while walking, HMD captions can support communication access and improve attentional balance between the speakers(s) and navigating the environment. We close by describing open questions in the mobile context space and design guidelines for future technology.

Walkability indices such as walkscore.com model the proximity and density of walkable destinations within a neighborhood. While these metrics have gained widespread use (e.g., incorporated into real-estate tools), they do not integrate accessibility-related features such as sidewalk conditions or curb ramps-thereby excluding a significant portion of the population. In this poster paper, we explore the initial design and implementation of neighborhood accessibility models and visualizations for people with mobility impairments. We are able to overcome previous data availability challenges by using the Project Sidewalk API, which provides access to 255,000+ labels about the accessibility and location of DC sidewalks.

Previous work has explored scalable methods to collect data on the accessibility of the built environment by combining manual labeling, computer vision, and online map imagery. In this poster paper, we explore how to extend these methods to track the evolution of urban accessibility over time. Using Google Street View's "time machine" feature, we introduce a three-stage classification framework: (i) manually labeling accessibility problems in one time period; (ii) classifying the labeled image patch into one of five accessibility categories; (iii) localizing the patch in all previous snapshots. Our preliminary results analyzing 1633 Street View images across 376 locations demonstrate feasibility.

Augmented reality (AR) systems that enhance visual capabilities could make text and other fine details more accessible for low vision users, improving independence and quality of life. Prior work has begun to investigate the potential of assistive AR, but recent advancements enable new AR visualizations and interactions not yet explored in the context of assistive technology. In this paper, we follow an iterative design process with feedback and suggestions from seven visually impaired participants, designing and testing AR magnification ideas using the Microsoft HoloLens. Participants identified several advantages to the concept of head-worn magnification (e.g., portability, privacy, ready availability), and to our AR designs in particular (e.g., a more natural reading experience and the ability to multitask). We discuss the strengths and weaknesses of this AR magnification approach and summarize lessons learned throughout the process.

Color identification tools do not identify visual patterns or allow users to quickly inspect multiple locations, which are both important for identifying clothing. We are exploring the use of a finger-based camera that allows users to query clothing colors and patterns by touch. Previously, we demonstrated the feasibility of this approach using a small, highly-controlled dataset and combining two image classification techniques commonly used for object recognition. Here, to improve scalability and robustness, we collect a dataset of fabric images from online sources and apply transfer learning to train an end-to-end deep neural network to recognize visual patterns. This new approach achieves 92% accuracy in a general case and 97% when tuned for images from a finger-mounted camera.

The emergence of low-cost thermographic cameras for mobile devices provides users with new practical and creative prospects. While recent work has investigated how novices use thermal cameras for energy auditing tasks in structured activities, open questions remain about "in the wild" use and the challenges or opportunities therein. To study these issues, we analyzed 1,000 YouTube videos depicting everyday uses of thermal cameras by non-professional, novice users. We coded the videos by content area, identified whether common misconceptions regarding thermography were present, and analyzed questions within the comment threads. To complement this analysis, we conducted an online survey of the YouTube content creators to better understand user behaviors and motivations. Our findings characterize common thermographic use cases, extend discussions surrounding the challenges novices encounter, and have implications for the design of future thermographic systems and tools.

We present early work developing an Augmented Reality (AR) system that allows young children to design and experiment with complex systems (e.g., bicycle gears, human circulatory system). Our novel approach combines low-fidelity prototyping to help children represent creative ideas, AR visualization to scaffold iterative design, and virtual simulation to support personalized experiments. To evaluate our approach, we conducted an exploratory study with eight children (ages 8-11) using an initial prototype. Our findings demonstrate the viability of our approach, uncover usability challenges, and suggest opportunities for future work. We also distill additional design implications from a follow-up participatory design session with children.

Tactile overlays with audio annotations can increase the accessibility of touchscreens for blind users; however, preparing these overlays is complex and labor intensive. We introduce TacTILE, a novel toolchain to more easily create tactile overlays with audio annotations for arbitrary touchscreen graphics (e.g., graphs, pictures, maps). The workflow includes: (i) an annotation tool to add audio to graphical elements, (ii) a fabrication process that generates 3D-printed tactile overlays, and (iii) a custom app for the user to explore graphics with these overlays. We close with a pilot study with one blind participant who explores three examples (floor plan, photo, and chart), and a discussion of future work.

Precisely guiding a blind person's hand can be useful for a range of applications from tracing printed text to learning and understanding shapes and gestures. In this paper, we evaluate wrist-worn haptics as a directional hand guide. We implemented and evaluated the following haptic wristband variations: (1) four versus eight vibromotor designs; (2) vibration from only a single motor at a time versus from two adjacent motors using interpolation. To evaluate our designs, we conducted two studies: Study 1 (N=13, 2 blind) showed that participants could non-visually find targets and trace paths more quickly and accurately with single-motor feedback than with interpolated feedback, particularly when only four motors were used. Study 2 (N=14 blind or visually impaired participants) found that single-motor feedback with four motors was faster, more accurate, and most preferred compared to similar feedback with eight motors. We derive implications for the design of wrist-worn directional haptic feedback and discuss future work.

We investigate clothing color and visual texture recognition using images from a finger-mounted camera to support people with visual impairments. Our approach mitigates issues with distance and lighting that can impact the accuracy of existing color and texture recognizers and allows for easy touch-based interrogation to better understand clothing appearance. We classify image textures by combining two off-the-shelf techniques commonly used for object recognition achieving 99.4% accuracy on a dataset of 520 clothing images across 9 texture categories. We close with a discussion of potential applications, user evaluation plans, and open questions.

For screenreader users who are blind or visually impaired (VI), today's mobile devices, while reasonably accessible, are not necessarily efficient. This inefficiency may be especially problematic for microinteractions, which are brief but high-frequency interactions that take only a few seconds for sighted users to complete (e.g., checking the weather or for new messages). One potential solution to support efficient non-visual microinteractions is on-body input, which appropriates the user's own body as the interaction medium. In this paper, we address two related research questions: How well are microinteractions currently supported for VI users' How should on-body interaction be designed to best support microinteractions for this user group? We conducted two studies: (1) an online survey to compare current microinteraction use between VI and sighted users (N=117); and (2) an in-person study where 12 VI screenreader users qualitatively evaluated a real-time on-body interaction system that provided three contrasting input designs. Our findings suggest that efficient microinteractions are not currently well-supported for VI users, at least using manual input, which highlights the need for new interaction approaches. On-body input offers this potential and the qualitative evaluation revealed tradeoffs with different on-body interaction techniques in terms of perceived efficiency, learnability, social acceptability, and ability to use on the go.

We present Project Sidewalk, a new online tool that allows anyone-from motivated citizens to government workers-to remotely label accessibility problems by virtually walking through city streets. Basic game design principles such as interactive onboarding, mission-based tasks, and stats dashboards are used to train, engage, and sustain users. We describe the current Project Sidewalk system, present results of a pilot public deployment with 581 users, and discuss open questions and future work.

Recent technical advances have enabled new wearable augmented reality (AR) solutions that can aid people with visual impairments (VI) in their everyday lives. Here, we investigate an AR-based magnification solution that combines a small finger-worn camera with a transparent augmented reality display (the Microsoft Hololens). The image from the camera is processed and projected on the Hololens to magnify visible content below the user's finger such as text and images. Our approach offers: (i) a close-up camera view (similar to a CCTV system) with the portability and processing power of a smartphone magnifier app, (ii) access to content through direct touch, and (iii) flexible placement of the magnified image within the wearer's field of view. We present three proof-of-concept interfaces and plans for a user evaluation.

Wearable sensing poses new opportunities to enhance personal connections to learning and authentic scientific inquiry experiences. In our work, we leverage the body and physical action as an engaging platform for learning through live physiological sensing and visualization (LPSV). Prior research suggests the potential of this approach, but was limited to single-session evaluations in informal environments. In this paper, we examine LPSV tools in a classroom environment during a four-day deployment. To highlight the complex interconnections between space, teachers, curriculum, and tool use, we analyze our data through the lens of Activity Theory. Our findings show the importance of integrating model-based representations for supporting exploration and analytic representations for scaffolding scientific inquiry. Activity Theory highlights leveraging life-relevant connections available within a physical space and considering policies and norms related to learners' physical bodies.

Controlled studies of touchscreen input performance for users with upper body motor impairments remain relatively sparse. To address this gap, we present a controlled lab study of mouse vs. touchscreen performance with 32 participants (16 with upper body motor impairments and 16 without). Our study examines: (1) how touch input compares to an indirect pointing device (a mouse); (2) how performance compares across a range of standard interaction techniques; and (3) how these answers differ for users with and without motor impairments. While the touchscreen was faster than the mouse overall, only participants without motor impairments benefited from a lower error rate on the touchscreen. Indeed, participants with motor impairments had a three-fold increase in pointing (tapping) errors on the touchscreen compared to the mouse. Our findings also highlight the high frequency of spurious touches for users with motor impairments and update past accessibility recommendations for minimum touchscreen target sizes to at least 18mm.

Research on the viability of using crowdsourcing for HCI performance experiments has concluded that online results are similar to those achieved in the lab---at least for desktop interactions. However, mobile devices, the most popular form of online access today, may be more problematic due to variability in the user's posture and in movement of the device. To assess this possibility, we conducted two experiments with 30 lab-based and 303 crowdsourced participants using basic mouse and touchscreen tasks. Our findings show that: (1) separately analyzing the crowd and lab data yields different study conclusions-touchscreen input was significantly less error prone than mouse input in the lab but more error prone online; (2) age-matched crowdsourced participants were significantly faster and less accurate than their lab-based counterparts, contrasting past work; (3) variability in mobile device movement and orientation increased as experimenter control decreased--a potential factor affecting the touchscreen error differences. This study cautions against assuming that crowdsourced data for performance experiments will directly reflect lab-based data, particularly for mobile devices.

Wearable construction toolkits have shown promise in broadening participation in computing and empowering users to create personally meaningful computational designs. However, these kits present a high barrier of entry for some users, particularly young children (K-6). In this paper, we introduce MakerWear, a new wearable construction kit for children that uses a tangible, modular approach to wearable creation. We describe our participatory design process, the iterative development of MakerWear, and results from single- and multi-session workshops with 32 children (ages 5-12; M=8.3 years). Our findings reveal how children engage in wearable design, what they make (and want to make), and what challenges they face. As a secondary analysis, we also explore age-related differences.

The recent integration of thermal cameras with commodity smartphones presents an opportunity to engage the public in evaluating energy-efficiency issues in the built environment. However, it is unclear how novice users without professional experience or training approach thermographic energy auditing activities. In this paper, we recruited 10 participants for a four-week field study of end-user behavior exploring novice approaches to semi-structured thermographic energy auditing tasks. We analyze thermographic imagery captured by participants as well as weekly surveys and post-study debrief interviews. Our findings suggest that while novice users perceived thermal cameras as useful in identifying energy-efficiency issues in buildings, they struggled with interpretation and confidence. We characterize how novices perform thermographic-based energy auditing, synthesize key challenges, and discuss implications for design.

We present SqueezaPulse, a technique for embedding interactivity into fabricated objects using soft, passive, low-cost bellow-like structures. When a soft cavity is squeezed, air pulses travel along a flexible pipe and into a uniquely designed corrugated tube that shapes the airflow into predictable sound signatures. A microphone captures and identifies these air pulses enabling interactivity. We describe the underlying acoustic theory used to inform our design, an informal examination of the effect of different 3D-printed corrugations on air signatures, and our resulting SqueezaPulse implementation. To demonstrate and evaluate the potential of SqueezaPulse, we present four prototype applications and a small, lab-based user study (N=9). Our evaluations show that our approach is accurate across users and robust to external noise. We conclude with a discussion of limitations and future work.

We present and evaluate a new mixed-reality tool called SharedPhys, which tightly integrates real-time physiological sensing, whole-body interaction, and responsive large-screen visualizations to support new forms of embodied interaction and collaborative learning. While our primary content area is the human body, we use the body and physical activity as a pathway to other STEM areas such as biology, health, and mathematics. We describe our participatory design process with 20 elementary school teachers, the development of three contrasting SharedPhys prototypes, and results from six exploratory evaluations in two after-school programs. Our findings suggest that the tight coupling between physical interaction, sensing, and visualization in a multi-user environment helps promote engagement, allows children to easily explore cause-and-effect relationships, supports and shapes social interactions, and promotes playful experiences.

In this paper, we investigate how people with mobility impairments assess and evaluate accessibility in the built environment and the role of current and emerging location-based technologies therein. We conducted a three-part formative study with 20 mobility impaired participants: a semi-structured interview (Part 1), a participatory design activity (Part 2), and a design probe activity (Part 3). Part 2 and 3 actively engaged our participants in exploring and designing the future of what we call assistive location-based technologies (ALTs) location-based technologies that specifically incorporate accessibility features to support navigating, searching, and exploring the physical world. Our Part 1 findings highlight how existing mapping tools provide accessibility benefits even though often not explicitly designed for such uses. Findings in Part 2 and 3 help identify and uncover useful features of future ALTs. In particular, we synthesize 10 key features and 6 key data qualities. We conclude with ALT design recommendations.

Skin-based biometrics rely on the distinctiveness of skin patterns across individuals for identification. In this paper, we investigate whether small image patches of the skin can be localized on a user's body, determining not “who?” instead “where?” Applying techniques from biometrics and computer vision, we introduce a hierarchical classifier that estimates a location from the image texture and refines the estimate with keypoint matching and geometric verification. To evaluate our approach, we collected 10,198 close-up images of 17 hand and wrist locations across 30 participants. Within-person algorithmic experiments demonstrate that an individual's own skin features can be used to localize their skin surface image patches with an F1score of 96.5%. As secondary analyses, we assess the effects of training set size and between-person classification. We close with a discussion of the strengths and limitations of our approach and evaluation methods as well as implications for future applications using a wearable camera to support touch-based, location-specific taps and gestures on the surface of the skin.

Electronic textile (e-textile) toolkits have been successful in broadening participation in STEAM-related activities, in expanding perceptions of computing, and in engaging users in creative, expressive, and meaningful digital-physical design. While a range of well-designed e-textile toolkits exist (e.g., LilyPad), they cater primarily to adults and older children and have a high barrier of entry for some users. We are investigating new approaches to support younger children (K-4) in the creative design, play, and customization of e-textiles and wearables without requiring the creation of code. This demo paper presents one such example of ongoing work: MakerShoe, an e-textile platform for designing shoe-based interactive wearable experiences. We discuss our two participatory design sessions as well as our initial prototype, which uses single-function magnetically attachable electronic modules to support circuit creation and the design of responsive, interactive behaviors.

The HandSight project investigates how wearable micro-cameras can be used to augment a blind or visually impaired user--s sense of touch with computer vision. Our goal is to support an array of activities of daily living by sensing and feeding back non-tactile information (e.g., color, printed text, patterns) about an object as it is touched. In this poster paper, we provide an overview of the project, our current proof-of-concept prototype, and a summary of findings from finger-based text reading studies. As this is an early-stage project, we also enumerate current open questions.

Persons with hearing loss use visual signals such as gestures and lip movement to interpret speech. While hearing aids and cochlear implants can improve sound recognition, they generally do not help the wearer localize sound necessary to leverage these visual cues. In this paper, we design and evaluate visualizations for spatially locating sound on a head-mounted display (HMD). To investigate this design space, we developed eight high-level visual sound feedback dimensions. For each dimension, we created 3-12 example visualizations and evaluated these as a design probe with 24 deaf and hard of hearing participants (Study 1). We then implemented a real-time proof-of-concept HMD prototype and solicited feedback from 4 new participants (Study 2). Study 1 findings reaffirm past work on challenges faced by persons with hearing loss in group conversations, provide support for the general idea of sound awareness visualizations on HMDs, and reveal preferences for specific design options. Although preliminary, Study 2 further contextualizes the design probe and uncovers directions for future work.

The building sector accounts for 41% of primary energy consumption in the US, contributing an increasing portion of the country's carbon dioxide emissions. With recent sensor improvements and falling costs, auditors are increasingly using thermography-infrared (IR) cameras-to detect thermal defects and analyze building efficiency. Research in automated thermography has grown commensurately, aimed at reducing manual labor and improving thermal models. Though promising, we could find no prior work exploring the professional auditor's perspectives of thermography or reactions to emerging automation. To address this gap, we present results from two studies: a semi-structured interview with 10 professional energy auditors, which includes design probes of five automated thermography scenarios, and an observational case study of a residential audit. We report on common perspectives, concerns, and benefits related to thermography and summarize reactions to our automated scenarios. Our findings have implications for thermography tool designers as well as researchers working on automated solutions in robotics, computer science, and engineering.

Internal organs are hidden and untouchable, making it difficult for children to learn their size, position, and function. Traditionally, human anatomy (body form) and physiology (body function) are taught using techniques ranging from worksheets to three-dimensional models. We present a new approach called BodyVis, an e-textile shirt that combines biometric sensing and wearable visualizations to reveal otherwise invisible body parts and functions. We describe our 15-month iterative design process including lessons learned through the development of three prototypes using participatory design and two evaluations of the final prototype: a design probe interview with seven elementary school teachers and three single-session deployments in after-school programs. Our findings have implications for the growing area of wearables and tangibles for learning.

Group exercise has multiple benefits including greater adherence to fitness regimens, increased enjoyment among participants, and enhanced workout intensity. While a large number of technology tools have emerged to support real-time feedback of individual performance, tools to support group fitness are limited. In this paper, we present a set of wearable e-textile displays for running groups called Social Fabric Fitness (SFF). SFF provides a glanceable, shared screen on the back of the wearer's shirt to increase awareness and motivation of group fitness performance. We discuss parallel prototyping of three designs-one flexible e-ink and two flexible LED-based displays; the selection and refinement of one design; and two evaluations'a field study of 10 running groups and two case studies of running races. Our qualitative findings indicate that SFF improves awareness of individual and group performance, helps groups stay together, and improves in-situ motivation. We close with reflections for future athletic e-textile displays.

Building on recent prior work that combines Google Street View (GSV) and crowdsourcing to remotely collect information on physical world accessibility, we present the first 'smart' system, Tohme, that combines machine learning, computer vision (CV), and custom crowd interfaces to find curb ramps remotely in GSV scenes. Tohme consists of two workflows, a human labeling pipeline and a CV pipeline with human verification, which are scheduled dynamically based on predicted performance. Using 1,086 GSV scenes (street intersections) from four North American cities and data from 403 crowd workers, we show that Tohme performs similarly in detecting curb ramps compared to a manual labeling approach alone (F- measure: 84% vs. 86% baseline) but at a 13% reduction in time cost. Our work contributes the first CV-based curb ramp detection system, a custom machine-learning based workflow controller, a validation of GSV as a viable curb ramp data source, and a detailed examination of why curb ramp detection is a hard problem along with steps forward.

Unlike external body parts, organs are invisible and untouchable, making it difficult for children to learn their size, position, and function. Traditionally, human anatomy (body form) and physiology (body function) are taught using a mixture of techniques from worksheets to three-dimensional models. With the advent of low-cost sensing, ubiquitous computation, and emerging e-textiles, new teaching approaches are developing that link the physical and virtual worlds. In this demo, we explore and illustrate the use of a custom wearable e-textile shirt to teach anatomy and physiology to children. Our current implementation uses dyed fabric to show the size and position of body organs and a mixture of electronic sensors and visualizations to dynamically exhibit the wearer's physiology (e.g., heart beat). Though we are still in an early design stage, we discuss our design process, our progress thus far, and plans for future iterations.

Low-vision and blind bus riders often rely on known physical landmarks to help locate and verify bus stop locations (e.g., by searching for a shelter, bench, newspaper bin). However, there are currently few, if any, methods to determine this information a priori via computational tools or services. In this paper, we introduce and evaluate a new scalable method for collecting bus stop location and landmark descriptions by combining online crowdsourcing and Google Street View (GSV). We conduct and report on three studies in particular: (i) a formative interview study of 18 people with visual impairments to inform the design of our crowdsourcing tool; (ii) a comparative study examining differences between physical bus stop audit data and audits conducted virtually with GSV; and (iii) an online study of 153 crowd workers on Amazon Mechanical Turk to examine the feasibility of crowdsourcing bus stop audits using our custom tool with GSV. Our findings reemphasize the importance of landmarks in non-visual navigation, demonstrate that GSV is a viable bus stop audit dataset, and show that minimally trained crowd workers can find and identify bus stop landmarks with 82.5% accuracy across 150 bus stop locations (87.3% with simple quality control).

Touchscreen devices have become one of the most pervasive video game platforms in the world and, in turn, an integral part of popular culture; however, little work exists on comprehensively examining their accessibility. In this poster paper, we present initial findings from a survey and qualitative analysis of popular iPad touchscreen games with a focus on exploring factors relevant to persons with motor impairments. This paper contributes a novel qualitative codebook with which to examine the accessibility of touchscreen games for users with motor impairments and the results from applying this codebook to 72 iPad games.

Despite the apparent popularity of touchscreens for older adults, little is known about the psychomotor performance of these devices. We compared performance between older adults and younger adults on four desktop and touchscreen tasks: pointing, dragging, crossing and steering. On the touchscreen, we also examined pinch-to-zoom. Our results show that while older adults were significantly slower than younger adults in general, the touchscreen reduced this performance gap relative to the desktop and mouse. Indeed, the touchscreen resulted in a significant movement time reduction of 35% over the mouse for older adults, compared to only 16% for younger adults. Error rates also decreased.

Poorly maintained sidewalks, missing curb ramps, and other obstacles pose considerable accessibility challenges; however, there are currently few, if any, mechanisms to determine accessible areas of a city a priori. In this paper, we investigate the feasibility of using untrained crowd workers from Amazon Mechanical Turk (turkers) to find, label, and assess sidewalk accessibility problems in Google Street View imagery. We report on two studies: Study 1 examines the feasibility of this labeling task with six dedicated labelers including three wheelchair users; Study 2 investigates the comparative performance of turkers. In all, we collected 13,379 labels and 19,189 verification labels from a total of 402 turkers. We show that turkers are capable of determining the presence of an accessibility problem with 81% accuracy. With simple quality control methods, this number increases to 93%. Our work demonstrates a promising new, highly scalable method for acquiring knowledge about sidewalk accessibility.

Researchers in HCI and behavioral science are increasingly exploring the use of technology to support behavior change in domains such as health and sustainability. This work, however, remain largely siloed within the two communities. We begin to address this silo problem by attempting to build a bridge between the two disciplines at the level of behavioral theory. Specifically, we define core theoretical terms to create shared understanding about what theory is, discuss ways in which behavioral theory can be used to inform research on behavior change technologies, identify shortcomings in current behavioral theories, and outline ways in which HCI researchers can not only interpret and utilize behavioral science theories but also contribute to improving them.

We explore the feasibility of using crowd workers from Amazon Mechanical Turk to identify and rank sidewalk accessibility issues from a manually curated database of 100 Google Street View images. We examine the effect of three different interactive labeling interfaces (Point, Rectangle, and Outline) on task accuracy and duration. We close the paper by discussing limitations and opportunities for future work.

Few means currently exist for home occupants to learn about their water consumption: e.g., where water use occurs, whether such use is excessive and what steps can be taken to conserve. Emerging water sensing systems, however, can provide detailed usage data at the level of individual water fixtures (i.e., disaggregated usage data). In this paper, we perform formative evaluations of two sets of novel eco-feedback displays that take advantage of this disaggregated data. The first display set isolates and examines specific elements of an eco-feedback design space such as data and time granularity. Displays in the second set act as design probes to elicit reactions about competition, privacy, and integration into domestic space. The displays were evaluated via an online survey of 651 North American respondents and in-home, semi-structured interviews with 10 families (20 adults). Our findings are relevant not only to the design of future water eco-feedback systems but also for other types of consumption (e.g., electricity and gas).

Eco-feedback technology provides feedback on individual or group behaviors with a goal of reducing environmental impact. The history of eco-feedback extends back more than 40 years to the origins of environmental psychology. Despite its stated purpose, few HCI eco-feedback studies have attempted to measure behavior change. This leads to two overarching questions: (1) what can HCI learn from environmental psychology and (2) what role should HCI have in designing and evaluating eco-feedback technology? To help answer these questions, this paper conducts a comparative survey of eco-feedback technology, including 89 papers from environmental psychology and 44 papers from the HCI and UbiComp literature. We also provide an overview of predominant models of proenvironmental behaviors and a summary of key motivation techniques to promote this behavior.

Traveller information, route planning, and service updates have become essential components of public transport systems: they help people navigate built environments by providing access to information regarding delays and service disruptions. However, one aspect that these systems lack is a way of tailoring the information they offer in order to provide personalised trip time estimates and relevant notifications to each traveller. Mining each user's travel history, collected by automated ticketing systems, has the potential to address this gap. In this work, we analyse one such dataset of travel history on the London underground. We then propose and evaluate methods to (a) predict personalised trip times for the system users and (b) rank stations based on future mobility patterns, in order to identify the subset of stations that are of greatest interest to the user and thus provide useful travel updates.

The greatest contributor of CO2 emissions in the average American household is personal transportation. Because transportation is inherently a mobile activity, mobile devices are well suited to sense and provide feedback about these activities. In this paper, we explore the use of personal ambient displays on mobile phones to give users feedback about sensed and self-reported transportation behaviors. We first present results from a set of formative studies exploring our respondents' existing transportation routines, willingness to engage in and maintain green transportation behavior, and reactions to early mobile phone "green" application design concepts. We then describe the results of a 3-week field study (N=13) of the UbiGreen Transportation Display prototype, a mobile phone application that semi-automatically senses and reveals information about transportation behavior. Our contributions include a working system for semi-automatically tracking transit activity, a visual design capable of engaging users in the goal of increasing green transportation, and the results of our studies, which have implications for the design of future green applications.

Recent work has examined infrastructure-mediated sensing as a practical, low-cost, and unobtrusive approach to sensing human activity in the physical world. This approach is based on the idea that human activities (e.g., running a dishwasher, turning on a reading light, or walking through a doorway) can be sensed by their manifestations in an environment's existing infrastructures (e.g., a home's water, electrical, and HVAC infrastructures). This paper presents HydroSense, a low-cost and easily-installed single-point sensor of pressure within a home's water infrastructure. HydroSense supports both identification of activity at individual water fixtures within a home (e.g., a particular toilet, a kitchen sink, a particular shower) as well as estimation of the amount of water being used at each fixture. We evaluate our approach using data collected in ten homes. Our algorithms successfully identify fixture events with 97.9% aggregate accuracy and can estimate water usage with error rates that are comparable to empirical studies of traditional utility-supplied water meters. Our results both validate our approach and provide a basis for future improvements.

In a Mobile Living Lab, mobile devices are used to evaluate concepts and prototypes in real-life settings. In other words, the lab is brought to the people. This workshop provides a forum for researchers and practitioners to share experiences and issues with methods and tools for Mobile Living Labs. In particular, we seek to bring together people who have applied methods for Mobile Living Labs and people who build tools for those methods.

Recent advances in small inexpensive sensors, low-power processing, and activity modeling have enabled applications that use on-body sensing and machine learning to infer people's activities throughout everyday life. To address the growing rate of sedentary lifestyles, we have developed a system, UbiFit Garden, which uses these technologies and a personal, mobile display to encourage physical activity. We conducted a 3-week field trial in which 12 participants used the system and report findings focusing on their experiences with the sensing and activity inference. We discuss key implications for systems that use on-body sensing and activity inference to encourage physical activity.

Personal, mobile displays, such as those on mobile phones, are ubiquitous, yet for the most part, underutilized. We present results from a field experiment that investigated the effectiveness of these displays as a means for improving awareness of daily life (in our case, self-monitoring of physical activity). Twenty-eight participants in three experimental conditions used our UbiFit system for a period of three months in their day-to-day lives over the winter holiday season. Our results show, for example, that participants who had an awareness display were able to maintain their physical activity level (even during the holidays), while the level of physical activity for participants who did not have an awareness display dropped significantly. We discuss our results and their general implications for the use of everyday mobile devices as awareness displays.

Users’ ability to accurately recall frequent, habitual activities is fundamental to a number of disciplines, from health sciences to machine learning. However, few, if any, studies exist that have assessed optimal sampling strategies for in situ self-reports. In addition, few technologies exist that facilitate benchmarking self-report accuracy for routine activities. We report on a study investigating the effect of sampling frequency of self-reports of two routine activities (sitting and walking) on recall accuracy and annoyance. We used a novel wearable sensor platform that runs a real time activity inference engine to collect in situ ground truth. Our results suggest that a sampling frequency of five to eight times per day may yield an optimal balance of recall and annoyance. Additionally, requesting self-reports at regular, predetermined times increases accuracy while minimizing perceived annoyance since it allows participants to anticipate these requests. We discuss our results and their implications for future studies.

Mobile phones and personal digital assistants (PDAs) are incredibly popular pervasive technologies. Many of these devices contain touch screens, which can present problems for users with motor impairments due to small targets and their reliance on tapping for target acquisition. In order to select a target, users must tap on the screen, an action which requires the precise motion of flying into a target and lifting without slipping. In this paper, we propose a new technique for target acquisition called barrier pointing, which leverages the elevated physical edges surrounding the screen to improve pointing accuracy. After designing a series of barrier pointing techniques, we conducted an initial study with 9 able bodied users and 9 users with motor impairments in order to discover the parameters that make barrier pointing successful. From this data, we offer an in-depth analysis of the performance of two motor impaired users for whom barrier pointing was especially beneficial. We show the importance of providing physical stability by allowing the stylus to press against the screen and its physical edge. We offer other design insights and lessons learned that can inform future attempts at leveraging the physical properties of mobile devices to improve accessibility.

This paper presents MyExperience, a system for capturing both objective and subjective in situ data on mobile computing activities. MyExperience combines the following two techniques: 1) passive logging of device usage, user context, and environmental sensor readings, and 2) active context-triggered user experience sampling to collect in situ, subjective user feedback. MyExperience currently runs on mobile phones and supports logging of more than 140 event types, including: 1) device usage such as communication, application usage, and media capture, 2) user context such as calendar appointments, and 3) environmental sensing such as Bluetooth and GPS. In addition, user experience sampling can be targeted to moments of interest by triggering off sensor readings. We present several case studies of field deployments on people's personal phones to demonstrate how MyExperience can be used effectively to understand how people use and experience mobile technology.

To evaluate ubiquitous computing technologies, which may be embedded in the environment, embedded in objects, worn, or carried by the user throughout everyday life, it is essential to use methods that accommodate the often unpredictable, real-world environments in which the technologies are used. This article discusses how we have adapted and applied traditional methods from psychology and human-computer interaction, such as Wizard of Oz and Experience Sampling, to be more amenable to the in situ evaluations of ubiquitous computing applications, particularly in the early stages of design. The way that ubiquitous computing technologies can facilitate the in situ collection of self-report data is also discussed. Although the focus is on ubiquitous computing applications and tools for their assessment, it is believed that the in situ evaluation tools that are proposed will be generally useful for field trials of other technology, applications, or formative studies that are concerned with collecting data in situ.

In distributed software development, two sorts of dependencies can arise. The structure of the software system itself can create dependencies between software elements, while the structure of the development process can create dependencies between software developers. Each of these both shapes and reflects the development process. Our research concerns the extent to which, by looking uniformly at artifacts and activities, we can uncover the structures of software projects, and the ways in which development processes are inscribed into software artifacts. We show how a range of organizational processes and arrangements can be uncovered in software repositories, with implications for collaborative work in large distributed groups such as open source communities.

In large projects, software developers struggle with two sources of complexity - the complexity of the code itself, and the complexity of the process of producing it. Both of these concerns have been subjected to considerable research investigation, and tools and techniques have been developed to help manage them. However, these solutions have generally been developed independently, making it difficult to deal with problems that inherently span both dimensions. We describe Augur, a visualization tool that supports distributed software development processes. Augur creates visual representations of both software artifacts and software development activities, and, crucially, allows developers to explore the relationship between them. Augur is designed not for managers, but for the developers participating in the software development process. We discuss some of the early results of informal evaluation with open source software developers. Our experiences to date suggest that combining views of artifacts and activities is both meaningful and valuable to software developers.