EDBT 2026 Demo / reviewers in the wild / expert
Sooyeon Lee
dblp:51/7304
· DBLP profile ↗
34ranked-venue papers
9as first author
26since 2021 · last 2026
—ORCID · conflict
Domains — the database's venue-derived domains; a paper can count in several
Human-computer interaction and ubiquitous computing · 30 · 6 first-author · 24 since 2021Software engineering, systems software and programming languages · 2 · 2 first-author · 1 since 2021Applied, interdisciplinary, general and emerging computing · 2 · 1 first-authorGraphics, computer vision, multimedia, augmented reality and games · 1 · 1 first-author · 1 since 2021
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | Like, Comment & Caption: A Decade of Social Media Video Caption Research (2015-2025)abstractAs video has become the dominant mode of content on platforms such as YouTube, TikTok, and Instagram, captioning has emerged as a critical factor for accessibility, engagement, and visibility. While prior studies have examined different types of social media video captions or communities’ captioning usage, a systematic synthesis has not been undertaken, leading to the risk of proposing interventions that overlook core platform constraints or miss critical accessibility needs. This paper reviews 36 peer-reviewed papers published between 2015 and 2025 across fields such as Human-Computer Interaction (HCI), accessibility, media studies, education, and language learning. We note that captions operate as collective infrastructure co-produced by viewers, creators, and platforms. Deaf and Hard of Hearing (DHH), neurodivergent, and multilingual viewers depend on captions and increasingly expect mechanisms for feedback, while creators face inadequate tool support. Building on these insights, we propose the framework of Participatory Captioning and suggest design implications, highlighting future directions for social media video caption research. Huong Nguyen, Emma McDonnell, Lloyd May, Alexander Druzenko, Zoobia Saifullah Syeda, Mark Cartwright, Sooyeon Lee |
CHI | 7 |
| 2025 | Participant Recruitment in Accessibility ResearchabstractRecruiting participants from disability communities for accessibility research presents unique challenges that require careful consideration of ethical practices, intersectional representation, methodological rigor, and community sustainability.As accessibility research continues to grow and evolve, researchers face tensions between meaningfully including participants with disabilities and addressing emerging concerns around recruited participants not adequately representing the diversity of the community, overburdening certain participants, participant verification, and fair compensation practices.This workshop will bring together members of the ASSETS community to examine current recruiting practices and document insights into ethical, rigorous, and inclusive participant recruitment in disability research.Through facilitated discussions, we will explore three main themes: (1) methods and models, (2) eligibility criteria and participant verification, and (3) ethical and sustainability considerations.The workshop aims to share current practices, identify key challenges, and develop preliminary guidelines to support accessibility researchers in more sustainable participant recruitment. Lloyd May, Saad Hassan, Khang Dang, Sooyeon Lee, Oliver Alonzo |
ASSETS | 4 |
| 2025 | Beyond Visual Perception: Insights from Smartphone Interaction of Visually Impaired Users with Large Multimodal ModelsabstractLarge multimodal models (LMMs) have enabled new AI-powered applications that help people with visual impairments (PVI) receive natural language descriptions of their surroundings through audible text. We investigated how this emerging paradigm of visual assistance transforms how PVI perform and manage their daily tasks. Moving beyond basic usability assessments, we examined both the capabilities and limitations of LMM-based tools in personal and social contexts, while exploring design implications for their future development. Through interviews with 14 visually impaired users and analysis of image descriptions from both participants and social media using Be My AI (an LMM-based application), we identified two key limitations. First, these systems' context awareness suffers from hallucinations and misinterpretations of social contexts, styles, and human identities. Second, their intent-oriented capabilities often fail to grasp and act on users' intentions. Based on these findings, we propose design strategies for improving both human-AI and AI-AI interactions, contributing to the development of more effective, interactive, and personalized assistive technologies. Jingyi Xie 0001, Rui Yu 0002, He Zhang 0033, Syed Masum Billah, Sooyeon Lee, John M. Carroll 0001 |
CHI | 5 |
| 2024 | Design and Evaluation of an Automatic Text Simplification Prototype with Deaf and Hard-of-hearing ReadersabstractResearch has observed benefits from providing lexical and syntactic approaches to Automatic Text Simplification (ATS) to Deaf and Hard-of-hearing (DHH) readers. However, little research has explored DHH readers’ design preferences and interactions with these approaches. This work first explores the design space of ATS systems with DHH readers, identifying potential design configurations for evaluation. Open-ended discussion of participants’ design preferences reveal values informing those preferences, including maintaining reading fluency and efficiency, and control over the tool. Using popular design choices from our formative study, we evaluated a prototype that provides various simplification types to explore DHH readers’ interactions with the system. We observed potential conflicts between participants’ values and design preferences, such as the prototype’s impact on participants’ reading speed and participants’ perceived need to reread simplifications suggested by the tool. However, participants found the tool useful, showing a nuanced preference towards world-level lexical simplifications using pop-ups. Our findings highlight the importance of the tool’s design on users’ reading experiences, and provide implications for the design and evaluation of ATS prototypes with target readers. Oliver Alonzo, Sooyeon Lee, Akhter Al Amin, Mounica Maddela, Wei Xu 0004, Matt Huenerfauth |
ASSETS | 2 |
| 2024 | Towards Accessible Musical Performances in Virtual Reality: Designing a Conceptual Framework for Omnidirectional Audio DescriptionsabstractOur research focuses on making musical performance experience in virtual reality (VR) settings non-visually accessible for Blind and Low Vision (BLV) individuals by designing a conceptual framework for omnidirectional audio descriptions (AD). We address BLV users’ prevalent challenges in accessing effective AD during VR musical performances. Employing a two-phased interview methodology, we initially collected qualitative data about BLV AD users’ experiences, followed by gathering insights from BLV professionals who specialize in AD. This approach ensures that the developed solutions are both user-centric and practically feasible. The study devises strategies for three design concepts of omnidirectional AD (Spatial AD, View-dependent AD, and Explorative AD) tailored to different types of musical performances, which vary in their visual and auditory components. Each design concept offers unique benefits; collectively, they enhance accessibility and enjoyment for BLV audiences by addressing specific user needs. Key insights highlight the crucial role of flexibility and user control in AD implementation. Based on these insights, we propose a comprehensive conceptual framework to enhance musical experiences for BLV users within VR environments. Khang Dang, Grace Burke, Hamdi Korreshi, Sooyeon Lee |
ASSETS | 4 |
| 2024 | Musical Performances in Virtual Reality with Spatial and View-Dependent Audio Descriptions for Blind and Low-Vision UsersabstractVirtual reality (VR), inherently reliant on spatial interaction, poses significant accessibility barriers for individuals who are blind or have low vision (BLV). Traditional audio descriptions (AD) typically provide a verbal explanation of visual elements in 2D or flat video media, facilitating access for BLV audiences but failing to convey the complex spatial information essential in VR. This shortfall is especially pronounced in musical performances, where understanding the spatial arrangement of the stage setup and movements of performers is crucial. To overcome these limitations, we have developed two AD approaches—Spatial AD for a dance performance and View-dependent AD for an instrumental performance—within VR-based 360° environments. Spatial AD employs spatial audio technology to align descriptions with corresponding visuals, dynamically adjusting to follow the visuals, such as the movements of performers in the dance performance. Meanwhile, View-dependent AD adapts descriptions based on the orientation of the VR headset, activating when particular visuals enter the central view of the camera, ensuring that the description aligns with the user’s attention directed to a particular location within the VR environment. These methods are designed as enhancements to traditional AD, aiming to improve spatial orientation and immersive experiences for BLV audiences. This demonstration showcases the potential of these AD approaches to improve interaction and engagement, furthering the development of inclusive virtual environments. Khang Dang, Sooyeon Lee |
ASSETS | 2 |
| 2024 | Towards a Rich Format for Closed-CaptioningabstractClosed-captioning is an essential part of viewing audio-visual content for many people, including those who are D/deaf and Hard-of-Hearing. Traditional closed-captioning systems generally consist of a single track of timed text that offers limited options for personalization. Research into extending the capabilities of captioning, such as affective, poetic, and customizable captions has shown a desire among a subset of users for these features, but only in specific contexts. However, due to the difficulty in creating custom stimuli videos utilizing the custom captioning system, comparisons between systems and longitudinal studies have not been pursued. This demo paper introduces Rich Captions, a structured system that allows for a single closed-caption file to be tagged with additional information that can then be flexibly leveraged to render different customizable, creative, and poetic captions from the same file. Additionally, we introduce the Rich Caption Editor 1, a free, open-source software system designed to author, edit, and render rich captions. The system design was informed by a formative design workshop with closed-captioning researchers and advocates. The current design allows researchers to generate reproducible stimuli for closed-captioning studies. Once the design space and user preferences are better understood, the rich captioning framework could be refined to serve a general audience. Lloyd May, Alex C. Williams, Saad Hassan, Mark Cartwright, Sooyeon Lee |
ASSETS | 5 |
| 2024 | Unspoken Sound: Identifying Trends in Non-Speech Audio Captioning on YouTubeabstractHigh-quality closed captioning of both speech and non-speech elements (e.g., music, sound effects, manner of speaking, and speaker identification) is essential for the accessibility of video content, especially for d/Deaf and hard-of-hearing individuals. While many regions have regulations mandating captioning for television and movies, a regulatory gap remains for the vast amount of web-based video content, including the staggering 500+ hours uploaded to YouTube every minute. Advances in automatic speech recognition have bolstered the presence of captions on YouTube. However, the technology has notable limitations, including the omission of many non-speech elements, which are often crucial for understanding content narratives. This paper examines the contemporary and historical state of non-speech information (NSI) captioning on YouTube through the creation and exploratory analysis of a dataset of over 715k videos. We identify factors that influence NSI caption practices and suggest avenues for future research to enhance the accessibility of online video content. Lloyd May, Keita Ohshiro, Khang Dang, Sripathi Sridhar, Jhanvi Pai, Magdalena Fuentes, Sooyeon Lee, Mark Cartwright |
CHI | 7 |
| 2024 | BubbleCam: Engaging Privacy in Remote Sighted AssistanceabstractRemote sighted assistance (RSA) offers prosthetic support to people with visual impairments (PVI) through image- or video-based conversations with remote sighted assistants. While useful, RSA services introduce privacy concerns, as PVI may reveal private visual content inadvertently. Solutions have emerged to address these concerns on image-based asynchronous RSA, but exploration into solutions for video-based synchronous RSA remains limited. In this study, we developed BubbleCam, a high-fidelity prototype allowing PVI to conceal objects beyond a certain distance during RSA, granting them privacy control. Through an exploratory field study with 24 participants, we found that 22 appreciated the privacy enhancements offered by BubbleCam. The users gained autonomy, reducing embarrassment by concealing private items, messy areas, or bystanders, while assistants could avoid irrelevant content. Importantly, BubbleCam maintained RSA’s primary function without compromising privacy. Our study highlighted a cooperative approach to privacy preservation, transitioning the traditionally individual task of maintaining privacy into an interactive, engaging privacy preserving experience. Jingyi Xie 0001, Rui Yu 0002, He Zhang 0033, Sooyeon Lee, Syed Masum Billah, John M. Carroll 0001 |
CHI | 4 |
| 2023 | Are Two Heads Better than One? Investigating Remote Sighted Assistance with Paired VolunteersabstractRemote Sighted Assistance (RSA) is a popular smartphone-mediated aid for people with blindness, where a sighted individual converses with a blind individual in a one-on-one (1:1) session. Since sighted assistants outnumber blind individuals (13:1), this paper investigates what happens when more than one sighted individual assists a single blind individual in a session. Specifically, we propose paired-volunteer RSA, a new paradigm where two sighted volunteers assist a single user with blindness. We investigate the feasibility, desirability, and challenges of this paradigm and explore its opportunities. Our study with 8 sighted volunteers and 9 blind users reveals that the proposed paradigm extends the one-on-one RSA to cover a broader range of more intellectual and experiential tasks, providing new and distinctive opportunities in supporting complex, open-ended tasks (e.g., pursuing hobbies, appreciating arts, and seeking entertainment). These opportunities can not only enrich the blind users' quality of life and independence but also offer a fun and engaging experience for the sighted volunteers. The study also reveals the costs of extended collaboration in this paradigm. Finally, we synthesize a taxonomy of tasks where the proposed RSA paradigm can succeed and outline how HCI researchers and system designers can realize this paradigm. Jingyi Xie 0001, Rui Yu 0002, Kaiming Cui, Sooyeon Lee, John M. Carroll 0001, Syed Masum Billah |
Conference on Designing Interactive Systems | 4 |
| 2023 | Visualization of Speech Prosody and Emotion in Captions: Accessibility for Deaf and Hard-of-Hearing UsersabstractSpeech is expressive in ways that caption text does not capture, with emotion or emphasis information not conveyed. We interviewed eight Deaf and Hard-of-Hearing (dhh) individuals to understand if and how captions’ inexpressiveness impacts them in online meetings with hearing peers. Automatically captioned speech, we found, lacks affective depth, lending it a hard-to-parse ambiguity and general dullness. Interviewees regularly feel excluded, which some understand is an inherent quality of these types of meetings rather than a consequence of current caption text design. Next, we developed three novel captioning models that depicted, beyond words, features from prosody, emotions, and a mix of both. In an empirical study, 16 dhh participants compared these models with conventional captions. The emotion-based model outperformed traditional captions in depicting emotions and emphasis, with only a moderate loss in legibility, suggesting its potential as a more inclusive design for captions. Caluã de Lacerda Pataca, Matthew Watkins, Roshan Lalintha Peiris, Sooyeon Lee, Matt Huenerfauth |
CHI | 4 |
| 2022 | Helping Helpers: Supporting Volunteers in Remote Sighted Assistance with Augmented Reality Mapsabstract., agents, provide real-time assistance to blind users via video-chat-like communication. Prior work identified several challenges for the agents to provide navigational assistance to users and proposed computer vision-mediated RSA service to address those challenges. We present an interactive system implementing a high-fidelity prototype of RSA service using augmented reality (AR) maps with localization and virtual elements placement capabilities. The paper also presents a confederate-based study design to evaluate the effects of AR maps with 13 untrained agents. The study revealed that, compared to baseline RSA, agents were significantly faster in providing indoor navigational assistance to a confederate playing the role of users, and agents' mental workload was significantly reduced-all indicate the feasibility and scalability of AR maps in RSA services. Jingyi Xie 0001, Rui Yu 0002, Sooyeon Lee, Yao Lyu, Syed Masum Billah, John M. Carroll 0001 |
Conference on Designing Interactive Systems | 3 |
| 2022 | Support in the Moment: Benefits and use of video-span selection and search for sign-language video comprehension among ASL learnersabstractAs they develop comprehension skills, American Sign Language (ASL) learners often view challenging ASL videos, which may contain unfamiliar signs. Current dictionary tools require students to isolate a single sign they do not understand and input a search query, by selecting linguistic properties or by performing the sign into a webcam. Students may struggle with extracting and re-creating an unfamiliar sign, and they must leave the video-watching task to use an external dictionary tool. We investigate a technology that enables users, in the moment, i.e., while they are viewing a video, to select a span of one or more signs that they do not understand, to view dictionary results. We interviewed 14 American Sign Language (ASL) learners about their challenges in understanding ASL video and workarounds for unfamiliar vocabulary. We then conducted a comparative study and an in-depth analysis with 15 ASL learners to investigate the benefits of using video sub-spans for searching, and their interactions with a Wizard-of-Oz prototype during a video-comprehension task. Our findings revealed benefits of our tool in terms of quality of video translation produced and perceived workload to produce translations. Our in-depth analysis also revealed benefits of an integrated search tool and use of span-selection to constrain video play. These findings inform future designers of such systems, computer vision researchers working on the underlying sign matching technologies, and sign language educators. Saad Hassan, Akhter Al Amin, Caluã de Lacerda Pataca, Diego Navarro, Alexis Gordon, Sooyeon Lee, Matt Huenerfauth |
ASSETS | 6 |
| 2022 | Understanding ASL Learners' Preferences for a Sign Language Recording and Automatic Feedback System to Support Self-StudyabstractAdvancements in AI will soon enable tools for providing automatic feedback to American Sign Language (ASL) learners on some aspects of their signing, but there is a need to understand their preferences for submitting videos and receiving feedback. Ten participants in our study were asked to record a few sentences in ASL using software we designed, and we provided manually curated feedback on one sentence in a manner that simulates the output of a future automatic feedback system. Participants responded to interview questions and a questionnaire eliciting their impressions of the prototype. Our initial findings provide guidance to future designers of automatic feedback systems for ASL learners. Saad Hassan, Sooyeon Lee, Dimitris N. Metaxas, Carol Neidle, Matt Huenerfauth |
ASSETS | 2 |
| 2022 | Methods for Evaluating the Fluency of Automatically Simplified Texts with Deaf and Hard-of-Hearing Adults at Various Literacy LevelsabstractResearch has revealed benefits and interest among Deaf and Hard-of-Hearing (DHH) adults in reading-assistance tools powered by Automatic Text Simplification (ATS), a technology whose development benefits from evaluations by specific user groups. While prior work has provided guidance for evaluating text complexity among DHH adults, researchers lack guidance for evaluating the fluency of automatically simplified texts, which may contain errors from the simplification process. Thus, we conduct methodological research on the effectiveness of metrics (including reading speed; comprehension questions; and subjective judgements of understandability, readability, grammaticality, and system performance) for evaluating texts controlled to be at different levels of fluency, when measured among DHH participants at different literacy levels. Reading speed and grammaticality judgements effectively distinguished fluency levels among participants across literacy levels. Readability and understandability judgements, however, only worked among participants with higher literacy. Our findings provide methodological guidance for designing ATS evaluations with DHH participants. Oliver Alonzo, Jessica Trussell, Matthew Watkins, Sooyeon Lee, Matt Huenerfauth |
CHI | 4 |
| 2022 | Watch It, Don't Imagine It: Creating a Better Caption-Occlusion Metric by Collecting More Ecologically Valid Judgments from DHH ViewersabstractTelevision captions blocking visual information causes dissatisfaction among Deaf and Hard of Hearing (DHH) viewers, yet existing caption evaluation metrics do not consider occlusion. To create such a metric, DHH participants in a recent study imagined how bad it would be if captions blocked various on-screen text or visual content. To gather more ecologically valid data for creating an improved metric, we asked 24 DHH participants to give subjective judgments of caption quality after actually watching videos, and a regression analysis revealed which on-screen contents’ occlusion related to users’ judgments. For several video genres, a metric based on our new dataset out-performed the prior state-of-the-art metric for predicting the severity of captions occluding content during videos, which had been based on that prior study. We contribute empirical findings for improving DHH viewers’ experience, guiding the placement of captions to minimize occlusions, and automated evaluation of captioning quality in television broadcasts. Akhter Al Amin, Saad Hassan, Sooyeon Lee, Matt Huenerfauth |
CHI | 3 |
| 2022 | Analyzing Deaf and Hard-of-Hearing Users' Behavior, Usage, and Interaction with a Personal Assistant Device that Understands Sign-Language InputabstractAs voice-based personal assistant technologies proliferate, e.g., smart speakers in homes, and more generally as voice-control of technology becomes increasingly ubiquitous, new accessibility barriers are emerging for many Deaf and Hard of Hearing (DHH) users. Progress in sign-language recognition may enable devices to respond to sign-language commands and potentially mitigate these barriers, but research is needed to understand how DHH users would interact with these devices and what commands they would issue. In this work, we directly engage with the DHH community, using a Wizard-of-Oz prototype that appears to understand American Sign Language (ASL) commands. Our analysis of video recordings of DHH participants revealed how they woke-up the device to initiate commands, structured commands in ASL, and responded to device errors, providing guidance to future designers and researchers. We share our dataset of over 1400 commands, which may be of interest to sign-language-recognition researchers. Abraham Glasser, Matthew Watkins, Kira Hart, Sooyeon Lee, Matt Huenerfauth |
CHI | 4 |
| 2022 | Design and Evaluation of Hybrid Search for American Sign Language to English Dictionaries: Making the Most of Imperfect Sign RecognitionabstractSearching for the meaning of an unfamiliar sign-language word in a dictionary is difficult for learners, but emerging sign-recognition technology will soon enable users to search by submitting a video of themselves performing the word they recall. However, sign-recognition technology is imperfect, and users may need to search through a long list of possible results when seeking a desired result. To speed this search, we present a hybrid-search approach, in which users begin with a video-based query and then filter the search results by linguistic properties, e.g., handshape. We interviewed 32 ASL learners about their preferences for the content and appearance of the search-results page and filtering criteria. A between-subjects experiment with 20 ASL learners revealed that our hybrid search system outperformed a video-based search system along multiple satisfaction and performance metrics. Our findings provide guidance for designers of video-based sign-language dictionary search systems, with implications for other search scenarios. Saad Hassan, Akhter Al Amin, Alexis Gordon, Sooyeon Lee, Matt Huenerfauth |
CHI | 4 |
| 2022 | Remotely Co-Designing Features for Communication Applications using Automatic Captioning with Deaf and Hearing PairsabstractDeaf and Hard-of-Hearing (DHH) users face accessibility challenges during in-person and remote meetings. While emerging use of applications incorporating automatic speech recognition (ASR) is promising, more user-interface and user-experience research is needed. While co-design methods could elucidate designs for such applications, COVID-19 has interrupted in-person research. This study describes a novel methodology for conducting online co-design workshops with 18 DHH and hearing participant pairs to investigate ASR-supported mobile and videoconferencing technologies along two design dimensions: Correcting errors in ASR output and implementing notification systems for influencing speaker behaviors. Our methodological findings include an analysis of communication modalities and strategies participants used, use of an online collaborative whiteboarding tool, and how participants reconciled differences in ideas. Finally, we present guidelines for researchers interested in online DHH co-design methodologies, enabling greater geographically diversity among study participants even beyond the current pandemic. Matthew Seita, Sooyeon Lee, Sarah Andrew, Kristen Shinohara, Matt Huenerfauth |
CHI | 2 |
| 2022 | Opportunities for Human-AI Collaboration in Remote Sighted AssistanceabstractRemote sighted assistance (RSA) has emerged as a conversational assistive technology for people with visual impairments (VI), where remote sighted agents provide realtime navigational assistance to users with visual impairments via video-chat-like communication. In this paper, we conducted a literature review and interviewed 12 RSA users to comprehensively understand technical and navigational challenges in RSA for both the agents and users. Technical challenges are organized into four categories: agents' difficulties in orienting and localizing the users; acquiring the users' surroundings and detecting obstacles; delivering information and understanding user-specific situations; and coping with a poor network connection. Navigational challenges are presented in 15 real-world scenarios (8 outdoor, 7 indoor) for the users. Prior work indicates that computer vision (CV) technologies, especially interactive 3D maps and realtime localization, can address a subset of these challenges. However, we argue that addressing the full spectrum of these challenges warrants new development in Human-CV collaboration, which we formalize as five emerging problems: making object recognition and obstacle avoidance algorithms blind-aware; localizing users under poor networks; recognizing digital content on LCD screens; recognizing texts on irregular surfaces; and predicting the trajectory of out-of-frame pedestrians or objects. Addressing these problems can advance computer vision research and usher into the next generation of RSA service. Sooyeon Lee, Rui Yu 0002, Jingyi Xie 0001, Syed Masum Billah, John M. Carroll 0001 |
IUI | 1 |
| 2022 | Can Haptic Feedback on One Virtual Object Increase the Presence of Another Virtual Object?abstractThis paper investigated whether increased presence from experiencing haptic feedback on one virtual object can transfer to another virtual object. Two similar studies were run in different environments: an immersive virtual environment and a mixed environment. Results showed that participants reported a high presence of untouched virtual object after touching a virtual object in a virtual reality environment. On the other hand, it was difficult to confirm that such presence transfers occurred in an augmented reality environment. Sooyeon Lee, Myungho Lee |
VRST | 1 |
| 2022 | Iterative Design and Prototyping of Computer Vision Mediated Remote Sighted AssistanceabstractRemote sighted assistance (RSA) is an emerging navigational aid for people with visual impairments (PVI). Using scenario-based design to illustrate our ideas, we developed a prototype showcasing potential applications for computer vision to support RSA interactions. We reviewed the prototype demonstrating real-world navigation scenarios with an RSA expert, and then iteratively refined the prototype based on feedback. We reviewed the refined prototype with 12 RSA professionals to evaluate the desirability and feasibility of the prototyped computer vision concepts. The RSA expert and professionals were engaged by, and reacted insightfully and constructively to the proposed design ideas. We discuss what we learned about key resources, goals, and challenges of the RSA prosthetic practice through our iterative prototype review, as well as implications for the design of RSA systems and the integration of computer vision technologies into RSA. Jingyi Xie 0001, Madison Reddie, Sooyeon Lee, Syed Masum Billah, Zihan Zhou 0001, Chun-Hua Tsai, John M. Carroll 0001 |
ACM Trans. Comput. Hum. Interact. | 3 |
| 2021 | At a Different Pace: Evaluating Whether Users Prefer Timing Parameters in American Sign Language Animations to Differ from Human Signers' TimingabstractAdding American Sign Language (ASL) versions of information content to websites can improve information accessibility for many people who are Deaf or Hard of Hearing (DHH) who may have lower levels of English literacy. Generating animations from a script representation would enable this content to be easily updated, yet software is needed that can set detailed speed and timing parameters for such animations, which prior work has revealed to be critical for their understandability and acceptance among DHH users. Despite recent work on predicting these parameters using AI models trained on recordings of human signers, no prior work had examined whether DHH users actually prefer for these speed and timing properties to be similar to humans, or to be exaggerated, e.g. for additional clarity. We conducted two empirical studies to investigate preferences of ASL signers for speed and timing parameters of ASL animations, including: sign duration, transition time, differential signing rate, pause length, and pausing frequency. Our first study (N=20) identified two preferred values from among five options for each parameter, one of which included a typical human value for this parameter, and a second study (N=20) identified the most preferred value. We found that while ASL signers preferred pause length and frequency to be similar to those of humans, they actually preferred animations to have faster signs, slower transitions, and less dynamic variation in differential signing speed, as compared to the timing of human signers. This study provides specific empirical guidance for creators of future ASL animation technologies, and more broadly, it demonstrates that it is not safe to assume that ASL signers will simply prefer for properties of ASL animations to be as similar as possible to human signers. Sedeeq Al-khazraji, Becca Dingman, Sooyeon Lee, Matt Huenerfauth |
ASSETS | 3 |
| 2021 | American Sign Language Video Anonymization to Support Online Participation of Deaf and Hard of Hearing UsersabstractWithout a commonly accepted writing system for American Sign Language (ASL), Deaf or Hard of Hearing (DHH) ASL signers who wish to express opinions or ask questions online must post a video of their signing, if they prefer not to use written English, a language in which they may feel less proficient. Since the face conveys essential linguistic meaning, the face cannot simply be removed from the video in order to preserve anonymity. Thus, DHH ASL signers cannot easily discuss sensitive, personal, or controversial topics in their primary language, limiting engagement in online debate or inquiries about health or legal issues. We explored several recent attempts to address this problem through development of “face swap” technologies to automatically disguise the face in videos while preserving essential facial expressions and natural human appearance. We presented several prototypes to DHH ASL signers (N=16) and examined their interests in and requirements for such technology. After viewing transformed videos of other signers and of themselves, participants evaluated the understandability, naturalness of appearance, and degree of anonymity protection of these technologies. Our study revealed users’ perception of key trade-offs among these three dimensions, factors that contribute to each, and their views on transformation options enabled by this technology, for use in various contexts. Our findings guide future designers of this technology and inform selection of applications and design features. Sooyeon Lee, Abraham Glasser, Becca Dingman, Zhaoyang Xia, Dimitris N. Metaxas, Carol Neidle, Matt Huenerfauth |
ASSETS | 1 |
| 2021 | Lightweight extension of an execution environment for safer function calls in Solidity/Ethereum Virtual Machine smart contractsabstractSolidity, a programming language used to write smart contracts, has been improved since its initial release, but a number of vulnerabilities remain. As smart contracts are usually related to cryptocurrency, these vulnerabilities should be avoided to prevent the risk of financial loss. In this paper, we classify common vulnerabilities of function calls of Solidity programs into three groups and suggest a method to avoid them. The proposed method makes use of Ethereum Virtual Machine as well as Solidity extension. Experimental results with real-world smart contracts show that our method will detect and avoid these vulnerabilities. Sooyeon Lee, Eun-Sun Cho |
SANER | 1 |
| 2021 | Designing for Independence for People with Visual ImpairmentsabstractThe notion of "independence'' is frequently used to motivate technology design in the HCI sub-field of accessible and assistive technology for people with disabilities. Despite the term's pervasive use, the literature lacks a recent articulation of its meaning in the context of assistive technology. What does independence really mean? Scoping the study to people with visual impairments (PVI), we posed this question to 10 individuals from diverse backgrounds through in-depth, semi-structured interviews. Our findings reveal that PVI have internal experiences of independence and social experiences of independence, with surprising insights into the roles that people and technology play in supporting independence. We also discuss new ways to design for independence. Sooyeon Lee, Madison Reddie, John M. Carroll 0001 |
Proc. ACM Hum. Comput. Interact. | 1 |
| 2020 | AIGuide: An Augmented Reality Hand Guidance Application for People with Visual ImpairmentsabstractLocating and grasping objects is a critical task in people’s daily lives. For people with visual impairments, this task can be a daily struggle. The support of augmented reality frameworks in smartphones has the potential to overcome the limitations of current object detection applications designed for people with visual impairments. We present AIGuide, a self-contained offline smartphone application that leverages augmented reality technology to help users locate and pick up objects around them. We conducted a user study to validate its effectiveness at providing guidance, compare it to other assistive technology form factors, evaluate the use of multimodal feedback, and provide feedback about the overall experience. Our results show that AIGuide is a promising technology to help people with visual impairments locate and acquire objects in their daily routine. Nelson Daniel Troncoso Aldas, Sooyeon Lee, Chonghan Lee, Mary Beth Rosson, John M. Carroll 0001, Narayanan Vijaykrishnan |
ASSETS | 2 |
| 2020 | The Emerging Professional Practice of Remote Sighted Assistance for People with Visual ImpairmentsabstractPeople with visual impairments (PVI) must interact with a world they cannot see. Remote sighted assistance (RSA) has emerged as a conversational assistive technology. We interviewed RSA assistants ("agents") who provide assistance to PVI via a conversational prosthetic called Aira (https://aira.io/) to understand their professional practice. We identified four types of support provided: scene description, navigation, task performance, and social engagement. We discovered that RSA provides an opportunity for PVI to appropriate the system as a richer conversational/social support tool. We studied and identified patterns in how agents provide assistance and how they interact with PVI as well as the challenges and strategies associated with each context. We found that conversational interaction is highly context-dependent. We also discuss implications for design. Sooyeon Lee, Madison Reddie, Chun-Hua Tsai, Jordan Beck, Mary Beth Rosson, John M. Carroll 0001 |
CHI | 1 |
| 2019 | Understanding and Designing for Deaf or Hard of Hearing Drivers on UberabstractWe used content analysis of in-app driver survey responses, customer support tickets, and tweets, and face-to-face interviews of DHH Uber drivers to better understand the DHH driver experience. Here we describe challenges DHH drivers experience and how they address those difficulties via Uber's accessibility features and their own workarounds. We also identify and discuss design and product opportunities to improve the DHH driver experience on Uber. Sooyeon Lee, Bjorn Hubert-Wallander, Molly Stevens, John M. Carroll 0001 |
CHI | 1 |
| 2019 | A Modified Smart Contract Execution Enviroment for Safe Function CallsabstractWhen a Solidity smart contract has a problem in calling a function of another contract, the "fallback function" of the contract is supposed to be executed automatically. However, in many cases, a fallback function is arbitrarily created and called, with their behaviors unknown to developers, so that its execution is vulnerable to exploits by attackers. To reduce these risks, this paper proposes a method that provides developers with new keywords by modifying existing Solidity compiler and Ethereum Virtual Machine (EVM). Developers mark their intention using the newly introduced keywords, and the modified existing Solidity compiler and EVM uses flags and conditional statements to prevent calls of fallback functions to reduce the risk of calls to fallback functions. Sooyeon Lee, Eun-Sun Cho |
COMPSAC (1) | 1 |
| 2018 | OneNote Meal: A Photo-Based Food Diary Study for Reflective Meal Tracking
Johnna Blair, Yuhan Luo 0002, Ning F. Ma, Sooyeon Lee, Eun Kyoung Choe |
AMIA | 4 |
| 2017 | Reaching Out: Investigating Different Modalities to Help People with Visual Impairments Acquire ItemsabstractWe present a lab study of multiple feedback designs for guiding small-scale arm-and-hand movement for people with visual impairments (PVI), so that they can reach out to and grasp an item on a shelf. Little attention has been paid to the guidance of small-scale arm-and-hand movements by PVI, yet this is an essential element of product acquisition in a grocery shopping task and other similar daily activities. We developed a feedback interface that allowed us to explore two types of auditory feedback (speech and tones), haptic vibration feedback, and a combination of both. The result of the study demonstrated that the multi-modal navigational feedback, specifically speech and haptic, was the most effective and preferred mode for small-scale navigation. Sooyeon Lee, Tina Chien-Wen Yuan, Benjamin V. Hanrahan, Mary Beth Rosson, John M. Carroll 0001 |
ASSETS | 1 |
| 2017 | I Didn't Know that You Knew I Knew: Collaborative Shopping Practices between People with Visual Impairment and People with VisionabstractIt is important to support independent living for people with visual impairments (PVI). Part of this can be accomplished with individual assistive technologies. However, in this paper we emphasize the social and collaborative needs for PVI to fully integrate into society as equals. The study assesses how PVI collaborate with different types of sighted partners when shopping together. We chose to study grocery shopping because it is a critical and challenging task for PVI. We conducted field observations and in-depth interviews with five PVI and their sighted shopping partners, including spouses, caseworkers, and store-provided courtesy shoppers. We found several factors that modulated these collaborations with varying forms of common ground: 1) knowledge about how to assist PVI; 2) interpersonal knowledge resulting from common experience and interpersonal relationship history; and 3) knowledge of shopping as a practice. We discuss our findings with respect to the implications for designing collaborative interactions. Tina Chien-Wen Yuan, Benjamin V. Hanrahan, Sooyeon Lee, Mary Beth Rosson, John M. Carroll 0001 |
Proc. ACM Hum. Comput. Interact. | 3 |
| 2016 | 'MASTerful' Matchmaking in Service Transactions: Inferred Abilities, Needs and Interests versus Activity HistoriesabstractTimebanking is a growing type of peer-to-peer service exchange, but is hampered by the effort of finding good transaction partners. We seek to reduce this effort by using a Matching Algorithm for Service Transactions (MAST). MAST matches transaction partners in terms of similarity of interests and complementarity of abilities and needs. We present an experiment involving data and participants from a real timebanking network, that evaluates the acceptability of MAST, and shows that such an algorithm can retrieve matches that are subjectively better than matches based on matching the category of people's historical offers or requests to the category of a current transaction request. Hyunggu Jung, Victoria Bellotti, Afsaneh Doryab, Dean Leitersdorf, Jiawei Chen 0003, Benjamin V. Hanrahan, Sooyeon Lee, Daniel Turner, Anind K. Dey, John M. Carroll 0001 |
CHI | 7 |