Social play is an essential pathway for emotional, cognitive, and social development in children. However, Deaf and Hard of Hearing (DHH) children often experience barriers to social play, namely in mixed-hearing ability environments (e.g., school playground). In this paper, we conducted interviews with six educators and 19 children with and without hearing loss at a Partially Bilingual School, to better understand their experiences during social play. Moreover, we observed a school playground with 46 children over seven weeks at a Full Bilingual School. Findings show that social play between DHH and hearing children is influenced by school culture, peer culture, and child agency. Importantly, some of these barriers can be (partially) overcome through a supportive bilingual and bicultural environment. We propose the concept of contextualized social play technology, which defines a design space aimed at fostering peer culture and individual agency through contextualization within schools. We also provide design insights to inform the development of future inclusive play technologies.

Figure 1: Left: Creat'AI, a tangible storytelling prototype designed to teach AI concepts to young children.It includes a keyboard, mirror, external speaker, external enter button, and tangible elements such as QR code cards, paper, and pens.Right: Drawings created by children to interact with the AI and contribute to the storytelling process, depicting a soccer game, a strawberry, and a slice of pizza.

Access to public spaces is of the utmost importance for social cohesion, inclusion, and civic engagement.Nevertheless, a large majority of public spaces remain incredibly uncomfortable environments for neurodivergent individuals due to, for instance, the unpredictability of such spaces and the sensory stimuli within them.Smart City technologies present an exciting opportunity to improve the accessibility and enjoyment of the spaces where they are deployed by, for instance, offering users the ability to customise a space to their specific sensory needs.However, the research topic of public space technologies for neurodivergent individuals remains scattered and sparsely documented.This critical review analyses the existing domains of inquiry, contributing a theoretical framework based on Spatial Justice and Neuroqueer Technoscience and suggests future research avenues informed by this framework.We advocate for the participatory co-creation of a neurodivergent-affirming landscape of public space technologies that both support neurodivergent needs and promote neurodivergent joy.

There is an increasing interest in Human–computer Interaction in multisensory interactive systems, creating a need to deepen our understanding of how multiple sensory modalities relate to and affect each other. We extend prior research on crossmodal correspondences by investigating colour and emotional associations with pure vibrotactile stimuli. We asked 32 participants to assign colour properties (hue and brightness) and emotional categories (pleasure, arousal, and dominance) to stimuli of varying amplitude and angular frequency . We found that perceptions of pleasure and arousal increased with amplitude and angular frequency, but negative and relaxing emotional states were not able to convey. We also found associations between vibrotactile stimuli and colour properties. High amplitude stimuli were linked to warm colours and darker shades, while low amplitude was associated with brighter shades and greater variance in hue. We finish by discussing the causal mechanisms of crossmodal correspondences and contribute a design space for creating multisensory experiences.

Multidisciplinary teams are crucial in tailoring cancer care through collaborative decision-making involving several clinical specialties. The inherent complexity of clinical cases, the increasing abundance of unstructured textual data, and the time restrictions of professionals pose significant challenges to team coordination and patient care. This creates an opportunity for generative AI technologies, such as LLMs, to enhance collaborative work. Despite the growing interest in HCI research to explore LLMs in healthcare, we have yet to understand clinicians' perspectives on this emerging technology in multidisciplinary teams. Our work investigates the challenges, expectations and opportunities for LLMs in this context through a speculative approach. We leveraged the Futures Cone framework and conducted a qualitative study with 11 physicians from different cancer multidisciplinary teams. We contribute with an analysis of themes that emerged from individual interviews and a focus group, highlighting LLMs' potential to enhance and reshape multidisciplinary teams' practices. In addition, we uncover concerns and coping strategies related to LLMs' adoption and provide a set of design opportunities to inform the development of technologies for LLM-enhanced multidisciplinary teams.

Introductory coding environments have been used in early education to promote computational thinking, supporting the development of cognitive, critical, and social skills. Many environments focus on individual use, which has limited benefits compared to collaborative learning. In this paper, we present the results of a 10-session study at a local primary school engaging eleven children with visual impairments and three inclusive education teachers in collaborative programming activities. Based on participants' behavior, reactions, and feedback, we contribute an improved understanding of collaborative design in educational settings, focusing on the impact of Goals, Workspace, Interdependence, and Shared Awareness. Our main findings outline how collaboration dynamics can be shaped by asymmetric tasks, workspace proximity, and group awareness. We further discuss factors that led to a lack of investment in the shared goal and instances of unbalanced collaboration, reflecting on challenges and opportunities for designing collaborative inclusive coding kits.

Stroke survivors experience a wide range of sequelae that significantly change their lives and those of their caregivers. These changes affect relationships, professional life, and daily routines, leading to emotional distress for both. However, existing research in HCI has predominantly focused on physical and cognitive rehabilitation, neglecting the impact of stroke on emotional wellbeing. Our work explores the potential of social sensemaking technologies to support wellbeing awareness and reflection of stroke survivors and their caregivers. We developed Emotions2Us, an interactive ambient display designed to foster interdependent communication, expression, support, and reflection on emotional wellbeing among dyads. We contribute with qualitative findings from a four-week field deployment (N=8). We present an analysis of the themes that emerged from interviews with stroke survivors and caregivers, highlighting the impact of Emotions2Us on wellbeing practices and reflecting on the challenges, benefits, and opportunities of designing for social sensemaking following a stroke.

This work-in-progress presents ALMA, an innovative prototype for storytelling with a smart soft toy inspired by Snoezelen principles. Its objective is to improve children’s emotion regulation while facilitating children’s exploration of sensory perceptions, emotion labeling, and self-reflection. While current methods in Child-Computer Interaction (CCI) frequently emphasize individual aspects like storytelling or multisensory experiences, there is a gap in interactive storytelling incorporating soft toys that integrate multisensory and Snoezelen principles, despite the well-documented advantages of such integration. By leveraging the synergies between multisensory experiences and storytelling, ALMA seeks to foster children’s emotion regulation and, therefore a holistic development.

This paper introduces a hands-on workshop centered on participatory design (PD) approaches tailored for engaging young children, with a special focus on failures, challenges, and successes in prior experiences within the child-computer interaction (CCI) domain. Although previous efforts have highlighted the advantages of engaging young children in PD, research has overlooked their involvement as co-designers, leading to a lack of exploration and understanding of their unique perspectives and challenges in the design process. Through an interactive session and collaborative activities, this workshop will facilitate discussions surrounding challenges, successes, and lessons learned through PD with young children. By evaluating and exchanging experiences, we aim to enhance our understanding of PD and refine its methodologies for this particular population. By synthesizing the shortcomings, difficulties, and successes of past experiences, the workshop will bring together researchers and practitioners to initiate efforts toward closing this research gap. Together we will establish the groundwork for enhanced approaches and a deeper understanding of how to involve young children in PD, which will enhance future efforts in the field of CCI.

Accessibility research has gained traction, yet ethical gaps persist in the inclusion of individuals with disabilities, especially children. Inclusive research practices are essential to ensure that research and design solutions cater to the needs of all individuals, regardless of their abilities. Working with children with disabilities in Human-Computer Interaction and Human-Robot Interaction presents a unique set of ethical dilemmas. These young participants often require additional care, support, and accommodations, which can fall off researchers’ resources or expertise. The lack of clear guidance on navigating these challenges further aggravates the problem. To provide a basis on which to address this issue, we adopt a critical reflective approach, evaluating our impact by analyzing two case studies involving children with disabilities in HCI/HRI research. Flowing from these, we call for a shift in our approach to ethics in participatory research contexts to one that is processual, situational, and community-led.

Neurodivergent children spend most of their time in neurodiverse schools alongside their neurotypical peers and often face social exclusion. Inclusive play activities are a strong vehicle of inclusion. Unfortunately, games designed for the specific needs of neurodiverse groups are scarce. Given the potential of robots to support play, we led a co-design process to build an inclusive robotic game for neurodiverse classrooms. We conducted five co-design workshops, engaging 80 children from neurodiverse classrooms in designing an inclusive game. Employing the resulting design insights, we iteratively prototyped and playtested a tabletop robotic game leveraging off-the-shelf robots. Reflecting upon our findings, we discuss how the longitudinal co-design process (rather than the resulting game) was key in allowing children the space to learn how to accommodate accessibility needs and create inclusive play experiences. We posit the use of co-design to enhance children’s interpersonal relationships, fosters feelings of ownership, and encourages appropriation practices as a strategy to sustain inclusive experiences that extend beyond project timelines or artefact designs.

Shape-changing skin is an exciting modality due to its accessible and engaging nature. Its softness and flexibility make it adaptable to different interactive devices that children with and without visual impairments can share. Although their potential as an emotionally expressive medium has been shown for sighted adults, their potential as an inclusive modality remains unexplored. This work explores the shape-emotional mappings in children with and without visual impairment. We conducted a user study with 50 children (26 with visual impairment) to investigate their emotional associations with five skin shapes and two movement conditions. Results show that shape-emotional mappings are dependent on visual abilities. Our study raises awareness of the influence of visual experiences on tactile vocabulary and emotional mapping among sighted, low-vision, and blind children. We finish discussing the causal associations between tactile stimuli and emotions and suggest inclusive design recommendations for shape-changing devices.

Research on robotic ostracism is still scarce and has only explored its effects on adult populations. Although the results revealed important carryover effects of robotic exclusion, there is no evidence yet that those results occur in child-robot interactions. This paper provides the first exploration of robotic ostracism with children. We conducted a study using the Robotic Cyberball Paradigm in a third-person perspective with a sample of 52 children aged between five to ten years old. The experimental study had two conditions: Exclusion and Inclusion. In the Exclusion condition, children observed a peer being excluded by two robots; while in the Inclusion condition, the observed peer interacted equally with the robots. Notably, even 5-year-old children could discern when robots excluded another child. Children who observed exclusion reported lower levels of belonging and control, and exhibited higher prosocial behaviour than those witnessing inclusion. However, no differences were found in children's meaningful existence, self-esteem, and physical proximity across conditions. Our user study provides important methodological considerations for applying the Robotic Cyberball Paradigm with children. The results extend previous literature on both robotic ostracism with adults and interpersonal ostracism with children. We finish discussing the broader implications of children observing ostracism in human-robot interactions.

Children with visual impairments often struggle to fully participate in group activities due to limited access to visual cues. They have difficulty perceiving what is happening, when, and how to act-leading to children with and without visual impairments being frustrated with the group activity, reducing mutual interactions. To address this, we created Touchibo, a tactile storyteller robot acting in a multisensory setting, encouraging touch-based interactions. Touchibo provides an inclusive space for group interaction as touch is a highly accessible modality in a mixed-visual ability context. In a study involving 107 children (37 with visual impairments), we compared Touchibo to an audio-only storyteller. Results indicate that Touchibo significantly improved children's individual and group participation perception, sparking touch-based interactions and the storyteller was more likable and helpful. Our study highlights touch-based robots' potential to enrich children's social interactions by prompting interpersonal touch, particularly in mixed-visual ability settings.

Playful robotics engages children in learning through play experiences while simultaneously developing critical thinking, and social, cognitive, and motor skills through play. Such playful experiences are particularly valuable in inclusive education to promote social and inclusive behaviors. We present TACTOPI, an inclusive and playful multisensory environment that leverages tangible interaction and a robot as the main character. We investigate how TACTOPI supports play in 10 dyads of children with mixed visual abilities. Results show that multisensory elements supported children to experience activities as joyful. Storytelling and guided-play added a layer of meaningfulness to the activities, and the robot engaged children in minds-on thinking. TACTOPI afforded children to engage in collaborative social play and facilitated supportive and inclusive behaviours. We contribute with a playful multisensory environment, an analysis of the effect of its components on social, cognitive, and inclusive play, and design considerations for inclusive multisensory environments that prioritize play.

Play is a central aspect of childhood development, with games as a vital tool to promote it. However, neurodivergent children, especially those in neurodiverse environments, are underserved by HCI games research. Most existing work takes on a top-down approach, disregarding neurodivergent interest for the majority of the design process. Co-design is often proposed as a tool to create truly accessible and inclusive gaming experiences. Nevertheless, co-designing with neurodivergent children within neurodiverse groups brings about unique challenges, such as different communication styles, sensory needs and preferences. Building upon recommendations from prior work in neurodivergent, mixed-ability, and child-led co-design, we propose a concrete participatory game design kit for neurodiverse classrooms: PartiPlay. Moreover, we present preliminary findings from an in-the-wild experiment with the said kit, showcasing its ability to create an inclusive co-design process for neurodiverse groups of children. We aim to provide actionable steps for future participatory design research with neurodiverse children.

Collaborative coding environments foster learning, social skills, computational thinking training, and supportive relationships. In the context of inclusive education, these environments have the potential to promote inclusive learning activities for children with mixed-visual abilities. However, there is limited research focusing on remote collaborative environments, despite the opportunity to design new modes of access and control of content to promote more equitable learning experiences. We investigated the tradeoffs between remote and co-located collaboration through a tangible coding kit. We asked ten pairs of mixed-visual ability children to collaborate in an interdependent and asymmetric coding game. We contribute insights on six dimensions - effectiveness, computational thinking, accessibility, communication, cooperation, and engagement - and reflect on differences, challenges, and advantages between collaborative settings related to communication, workspace awareness, and computational thinking training. Lastly, we discuss design opportunities of tangibles, audio, roles, and tasks to create inclusive learning activities in remote and co-located settings.

Inclusion is key in group work and collaborative learning. We developed a mediator robot to support and promote inclusion in group conversations, particularly in groups composed of children with and without visual impairment. We investigate the effect of two mediation strategies on group dynamics, inclusion, and perception of the robot. We conducted a within-subjects study with 78 children, 26 experienced visual impairments, in a decision-making activity. Results indicate that the robot can foster inclusion in mixed-visual ability group conversations. The robot succeeds in balancing participation, particularly when using a highly intervening mediating strategy (directive strategy). However, children feel more heard by their peers when the robot is less intervening (organic strategy). We extend prior work on social robots to assist group work and contribute with a mediator robot that enables children with visual impairments to engage equally in group conversations. We finish by discussing design implications for inclusive social robots.

Current signing avatars are often described as unnatural as they cannot accurately reproduce all the subtleties of synchronized body behaviors of a human signer. In this paper, we investigate a new dynamic approach for transitions between signs and the effect of mouthing behaviors. Although native signers preferred animations with dynamic transitions, we did not find significant differences in comprehension and perceived naturalness scores. On the other hand, we show that including mouthing behaviors improved comprehension and perceived naturalness for novice Portuguese sign language learners.

Abstract Transitioning to and through University is a delicate period for students’ well-being. Moreover, the recent COVID-19 pandemic added a further toll through the various challenges related to studying, socializing, community-building, and safety. These challenges inspired the design of a mobile application, called Tecnico GO!, to support university students’ well-being and academic performance. This paper presents the design rationale and evaluation of the app conducted during the academic year 2021-2022. Findings cluster around three themes: i) students studying needs; ii) building a sense of community; iii) gamification strategies. The discussion elaborates on the student’s perceptions of well-being during pandemics. Students’ perception of the app is positive, appreciative of the crowdsensing features, supporting learning goals, community building, and safety. On the other hand, the gamification features, as currently deployed, do not achieve the expected goals.

Dissociative Identity Disorder (DID) is characterized by the presence of at least two distinct identities in the same individual. This paper describes a co-design process with a person living with DID. We first aimed to uncover the main challenges experienced by the co-designer as well as design opportunities for novel technologies. We then engaged in a prototyping stage to design a wearable display (WhoDID) to facilitate in-person social interactions. The prototype aims to be used as a necklace and enable the user to make their fronting personality visible to others. Thus, facilitating social encounters or sudden changes of identity. We reflect on the design features of WhoDID in the broader context of supporting people with DID. Moreover, we provide insights on co-designing with someone with multiple (sometimes conflicting) personalities regarding requirement elicitation, decision-making, prototyping, and ethics. To our knowledge, we report the first design process with a DID user within the ASSETS and CHI communities. We aim to encourage other assistive technology researchers to design with DID users.

Typing on mobile devices is a common and complex task. The act of typing itself thereby encodes rich information, such as the typing method, the context it is performed in, and individual traits of the person typing. Researchers are increasingly using a selection or combination of experience sampling and passive sensing methods in real-world settings to examine typing behaviours. However, there is limited understanding of the effects these methods have on measures of input speed, typing behaviours, compliance, perceived trust and privacy. In this paper, we investigate the tradeoffs of everyday data collection methods. We contribute empirical results from a four-week field study (N=26). Here, participants contributed by transcribing, composing, passively having sentences analyzed and reflecting on their contributions. We present a tradeoff analysis of these data collection methods, discuss their impact on text-entry applications, and contribute a flexible research platform for in the wild text-entry studies.

Storytelling has the potential to be an inclusive and collaborative activity. However, it is unclear how interactive storytelling systems can support such activities, particularly when considering mixed-visual ability children. In this paper, we present an interactive multisensory storytelling system and explore the extent to which an emotional robot can be used to support inclusive experiences. We investigate the effect of the robot's emotional behavior on the joint storytelling process, resulting narratives, and collaboration dynamics. Results show that when children co-create stories with a robot that exhibits emotional behaviors, they include more emotive elements in their stories and explicitly accept more ideas from their peers. We contribute with a multisensory environment that enables children with visual impairments to engage in joint storytelling activities with their peers and analyze the effect of a robot's emotional behaviors on an inclusive storytelling experience.

The design of graphical user interfaces has been evolving from skeuomorph interfaces – which use elements that mimic the aesthetics and functionality of their real-world counterparts – to minimalist and flat designs. Despite the growing popularity of these new design approaches, they can be challenging for older adults who experience a decline in visual and cognitive abilities. Still, little is known about user performance, aesthetic perception, and preference of older adults, particularly in comparison to younger users and traditional skeuomorph interfaces. In this paper, we examine the performance and aesthetic perception of older (65–77 years old) and younger (20–40) adults with three design approaches: skeuomorph, skeuominimalist, and flat design. Results show flat design is either slower or less accurate than traditional skeuomorph interfaces for older adults across three tasks: visual search, identifying clickable objects, and multiple page navigation. Younger adults were less susceptible to performance differences between design approaches, but still subject to ‘click uncertainty’ with flat interfaces. Skeuominimalism did not show clear performance benefits over flat design or skeuomorphism, while the latter reduced the performance gap between age groups. Finally, younger adults preferred the simplicity of skeuominimalism, while older adults preferred skeuomorph interfaces because of the perceived usability, beauty, and trustiness.

Accessible introductory programming environments are scarce, and their study within ecological settings (e.g., at home) is almost non-existent. We present ACCembly, an accessible block-based environment that enables children with visual impairments to perform spatial programming activities. ACCembly allows children to assemble tangible blocks to program a multimodal robot. We evaluated this approach with seven families that used the system autonomously at home. Results showed that both the children and family members learned from what was an inclusive and engaging experience. Children leveraged fundamental computational thinking concepts to solve spatial programming challenges; parents took different roles as mediators, some actively teaching and scaffolding, others learning together with their child. We contribute with an environment that enables children with visual impairments to engage in spatial programming activities, an analysis of parent-child interactions, and reflections on inclusive programming environments within a shared family experience.

Visually impaired children (VI) face challenges in collaborative learning in classrooms. Robots have the potential to support inclusive classroom experiences by leveraging their physicality, bespoke social behaviors, sensors, and multimodal feedback. However, the design of social robots for mixed-visual abilities classrooms remains mostly unexplored. This paper presents a four-month-long community-based design process where we engaged with a school community. We provide insights into the barriers experienced by children and how social robots can address them. We also report on a participatory design activity with mixed-visual abilities children, highlighting the expected roles, attitudes, and physical characteristics of robots. Findings contextualize social robots within inclusive classroom settings as a holistic solution that can interact anywhere when needed and suggest a broader view of inclusion beyond disability. These include children’s personality traits, technology access, and mastery of school subjects. We finish by providing reflections on the community-based design process.

Previous attempts to make block-based programming accessible to visually impaired children have mostly focused on audio-based challenges, leaving aside spatial constructs, commonly used in learning settings. We sought to understand the qualities and flaws of current programming environments in terms of accessibility in educational settings. We report on a focus group with IT and special needs educators, where they discussed a variety of programming environments for children, identifying their merits, barriers and opportunities. We then conducted a workshop with 7 visually impaired children where they experimented with a bespoke tangible robot-programming environment. Video recordings of such activity were analyzed with educators to discuss children's experiences and emergent behaviours. We contribute with a set of qualities that programming environments should have to be inclusive to children with different visual abilities, insights for the design of situated classroom activities, and evidence that inclusive tangible robot-based programming is worth pursuing.

Geometry and handwriting rely heavily on the visual representation of basic shapes. It can become challenging for students with visual impairments to perceive these shapes and understand complex spatial constructs. For instance, knowing how to draw is highly dependent on spatial and temporal components, which are often inaccessible to children with visual impairments. Hand-held robots, such as the Cellulo robots, open unique opportunities to teach drawing and writing through haptic feedback. In this paper, we investigate how these tangible robots could support inclusive, collaborative learning activities, particularly for children with visual impairments. We conducted a user study with 20 pupils with and without visual impairments, where they engaged in multiple drawing activities with tangible robots. We contribute novel insights on the design of children-robot interaction, learning shapes and letters, children engagement, and responses in a collaborative scenario that address the challenges of inclusive learning.

There are over 80 million stroke survivors globally, making it the main cause of long-term disability worldwide. Not only do the challenges associated with stroke affect the quality of life (QoL) of survivors, but also of their families. To explore these challenges and define design opportunities for technologies to improve the QoL of both stakeholders, we conducted semi-structured interviews with 10 survivors and one of their family members. We uncovered three major interlinked themes: strategies to cope with technological barriers, the (in)adequacy of assistive technologies, and limitations of the rehabilitation process. Findings highlight multiple design opportunities, including the need for meaningful patient-centered tools and methods to improve rehabilitation effectiveness, emotion-aware computing for family emotional support, and re-thinking the nature of assistive technologies to consider the perception of transitory stroke-related disabilities. We thus argue for a new class of dual-purpose technologies that fit survivors' abilities while promoting the regain of function.

Sign Languages are visual languages and the main means of communication used by Deaf people. However, the majority of the information available online is presented through written form. Hence, it is not of easy access to the Deaf community. Avatars that can animate sign languages have gained an increase of interest in this area due to their flexibility in the process of generation and edition. Synthetic animation of conversational agents can be achieved through the use of notation systems. HamNoSys is one of these systems, which describes movements of the body through symbols. Its XML-compliant, SiGML, is a machine-readable input of HamNoSys able to animate avatars. Nevertheless, current tools have no freely available open source libraries that allow the conversion from HamNoSys to SiGML. Our goal is to develop a tool of open access, which can perform this conversion independently from other platforms. This system represents a crucial intermediate step in the bigger pipeline of animating signing avatars. Two cases studies are described in order to illustrate different applications of our tool.

Blind people face significant challenges when using smartphones. The focus on improving nonvisual mobile accessibility has been at the level of touchscreen access. Our research investigates the challenges faced by blind people in their everyday usage of mobile phones. In this article, we present a set of studies performed with the target population, novices and experts, using a variety of methods, targeted at identifying and verifying challenges; and coping mechanisms. Through a multiple methods approach we identify and validate challenges locally with a diverse set of user expertise and devices, and at scale through the analyses of the largest Android and iOS dedicate forums for blind people. We contribute with a prioritized corpus of smartphone challenges for blind people, and a discussion on a set of directions for future research, which tackle the open and often overlooked challenges.

Interactive tabletops offer unique collaborative features, particularly their size, geometry, orientation and, more importantly, the ability to support multi-user interaction. Although previous efforts were made to make interactive tabletops accessible to blind people, the potential to use them in collaborative activities remains unexplored. In this paper, we present the design and implementation of a multi-user auditory display for interactive tabletops, supporting three feedback modes that vary on how much information about the partners' actions is conveyed. We conducted a user study with ten blind people to assess the effect of feedback modes on workspace awareness and task performance. Furthermore, we analyze the type of awareness information exchanged and the emergent collaboration strategies. Finally, we provide implications for the design of future tabletop collaborative tools for blind users.

Word completion interfaces are ubiquitously available in mobile virtual keyboards; however, there is no prior research on how to design these interfaces for screen reader users. In addressing this, we propose a design space for nonvisual representation of word completions. The design space covers seven categories aiming to identify challenges and opportunities for interaction design in an unexplored research topic. It is intended to guide the design of novel interaction techniques, serving as a framework for researchers and practitioners working on nonvisual word completion. To demonstrate its potential, we engaged blind users in an exploration of the design space, to create their own bespoke word completion solutions. Through this study we found that users create alternative interfaces that extended current screen readers' capabilities. Resulting interfaces are less conservative than mainstream solutions on notification frequency and cardinality. Customization decisions were based on perceived benefits/costs and varied depending on multiple factors such as users' perceived prediction accuracy, potential keystroke gains, and situational restrictions.

Braille input enables fast nonvisual entry speeds on mobile touchscreen devices. Yet, the lack of tactile cues commonly results in typing errors, which are hard to correct. We propose Hybrid-Brailler, an input solution that combines physical and gestural interaction to provide fast and accurate Braille input. We use the back of the device for physical chorded input while freeing the touchscreen for gestural interaction. Gestures are used in editing operations, such as caret movement, text selection, and clipboard control, enhancing the overall text entry experience. We conducted two user studies to assess both input and editing performance. Results show that Hybrid-Brailler supports fast entry rates as its virtual counterpart, while significantly increasing input accuracy. Regarding editing performance, when compared with the mainstream technique, Hybrid-Brailler shows performance benefits of 21% in speed and increased editing accuracy. We finish with lessons learned for designing future nonvisual input and editing techniques.

Blind people face many barriers using smartphones. Still, previous research has been mostly restricted to non-visual gestural interaction, paying little attention to the deeper daily challenges of blind users. To bridge this gap, we conducted a series of workshops with 42 blind participants, uncovering application challenges across all levels of expertise, most of which could only be surpassed through a support network. We propose Hint Me!, a human-powered service that allows blind users to get in-app assistance by posing questions or browsing previously answered questions on a shared knowledge-base. We evaluated the perceived usefulness and acceptance of this approach with six blind people. Participants valued the ability to learn independently and anticipated a series of usages: labeling, layout and feature descriptions, bug workarounds, and learning to accomplish tasks. Creating or browsing questions depends on aspects like privacy, knowledge of respondents and response time, revealing the benefits of a hybrid approach.

Following multimedia lectures in mainstream classrooms is challenging for deaf and hard-of-hearing (DHH) students, even when provided with accessibility services. Due to multiple visual sources of information (e.g. teacher, slides, interpreter), these students struggle to divide their attention among several simultaneous sources, which may result in missing important parts of the lecture; as a result, access to information is limited in comparison to their hearing peers, having a negative effect in their academic achievements. In this paper we propose a novel approach to improve classroom accessibility, which focuses on improving the delivery of multimedia lectures. We introduce SlidePacer, a tool that promotes coordination between instructors and sign language interpreters, creating a single instructional unit and synchronizing verbal and visual information sources. We conducted a user study with 60 participants on the effects of SlidePacer in terms of learning performance and gaze behaviors. Results show that SlidePacer is effective in providing increased access to multimedia information; however, we did not find significant improvements in learning performance. We finish by discussing our results and limitations of our user study, and suggest future research avenues that build on these insights.

Touch-enabled devices have a growing variety of screen sizes; however, there is little knowledge on the effect of key size on non-visual text-entry performance. We conducted a user study with 12 blind participants to investigate how non-visual input performance varies with four QWERTY keyboard sizes (ranging from 15mm to 2.5mm). This paper presents an analysis of typing performance and touch behaviors discussing its implications for future research. Our findings show that there is an upper limit to the benefits of larger target sizes between 10mm and 15mm. Input speed decreases from 4.5 to 2.4 words per minute (WPM) for targets sizes below 10mm. The smallest size was deemed unusable by participants even though performance was in par with previous work.

Most work investigating mobile HCI is carried out within controlled laboratory settings; these spaces are not representative of the real-world environments for which the technology will predominantly be used. The result of which can produce a skewed or inaccurate understanding of interaction behaviors and users-- abilities. While mobile in-the-wild studies provide more realistic representations of technology usage, there are additional challenges to conducting data collection outside of the lab. In this paper we discuss these challenges and present TinyBlackBox, a standalone data collection framework to support mobile in-the-wild studies with today--s smartphone and tablet devices.

Non-visual text-entry for people with visual impairments has focused mostly on the comparison of input techniques reporting on performance measures, such as accuracy and speed. While researchers have been able to establish that non-visual input is slow and error prone, there is little understanding on how to improve it. To develop a richer characterization of typing performance, we conducted a longitudinal study with five novice blind users. For eight weeks, we collected in-situ usage data and conducted weekly laboratory assessment sessions. This paper presents a thorough analysis of typing performance that goes beyond traditional aggregated measures of text-entry and reports on character-level errors and touch measures. Our findings show that users improve over time, even though it is at a slow rate (0.3 WPM per week). Substitutions are the most common type of error and have a significant impact on entry rates. In addition to text input data, we analyzed touch behaviors, looking at touch contact points, exploration movements, and lift positions. We provide insights on why and how performance improvements and errors occur. Finally, we derive some implications that should inform the design of future virtual keyboards for non-visual input.

The advent of system-wide accessibility services on mainstream touch-based smartphones has been a major point of inclusion for blind and visually impaired people. Ever since, researchers aimed to improve the accessibility of specific tasks, such text-entry and gestural interaction. However, little work aimed to understand and improve the overall accessibility of these devices in real world settings. In this paper, we present an eight-week long study with five novice blind participants where we seek to understand major concerns, expectations, challenges, barriers, and experiences with smartphones. The study included pre-adoption and weekly interviews, weekly controlled task assessments, and in-the wild system-wide usage. Our results show that mastering these devices is an arduous and long task, confirming the users' initial concerns. We report on accessibility barriers experienced throughout the study, which could not be encountered in task-based laboratorial settings. Finally, we discuss how smartphones are being integrated in everyday activities and highlight the need for better adoption support tools.

Tablet devices can display full-size QWERTY keyboards similar to the physical ones. Yet, the lack of tactile feedback and the inability to rest the fingers on the home keys result in a highly demanding and slow exploration task for blind users. We present SpatialTouch, an input system that leverages previous experience with physical QWERTY keyboards, by supporting two-handed interaction through multitouch exploration and spatial, simultaneous audio feedback. We conducted a user study, with 30 novice touchscreen participants entering text under one of two conditions: (1) SpatialTouch or (2) mainstream accessibility method Explore by Touch. We show that SpatialTouch enables blind users to leverage previous experience as they do a better use of home keys and perform more efficient exploration paths. Results suggest that although SpatialTouch did not result in faster input rates overall, it was indeed able to leverage previous QWERTY experience in contrast to Explore by Touch.

Touchscreens are pervasive in mainstream technologies; they offer novel user interfaces and exciting gestural interactions. However, to interpret and distinguish between the vast ranges of gestural inputs, the devices require users to consistently perform interactions inline with the predefined location, movement and timing parameters of the gesture recognizers. For people with variable motor abilities, particularly hand tremors, performing these input gestures can be extremely challenging and impose limitations on the possible interactions the user can make with the device. In this paper, we examine touchscreen performance and interaction behaviors of motor-impaired users on mobile devices. The primary goal of this work is to measure and understand the variance of touchscreen interaction performances by people with motor-impairments. We conducted a four-week in-the-wild user study with nine participants using a mobile touchscreen device. A Sudoku stimulus application measured their interaction performance abilities during this time. Our results show that not only does interaction performance vary significantly between users, but also that an individual's interaction abilities are significantly different between device sessions. Finally, we propose and evaluate the effect of novel tap gesture recognizers to accommodate for individual variances in touchscreen interactions.

Braille has paved its way into mobile touchscreen devices, providing faster text input for blind people. This advantage comes at the cost of accuracy, as chord typing over a flat surface has proven to be highly error prone. A misplaced finger on the screen translates into a different or unrecognized character. However, the chord itself gathers information that can be leveraged to improve input performance. We present B#, a novel correction system for multitouch Braille input that uses chords as the atomic unit of information rather than characters. Experimental results on data collected from 11 blind people revealed that B# is effective in correcting errors at character-level, thus providing opportunities for instant corrections of unrecognized chords; and at word-level, where it outperforms a popular spellchecker by providing correct suggestions for 72% of incorrect words (against 38%). We finish with implications for designing chord-based correction system and avenues for future work.

Despite the overwhelming emergence of accessible digital technologies, Braille still plays a role in providing blind people with access to content. Nevertheless, many fail to see the benefits of nurturing Braille, particularly given the time and effort required to achieve proficiency. Our research focuses on maximizing access and motivation to learn and use Braille. We present initial insights from 5 interviews with blind people, comprising of Braille instructors and students, where we characterize the learning process and usage of Braille. Based on our findings, we have identified a set of opportunities around Braille education. Moreover, we devised scenarios, and built hardware and software solutions to motivate discovery and retention of Braille literacy.

Blind people typically resort to audio feedback to access information on electronic devices. However, this modality is not always an appropriate form of output. Novel approaches that allow for private and inconspicuous interaction are paramount. In this paper, we present a vibrotactile reading device that leverages the users' Braille knowledge to read textual information. UbiBraille consists of six vibrotactile actuators that are used to code a Braille cell and communicate single characters. The device is able to simultaneously actuate the users' index, middle, and ring fingers of both hands, providing fast and mnemonic output. We conducted two user studies on UbiBraille to assess both character and word reading performance. Character recognition rates ranged from 54% to 100% and were highly character- and user-dependent. Indeed, participants with greater expertise in Braille reading/writing were able to take advantage of this knowledge and achieve higher accuracy rates. Regarding word reading performance, we investigated four different vibrotactile timing conditions. Participants were able to read entire words and obtained recognition rates up to 93% with the most proficient ones being able achieve a rate of 1 character per second.

Touchscreen devices have become increasingly popular. Yet they lack of tactile feedback and motor stability, making it difficult effectively typing on virtual keyboards. This is even worse for elderly users and their declining motor abilities, particularly hand tremor. In this paper we examine text-entry performance and typing patterns of elderly users on touch-based devices. Moreover, we analyze users' hand tremor profile and its relationship to typing behavior. Our main goal is to inform future designs of touchscreen keyboards for elderly people. To this end, we asked 15 users to enter text under two device conditions (mobile and tablet) and measured their performance, both speed- and accuracy-wise. Additionally, we thoroughly analyze different types of errors (insertions, substitutions, and omissions) looking at touch input features and their main causes. Results show that omissions are the most common error type, mainly due to cognitive errors, followed by substitutions and insertions. While tablet devices can compensate for about 9% of typing errors, omissions are similar across conditions. Measured hand tremor largely correlates with text-entry errors, suggesting that it should be approached to improve input accuracy. Finally, we assess the effect of simple touch models and provide implications to design.

In this paper we present AppInsight, a visualization tool that enables users to reminisce on their computer usage history and derive meaningful insights about behaviors and trends. Human memory has the ability to re-experience episodes from our lives when supplied with suitable contextual cues, such as places, music, and so on. We explore a small set of properties, such as the application's name, URL and window title as contextual cues, in order to characterize the users' activity on their personal computers and how it evolves over time. Our user study shows that users enjoyed viewing their computer usage history and were able to both recall past events and introspect about their lives. Moreover, one of the most surprising outcomes is that they found several different applications for our tool, such as improving usage behaviors, controlling productivity, generating activity reports, and monitoring users in psychological studies. Finally, we discuss some lessons learned from our study and propose future research directions.

Mobile touch devices have become increasingly popular, yet typing on virtual keyboards whilst walking is still an overwhelming task. In this paper we analyze; firstly, the negative effect of walking on text-input performance, particularly the users' main difficulties and error patterns. We focused our research on thumb typing, since this is a commonly used technique to interact with touch interfaces. Secondly, we analyze how these effects can be compensated by two-hand interaction and increasing target size. We asked 22 participants to input text under three mobility conditions (seated, slow walking, and normal walking) and three hand conditions (one-hand/portrait, two-hand/portrait, and two-hand/landscape). Results show that independently of hand condition, mobility significantly decreased input quality, leading to specific error patterns. Moreover, it was shown that target size can compensate the negative effect of walking, while two-hand interaction does not provide additional stability or input accuracy. We finish with implications for future designs.

The emergence of touch-based mobile devices brought fresh and exciting possibilities. These came at the cost of a considerable number of novel challenges. They are particularly apparent with the blind population, as these devices lack tactile cues and are extremely visually demanding. Existing solutions resort to assistive screen reading software to compensate the lack of sight, still not all the information reaches the blind user. Good spatial ability is still required to have notion of the device and its interface, as well as the need to memorize buttons' position on screen. These abilities, as many other individual attributes as age, age of blindness onset or tactile sensibility are often forgotten, as the blind population is presented with the same methods ignoring capabilities and needs. Herein, we present a study with 13 blind people consisting of a touch screen text-entry task with four different methods. Results show that different capability levels have significant impact on performance and that this impact is related with the different methods' demands. These variances acknowledge the need of accounting for individual characteristics and giving space for difference, towards inclusive design.

Touch screen mobile devices bear the promise of endless leisure, communication, and productivity opportunities to motor-impaired people. Indeed, users with residual capacities in their upper extremities could benefit immensely from a device with no demands regarding strength. However, the precision required to effectively select a target without physical cues creates problems to people with limited motor abilities. Our goal is to thoroughly study mobile touch screen interfaces, their characteristics and parameterizations, thus providing the tools for informed interface design for motor-impaired users. We present an evaluation performed with 15 tetraplegic people that allowed us to understand the factors limiting user performance within a comprehensive set of interaction techniques (Tapping, Crossing, Exiting and Directional Gesturing) and parameterizations (Position, Size and Direction). Our results show that for each technique, accuracy and precision vary across different areas of the screen and directions, in a way that is directly dependent on target size. Overall, Tapping was both the preferred technique and among the most effective. This proves that it is possible to design inclusive unified interfaces for motor-impaired and able-bodied users once the correct parameterization or adaptability is assured.

Mobile devices are designed mostly to fit users with no particular disability. Tactile affordances are neglected at the expense of more attractive stylish interfaces and assistive solutions are stereotypical, also facing disabilities with a narrow perspective. A blind user is presented with screen reading software to overcome the inability to receive feedback from the device. However, these solutions go only half-way. In the absence of sight other capabilities stand up. Above all, the sense of touch plays an essential role while interacting with physical keypads. To empower these users, a deeper understanding of their capabilities and how they relate with technology is mandatory. We propose a user-product compatibility approach, taking in account that blind users have different tactile attributes. We expect to correlate the user's tactile sensitivity and keypad demands, enabling informed keypad design and selection.

Mobile touch-screen interfaces and tetraplegic people have a controversial connection. While users with residual capacities in their upper extremities could benefit immensely from a device which does not require strength to operate, the precision needed to effectively select a target bars these people access to countless communication, leisure and productivity opportunities. Insightful projects attempted to bridge this gap via either special hardware or particular interface tweaks. Still, we need further insight into the challenges and the frontiers separating failure from success for such applications to take hold. This paper discusses an evaluation conducted with 15 tetraplegic people to learn the limits to their performance within a comprehensive set of interaction methods. We then present the results concerning a particular interaction technique: Tapping. Results show that performance varies across different areas of the screen whose distribution changes with target size.

We are moving towards a future where people will be ever more surrounded by technology and multiple appliances, bringing about the promise of truly intelligent environments. However, this multitude of devices raises several issues to HCI practitioners. Indeed, our preliminary studies confirm that blind people experience difficulties with most appliances, due to inadequate interfaces. The research described here approaches this problem by moving the user interface away from appliances to an intermediary device, which blind people are familiar with and can fully control. Additionally, we propose an automatic generation algorithm, which provides a consistent user interface to all appliances in the environment.

NavTap is a navigational method that enables blind users to input text in a mobile device by reducing the associated cognitive load.