Abstract Access to music for people who are Deaf/Hard of Hearing (D/HoH) includes the instrumental portion and the lyrics. While closed captioning can provide the lyrics in text format, it does not necessarily provide accurate timing of the lyrics with the instrumental portion. This study aims to clarify how vibrotactile stimuli affect understanding the onset timing of song lyrics for D/HoH people. To achieve this goal, we developed a system called VIBES: VIBrotactile Engagement for Songs that simultaneously provides music and vibration playback as an iPhone app. Unlike other vibrotactile systems for music, which focus primarily on the percussion/beat or frequencies of the instrumental portions, VIBES presents vibrations for the timing of vocal utterances, syllable by syllable. We conducted a study with 10 participants to determine the system’s effectiveness by comparing the understanding of lyric timing with vibration and sound only. Statistically, one of the four songs in the experiment showed significant differences, where understanding lyric timing with vibration is better than in sound-only conditions. Two songs revealed the opposite results, and the other showed no differences between the two conditions. Although these results were unexpected, we obtained several findings to make the next version of VIBES, such as the frequency of vibrations.

An extended design and evaluation framework of eudaimonia (personal growth, expressiveness) and hedonia (pleasure, comfort) was applied to a cooperative game for older adults who rely on power mobility. The purpose was to address two psychosocial well-being needs (perceptions of performance mastery and empathy enhancement) through a game with an interaction format that augments the experience of powered chair use: mixed reality with power mobility-based interaction and movement. Two versions of the game—one in a mixed reality format and one without—were created to evaluate the efficacy of using this framework and explore relevant user experience factors. Qualitative findings suggest that the mixed reality version of the game elicited eudaimonic experiences, where the older adult participants who were typically reliant on power mobility perceived a positive change in performance mastery while their partners, who were not reliant on power mobility developed a greater appreciation for everyday powered chair use. Further, a role reversal, where the elders took on a ‘caregiving’ role in the context of powered chair expertise, was observed. Moreover, inferential findings showed evidence of a predictive relationship between hedonic and eudaimonic orientations, psychological well-being, and game engagement, regardless of age, disability status, and game version.

Closed Captioning (CC) is a service primarily designed for deaf and hard of hearing (D/HoH) viewers. The CC translates spoken speech into text for television or film screen display. The quality assessment methods for live captioning are limited to quantitative measures, while the viewers are still dissatisfied with the current quality. One method to improve the current quality assessment procedure is to include D/HoH viewers in the evaluation procedure for their subjective assessment input. However, it could be costly and impractical to perform evaluations for the entire broadcasted shows. Therefore, it would be helpful to model subjective assessments that could replicate and predict human decisions. In this article, we report on a model of probabilities of D/HoH viewer assessment decisions for CC quality factors based on actual user preferences. An online survey was designed and conducted to collect assessment data for 22 error variation samples from four quality factors: delay, speed, missing words, and paraphrasing of captions. The results are analyzed using the signal detection theory framework to create decision probability models for D/HoH viewers.

The advent of affordable and powerful mobile technology has allowed for explorations in mixed reality that merges virtual and physical space. However, the social and entertainment value and efficacy of mixed reality platforms for adult powered chair users has not been widely explored. In this article, we introduce the Mobility Games project, which aims to produce a series of inclusive entertainment technologies and services for people who use powered chairs. We describe our first offering: an accessible, social mixed reality game for co-located mobile play in everyday spaces. Findings from two exploratory field studies and a post hoc observer survey show that adult powered chair users found the game to be entertaining and used a variety of path strategies as they learned to play the game. An initial set of theoretically and empirically informed guidelines for making mobile mixed reality games accessible to adult powered chair users with diverse abilities is proposed.

Gamification has drawn the attention of academics, practitioners and business professionals in domains as diverse as education, information studies, human–computer interaction, and health. As yet, the term remains mired in diverse meanings and contradictory uses, while the concept faces division on its academic worth, underdeveloped theoretical foundations, and a dearth of standardized guidelines for application. Despite widespread commentary on its merits and shortcomings, little empirical work has sought to validate gamification as a meaningful concept and provide evidence of its effectiveness as a tool for motivating and engaging users in non-entertainment contexts. Moreover, no work to date has surveyed gamification as a field of study from a human–computer studies perspective. In this paper, we present a systematic survey on the use of gamification in published theoretical reviews and research papers involving interactive systems and human participants. We outline current theoretical understandings of gamification and draw comparisons to related approaches, including alternate reality games (ARGs), games with a purpose (GWAPs), and gameful design. We present a multidisciplinary review of gamification in action, focusing on empirical findings related to purpose and context, design of systems, approaches and techniques, and user impact. Findings from the survey show that a standard conceptualization of gamification is emerging against a growing backdrop of empirical participants-based research. However, definitional subjectivity, diverse or unstated theoretical foundations, incongruities among empirical findings, and inadequate experimental design remain matters of concern. We discuss how gamification may to be more usefully presented as a subset of a larger effort to improve the user experience of interactive systems through gameful design. We end by suggesting points of departure for continued empirical investigations of gamified practice and its effects.

Highly textual websites present barriers to Deaf people, primarily using American Sign Language for communication. Deaf people have been posting ASL content in form of vlogs to YouTube and specialized websites such as Deafvideo.TV. This paper presents some of the first insights into the use of vlogging technology and techniques among the Deaf community. The findings suggest that there are differences between YouTube and Deafvideo.TV due to differences between mainstream and specialized sites. Vlogging technology seems to influence use of styles that are not found or are used differently in face-to-face communications. Examples include the alteration of vloggers' signing space to convey different meanings on screen.

We present an experiment designed to reveal characteristics of a tactile display that presents vibrations representing music to the back of the body. Based on the model human cochlea, a sensory substitution system aimed at translating music into vibrations, we are investigating the use of larger contactor sizes (over 10mm in diameter) as an effective device for the detection of signals originating from music. Using the method of limits, we measured ability to discriminate the frequency of vibrotactile stimuli across a wide range of frequencies common to western classical harmonic music. Vibrotactile stimuli were presented to artificially deafened participants using a large contactor applied to the back. Between 65 Hz (C2) and 1047 Hz (C6), frequency difference limens (FDL) were consistently less than 1/3 of an octave and as small as 200 cents. These findings suggest that vibrotactile information can be used to support the experience of music even in the absence of sound, and that voice coils are effective in presenting some characteristics of sound as vibrations.

SLS:Caption provides captioning functionality for deaf and hearing users to provide captions to video content (including sign language content). Users are able to enter and modify text as well as adjust its font, colour, location and background opacity. An initial user study with hearing users showed that SLS:Caption was easy to learn and use. However, users seem reluctant to produce captions for their own video material; this was likely due to the task complexity and time required to create captions regardless of the usability of the captioning tool.

The model human cochlea (MHC) is a sensory substitution technique for presenting music as multiple discrete channels of vibrotactile stimuli. The MHC prototype was introduced to a group of deaf senior citizens at a community centre, where they could try the chair, and provide us with feedback about their experience. Preliminary results from this work suggest that the MHC can potentially offer a more effective means of expressing basic emotional information from music in a vibrotactile display when the signals more closely match the perceptual elements of the music such as melody and instrument parts. We describe the MHC prototype and present the results of our field study, which extends our lab experiments with the MHC to include deaf and hard of hearing participants in a real world setting.

Closed captioning has suffered from a lack of innovation since its inception in the early 1970s. However, television and film technologies and user preferences have changed dramatically. Sound from music, sound effects, and speech prosody are essentially missing from current closed captions. We used animated text to represent emotions contained in music and speech as well as sound effects. Twenty-five hard of hearing and hearing participants watched two short television clips with three different types of captions—conventional, enhanced, and extreme. Hard of hearing and hearing participants preferred enhanced, animated text captions as they provide improved access to the emotive information contained in the content. Text-based animated sound effects confused participants and animated symbols were recommended as a replacement.

Music, sound effects and speech prosody are missing from conventional closed captions. Despite advancements, closed captioning has yet to surpass the threshold of 1970s technology. In this study, animated text was used to convey missing sound information to hearing impaired users. Twenty-five hard of hearing and hearing participants watched two television clips containing three different styles of captions: conventional, enhanced and extreme. Participants preferred enhanced, animated text captions as their access to emotive information was improved.

There is a need to assess the applicability of conventional Usability Evaluation Methods to users with disabilities, given the growing importance of involving these users in the usability evaluation process. We found that conventional Think Aloud Protocol cannot be used as is, and will require modification to be useful, when evaluating websites with blind users.

We present two synchronous communication systems, a video-mediated communication system called PEBBLES, and accessible, text-based communication tools called JAMTalk. PEBBLES combines video-conferencing with simple robotics to link a child in the hospital with his/her regular school. JAMTalk tools provide accessible real-time communication tools for distance education. We suggest that results from PEBBLES and JAMTalk evaluations indicate that accessible electronic communication systems can provide unique advantages to students with disabilities.