Auditory user interfaces have great Web-access potential for billions of people with visual impairments, with limited literacy, who are driving, or who are otherwise unable to use a visual interface. However a sequential speech-based representation can only convey a limited amount of information. In addition, typical auditory user interfaces lose the visual cues such as text styles and page structures, and lack effective feedback about the current focus. To address these limitations, we created Sasayaki (from whisper in Japanese), which augments the primary voice output with a secondary whisper of contextually relevant information, automatically or in response to user requests. It also offers new ways to jump to semantically meaningful locations. A prototype was implemented as a plug-in for an auditory Web browser. Our experimental results show that the Sasayaki can reduce the task completion times for finding elements in webpages and increase satisfaction and confidence.

Haptic technologies are often used to improve access to the structural content of graphical user interfaces, thereby augmenting the interaction process for blind users. While haptic design guidelines offer valuable assistance when developing non-visual interfaces, the recommendations presented are often tailored to the feedback produced via one particular haptic input/output device. A blind user is therefore restricted to interacting with a device which may be unfamiliar to him/her, rather than selecting from the range of commercially available products. This paper reviews devices available on the first and second-hand markets, and describes an exploratory study undertaken with 12 blindfolded sighted participants to determine the effectiveness of three devices for non-visual web interaction. The force-feedback devices chosen for the study, ranged in the number of translations and rotations that the user was able to perform when interacting with them. Results have indicated that the Novint Falcon could be used to target items faster in the first task presented, compared with the other devices. However, participants agreed that the force-feedback mouse was most comfortable to use when interacting with the interface. Findings have highlighted the benefits which low cost haptic input/output devices can offer to the non-visual browsing process, and any changes which may need to be made to accommodate their deficiencies. The study has also highlighted the need for web designers to integrate appropriate haptic feedback on their web sites to cater for the strengths and weaknesses of various devices, in order to provide universally accessible sites and online applications.

While the usability of voice-based Web navigation has been steadily improving, it is still not as easy for users with visual impairments as it is for sighted users. One reason is that sequential voice representation can only convey a limited amount of information at a time. Another challenge comes from the fact that current voice browsers omit various visual cues such as text styles and page structures, and lack meaningful feedback about the current focus. To address these issues, we created Sasayaki, an intelligent voice-based user agent that augments the primary voice output of a voice browser with a secondary voice that whispers contextually relevant information as appropriate or in response to user requests. A prototype has been implemented as a plug-in for a voice browser. The results from a pilot study show that our Sasayaki agent is able to improve users' information search task time and their overall confidence level. We believe that our intelligent voice-based agent has great potential to enrich the Web browsing experiences of users with visual impairments.

Speech recognition is a promising alternative input technology for individuals with upper-body motor impairments that hinder the use of the standard keyboard and mouse. A recent long-term field study found that the users employed speech techniques for a variety of tasks beyond generating text documents [1]. One challenge with hands-free speech-based interactions is user authentication, which requires the users to speak their user IDs and passwords character by character. Unfortunately, speaking a password presents both security and privacy threats as well as usability problems. To address this challenge, we propose a new speech-based authentication model. An initial proof-of-concept prototype has been implemented and a pilot study was conducted. Preliminary results suggest several problems for further examination.

Existing knowledge on how people use speech-based technologies in realistic settings is limited. We are conducting a longitudinal field study, spanning six months, to investigate how users with no physical impairments and users with upper body physical impairments use speech technologies when interacting with computers in their home environment. Digital data logs, time diaries, and interviews are being used to record the types of applications used, frequency of use of each application, and difficulties experienced as well as subjective data regarding the usage experience. While confirming many expectations, initial results have provided several unexpected insights including a preference to use speech for navigation instead of dictation tasks, and the use of speech technology for programming and games.