This article presents an automatic approach, complete with a prototype system, to supporting fast and independent access to online maps for local navigation by people with visual impairment. With user-inputted start and end addresses from a keyboard, the approach first queries MapQuest (http://www.mapquest.com) for obtaining the walking directions and the corresponding map image. Then, it automatically converts the obtained information in a form that can be reproduced immediately through a tactile printer, and subsequently generates an SVG (Scalable Vector Graphics) file, which associates textual descriptions of the directions with a recreated tactile map. The tactile hard copy can be placed on a touchpad which is connected to a computer. With the generated SVG file opened in the computer, a user can explore the tactile map by hands, receiving instant audio feedback of the directions by pressing certain regions with special tactile patterns. This approach supports instant queries of walking directions without requiring tedious manual conversion by a sighted professional. The audio-tactile patterns, the adaptive representation scheme and the blind-friendly user interface are specifically designed for the visually-impaired users. Results from experimental evaluation based on a group of users with visual impairment suggest that the proposed approach is effective for providing blind computer users with independent access to geographic directions.

Face photos/Portraits play an important role in people's social and emotional life. Unfortunately, this type of media is inaccessible to people with visual impairment. We propose a novel, prototypical system that was designed to demonstrate the feasibility of bridging this accessibility gap through automatic and realtime conversion of face images into their tactile counterparts. Unlike conventional and existing tactile conversion efforts, which are largely done by human specialists, the proposed system serves to provide an intelligent interface between blind computer users and this important type of media, human face images.

Compared to well-edited videos with predefined structures (e.g., news or sports videos), extracting key frames from unconstrained consumer videos remains a much more challenging problem due to their extremely diverse contents (no pre-imposed structure) and uncontrolled video quality (e.g., due to poor lighting or camera shake). In order to exploit spatio-temporal correlation present in the video for key frame extraction, we propose a bi-layer group sparse representation in which the input video frames are first segmented into homogeneous patches and group sparsity is imposed at two levels simultaneously: (i) patch-to-frame, and (ii) frame-to-sequence. The grouped sparse coefficients are further combined with frame quality scores to generate key frames. Extensive experiments are performed on videos from actual end users. Results obtained by the proposed approach compare favorably with existing methods to confirm its effectiveness.

Automatic categorization of videos in a Web-scale unconstrained collection such as YouTube is a challenging task. A key issue is how to build an effective training set in the presence of missing, sparse or noisy labels. We propose to achieve this by first manually creating a small labeled set and then extending it using additional sources such as related videos, searched videos, and text-based webpages. The data from such disparate sources has different properties and labeling quality, and thus fusing them in a coherent fashion is another practical challenge. We propose a fusion framework in which each data source is first combined with the manually-labeled set independently. Then, using the hierarchical taxonomy of the categories, a Conditional Random Field (CRF) based fusion strategy is designed. Based on the final fused classifier, category labels are predicted for the new videos. Extensive experiments on about 80K videos from 29 most frequent categories in YouTube show the effectiveness of the proposed method for categorizing large-scale wild Web videos.

Portrait photos (facial images) play important social and emotional roles in our life. This type of visual media is unfortunately inaccessible by users with visual impairment. This paper proposes a systematic approach for automatically converting human facial images into a tactile form that can be printed on a tactile printer and explored by a user who is blind. We propose a deformable Bayesian Active Shape Model (BASM), which integrates anthropometric priors with shape and appearance information learnt from a face dataset. We design an inference algorithm under this model for processing new face images to create an input-adaptive face sketch. Further, the model is enhanced by input-specific details through semantic-aware processing. We report experiments on evaluating the accuracy of face alignment using the proposed method, with comparison with other state-of-the-art results. Furthermore, subjective evaluations of the produced tactile face images were performed by 17 persons including six visually-impaired users, confirming the effectiveness of the proposed approach in conveying via haptics vital visual information in a face image.

In this paper, we propose an automatic approach, complete with a prototype system, for supporting instant access to maps for local navigation by people with visual impairment. The approach first detects and segments texts from a map image and recreates the remaining graphical parts in a tactile form which can be reproduced immediately through a tactile printer. Then, it generates an SVG (Scalable Vector Graphics) file, which integrates both text and graphical information. The tactile hardcopy and the SVG file together are used to provide a user with interactive access to the map image through a touchpad, resulting in a tactile-audio representation of the original input image. This supports real-time access to the map without tedious conversion by a sighted professional. Evaluations with six users who are blind show that the created tactile-audio maps from our prototype system convey the most important map information and are deemed as potentially useful for local navigation.

Computer users with visual impairment cannot access the rich graphical contents in print or digital media unless relying on visual-to-tactile conversion, which is done primarily by human specialists. Automated approaches to this conversion are an emerging research field, in which currently only simple graphics such as diagrams are handled. This paper proposes a systematic method for automatically converting a human portrait image into its tactile form. We model the face based on deformable active shape model (ASM) (Cootes et al., 1995), which is enriched by local appearance models in terms of gradient profiles along the shape. The generic face model including the appearance components is learnt from a set of training face images. Given a new portrait image, the prior model is updated through Bayesian inference. To facilitate the incorporation of a pose-dependent appearance model, we propose a statistical sampling scheme for the inference task. Furthermore, to compensate for the simplicity of the face model, edge segments of a given image are used to enrich the basic face model in generating the final tactile printout. Experiments are designed to evaluate the performance of the proposed method.

Researches in psychology, perception and related fields show that there may be a two-stage process involved in human vision. In this paper, we propose an approach by following a two-stage framework for saliency detection. In the first stage, we extend an existing spectrum residual model for better locating visual pop-outs, while in the second stage we make use of coherence based propagation for further refinement of the results from the first step. For evaluation of the proposed approach, 300 images with diverse contents were manually and accurately labeled. Experiments show that our approach achieves much better performance than that from the existing state-of-art.