We propose a new head-related transfer function (HRTF) interpolation method using Isomap, a nonlinear dimensionality reduction technique. First, we construct a single manifold for all subjects across both azimuth and elevation angles through the construction of an intersubject graph (ISG) that includes important prior knowledge of the HRTFs such as correlations across individuals, directions, and ears. Then, for a new direction, we predict its corresponding low-dimensional HRTF by interpolating over same subject low-dimensional measured HRTFs. Finally, we use a local neighborhood mapping in the manifold to reconstruct the high-dimensional HRTF from measured HRTFs of all subjects. We show that a single manifold representation obtained through the ISG is a powerful way to allow measured HRTFs from different subjects to contribute for reconstructing the HRTFs for new directions. Moreover, our results suggest that a small number of spatial measurements capture most of acoustical properties of HRTFs. Finally, our approach outperforms other linear and nonlinear dimensionality reduction techniques such as principal component analysis, locally linear embedding, and Laplacian eigenmaps.

In this paper, we introduce a real-time face recognition (and announcement) system targeted at aiding the blind and low-vision people. The system uses a Microsoft Kinect sensor as a wearable device, performs face detection, and uses temporal coherence along with a simple biometric procedure to generate a sound associated with the identified person, virtualized at his/her estimated 3-D location. Our approach uses a variation of the K-nearest neighbors algorithm over histogram of oriented gradient descriptors dimensionally reduced by principal component analysis. The results show that our approach, on average, outperforms traditional face recognition methods while requiring much less computational resources (memory, processing power, and battery life) when compared with existing techniques in the literature, deeming it suitable for the wearable hardware constraints. We also show the performance of the system in the dark, using depth-only information acquired with Kinect's infrared camera. The validation uses a new dataset available for download, with 600 videos of 30 people, containing variation of illumination, background, and movement patterns. Experiments with existing datasets in the literature are also considered. Finally, we conducted user experience evaluations on both blindfolded and visually impaired users, showing encouraging results.

We present a new anthropometry-based method to personalize head-related transfer functions (HRTFs) using manifold learning in both azimuth and elevation angles with a single nonlinear regression model. The core element of our approach is a domain-specific nonlinear dimensionality reduction technique, denominated Isomap, over the intraconic component of HRTFs resulting from a spectral decomposition. HRTF intraconic components encode the most important cues for HRTF individualization, leaving out subject-independent cues. First, we modify the graph construction procedure of Isomap to integrate relevant prior knowledge of spatial audio into a single manifold for all subjects by exploiting the existing correlations among HRTFs across individuals, directions, and ears. Then, with the aim of preserving the multifactor nature of HRTFs (i.e. subject, direction and frequency), we train a single artificial neural network to predict low-dimensional HRTFs from anthropometric features. Finally, we reconstruct the HRTF from its estimated low-dimensional version using a neighborhood-based reconstruction approach. Our findings show that introducing prior knowledge in Isomap's manifold is a powerful way to capture the underlying factors of spatial hearing. Our experiments show, with p-values less than 0.05, that our approach outperforms using, either a PCA linear reduction, or the full HTRF, in its intermediate stages.

In this paper, we introduce a new anthropometric-based method for customizing of Head-Related Transfer Functions (HRTF) in the horizontal plane. The method uses Isomap, artificial neural networks (ANN), and a neighborhood-based reconstruction procedure. We first modify Isomap's graph construction step to emphasize the individuality of HRTFs and perform a customized nonlinear dimensionality reduction of the HTRFs. We then use an ANN to model the nonlinear relationship between anthropometric features and our low-dimensional HRTFs. Finally, we use a neighborhood-based reconstruction approach to reconstruct the HRTF from the estimated low-dimensional version. Simulations show that our approach performs better than PCA and confirm that Isomap is capable of discovering the underlying nonlinear relationships of sound perception.