Self-motion is an essential but often overlooked component of sound localisation. As the directional information of a source is implicitly contained in head-centred acoustic cues, that acoustic input needs to be continuously combined with sensorimotor information about the head orientation in order to decode to a world-centred frame of reference. When utilised, head movements significantly reduce ambiguities in the directional information provided by the incoming sound. In this work, we model human active sound localisation (considering small head rotations) as an ideal observer. In the evaluation, we compared human performance obtained in a free-field active localisation experiment with the predictions of a Bayesian model. Model noise parameters were set a-priori based on behavioural results from other studies, i.e., without any post-hoc parameter fitting to behavioural results. The model predictions showed a general agreement with actual human performance. However, a spatial analysis revealed that the ideal observer was not able to predict localisation behaviour for each source direction. A more detailed investigation into the effects of various model parameters indicated that uncertainty on head orientation significantly contributed to the observed differences. Yet, the biases and spatial distribution of the human responses remained partially unexplained by the presented ideal observer model, suggesting that human sound localisation is sub-optimal.

Time-frequency (TF) representations provide powerful and intuitive features for the analysis of time series such as audio. But still, generative modeling of audio in the TF domain is a subtle matter. Consequently, neural audio synthesis widely relies on directly modeling the waveform and previous attempts at unconditionally synthesizing audio from neurally generated invertible TF features still struggle to produce audio at satisfying quality. In this article, focusing on the short-time Fourier transform, we discuss the challenges that arise in audio synthesis based on generated invertible TF features and how to overcome them. We demonstrate the potential of deliberate generative TF modeling by training a generative adversarial network (GAN) on short-time Fourier features. We show that by applying our guidelines, our TF-based network was able to outperform a state-of-the-art GAN generating waveforms directly, despite the similar architecture in the two networks.

In this article, we study the ability of deep neural networks (DNNs) to restore missing audio content based on its context, i.e., inpaint audio gaps. We focus on a condition which has not received much attention yet: gaps in the range of tens of milliseconds. We propose a DNN structure that is provided with the signal surrounding the gap in the form of time-frequency (TF) coefficients. Two DNNs with either complex-valued TF coefficient output or magnitude TF coefficient output were studied by separately training them on inpainting two types of audio signals (music and musical instruments) having 64-ms long gaps. The magnitude DNN outperformed the complex-valued DNN in terms of signal-to-noise ratios and objective difference grades. Although, for instruments, a reference inpainting obtained through linear predictive coding performed better in both metrics, it performed worse than the magnitude DNN for music. This demonstrates the potential of the magnitude DNN, in particular for inpainting signals that are more complex than single instrument sounds.

Pre-trained models for:\n\nAdversarial Generation of Time-Frequency Features with application in audio synthesis\nA Marafioti, N Holighaus, N Perraudin, P Majdak - arXiv preprint arXiv:1902.04072, 2019\n\n

Head-related transfer functions (HRTFs) describe the acoustic filtering of incoming sounds by the human morphology and are essential for listeners to localize sound sources in virtual auditory displays. Since rendering complex virtual scenes is computationally demanding, we propose four algorithms for efficiently representing HRTFs in subbands, i.e., as an analysis filterbank (FB) followed by a transfer matrix and a synthesis FB. All four algorithms use sparse approximation procedures to minimize the computational complexity while maintaining perceptually relevant HRTF properties. The first two algorithms separately optimize the complexity of the transfer matrix associated to each HRTF for fixed FBs. The other two algorithms jointly optimize the FBs and transfer matrices for complete HRTF sets by two variants. The first variant aims at minimizing the complexity of the transfer matrices, while the second one does it for the FBs. Numerical experiments investigate the latency-complexity trade-off and show that the proposed methods offer significant computational savings when compared with other available approaches. Psychoacoustic localization experiments were modeled and conducted to find a reasonable approximation tolerance so that no significant localization performance degradation was introduced by the subband representation.

In the context of computational acoustics, a detailed evaluation of various 3-D scanning methods for the purpose of capturing the geometry of the human pinna (the visible part of the ear) is presented. Over 80 full 3-D scans of the head and ears of three subjects were performed with six different 3-D scanning methods, including photogrammetry with nine different camera and texturing options. We provide a numerical comparison of the scanning performance in terms of accuracy and completeness, and show the effect of ambient occlusion on the accuracy. The numerical and practical issues for scanning the ears for computational acoustics are discussed.

Exponential sweeps are widely used to measure impulse responses of electro-acoustic systems. Measurements are often contaminated by environmental noise and nonlinear distortions. We propose a method to increase the signal-to-noise ratio (SNR) by denoising the recorded signal in the time-frequency plane. In contrast to state-of-the art denoising methods, no assumption about the spectral characteristics of the noise is required. Numerical simulations demonstrate improvements in the SNR under low-SNR conditions even for measurements contaminated by colored noise.

A linear system can be approximated in the time-frequency domain by the composition of an analysis filterbank, a transfer matrix (subband model) and a synthesis filterbank, a method known as subband technique. In this paper we propose an iterative method to jointly optimize the subband model, analysis and synthesis filterbank. To this end we propose a minimization criterion which we solve using the so-called alternating least-squares method. As a possible application we consider the implementation of the so-called head related transfer functions which are used in virtual acoustics. Simulation results suggest that the subband technique, optimized using the proposed method proposed method, is a promising approach.