Abstract We present a technique to model wave‐based sound propagation to complement visual animation in fully dynamic scenes. We employ 2D wave simulation that captures geometry‐based diffraction effects such as obstruction, reverberation, and directivity of perceptually‐salient initial sound at the source and listener. We show real‐time performance on a single CPU core on modestly‐sized scenes that are nevertheless topologically complex. Our key ideas are to exploit reciprocity and use a perceptual encoding and rendering framework. These allow the use of low‐frequency finite‐difference simulations on static scene snapshots. Our results show plausible audio variation that remains robust to motion and geometry changes. We suggest that wave solvers can be a practical approach to real‐time dynamic acoustics. We share the complete C++ code of our “Planeverb” system.

Common acoustic sources, like voices or musical instruments, exhibit strong frequency and directional dependence. When transported through complex environments, their anisotropic radiated field undergoes scattering, diffraction, and occlusion before reaching a directionally-sensitive listener. We present the first wave-based interactive auralization system that encodes and renders a complete reciprocal description of acoustic wave fields in general scenes. Our method renders directional effects at freely moving and rotating sources and listeners and supports any tabulated source directivity function and head-related transfer function. We represent a static scene's global acoustic transfer as an 11-dimensional bidirectional impulse response (BIR) field, which we extract from a set of wave simulations. We parametrically encode the BIR as a pair of radiating and arriving directions for the perceptually-salient initial ( direct ) response, and a compact 6 × 6 reflections transfer matrix capturing indirect energy transfer with scene-dependent anisotropy. We render our encoded data with an efficient and scalable algorithm - integrated in the Unreal Engine ™ - whose CPU performance is agnostic to scene complexity and angular source/listener resolutions. We demonstrate convincing effects that depend on detailed scene geometry, for a variety of environments and source types.

We study the detailed temporal evolution of echo density in impulse responses for applications in acoustic analysis and rendering on general environments. For this purpose, we propose a smooth sorted density measure that yields an intuitive trend of echo density growth with time. This is fitted with a general power-law model motivated from theoretical considerations. We validate the framework against theory on simple room geometries and present experiments on measured and numerically simulated impulse responses in complex scenes. Our results show that the growth power of echo density is a promising statistical parameter that shows noticeable, consistent differences between indoor and outdoor responses, meriting further study.

Precomputed sound propagation samples acoustics at discrete scene probe positions to support dynamic listener locations. An offline 3D numerical simulation is performed at each probe and the resulting field is encoded for runtime rendering with dynamic sources. Prior work place probes on a uniform grid, requiring high density to resolve narrow spaces. Our adaptive sampling approach varies probe density based on a novel "local diameter" measure of the space surrounding a given point, evaluated by stochastically tracing paths in the scene. We apply this measure to layout probes so as to smoothly adapt resolution and eliminate undersampling in corners, narrow corridors and stairways, while coarsening appropriately in more open areas. Coupled with a new runtime interpolator based on radial weights over geodesic paths, we achieve smooth acoustic effects that respect scene boundaries as both the source or listener move, unlike existing visibility-based solutions. We consistently demonstrate quality improvement over prior work at fixed cost.

Many spectrum estimation methods and speech enhancement algorithms have previously been evaluated for noise-robust speaker identification (SID). However, these techniques have mostly been evaluated over artificially noised, mismatched training tasks with GMM-UBM speaker models. It is therefore unclear whether performance improvements observed with these methods translate to a broader range of noisy SID tasks. This study compares selected spectrum estimation methods from three classes: cochlear filterbanks, alternative time-domain windowing, and linear prediction-based techniques, as well as a set of frequencydomain noise reduction algorithms, across a suite of 8 evaluation tasks. The evaluation tasks are designed to expand upon the limited tasks addressed in past evaluations by exploring three research questions: performance on real noise versus artificial noise, performance on matched training tasks versus mismatched tasks, and performance when paired with an i-vector backend versus a GMM-UBM backend. We find that noise-robust spectrum estimation methods can improve the performance of SID systems over the range of noise tasks evaluated, including real noisy tasks, matched training tasks, and i-vector backends. However, performance on the typical GMM-UBM mismatched artificially noised case did not predict performance on other tasks. Finally, the matched enrollment case is a significantly different problem than the mismatched enrollment case. Index Terms: mismatched condition, noise robustness, robust features, speaker identification, speech enhancement

Physical task stress affects the acoustic speech wave in various ways. Motivated by observations that fundamental frequency and open quotient are affected by physical task stress, this study examines the effects of physical task stress on a set of glottal features. It is shown that a set of six glottal features can be used for physical stress detection, implying that physical task stress affects vocal fold behavior. It is also shown that the distributions of these six glottal features, across all available speech data, are not affected by physical task stress, leading to the conclusion that covariation in the features due to physical task stress represents different behavioral responses to physical stress. Age and exertion level are explored and rejected as explanatory variables for behavioral types. It is shown, however, that the glottal measurements show changes in distribution when restricted to one recording session of one speaker, suggesting that a given speaker may adopt and retain a particular response to physical task stress for the duration of a task.

Whisper is used by speakers in certain circumstances to protect personal information. Due to the differences in production mechanisms between neutral and whispered speech, there are considerable differences between the spectral structure of neutral and whispered speech, such as formant shifts and shifts in spectral slope. This study analyzes the dependency of these differences on speakers and phonemes by applying a Vector Taylor Series (VTS) approximation to a model of the transformation of neutral speech into whispered speech, and estimating the parameters of this model using an Expectation Maximization (EM) algorithm. The results from this study shed light on the speaker and phoneme dependency of the shifts of neutral to whisper speech, and suggest that similarly derived model adaptation or compensation schemes for whisper speech/speaker recognition will be highly speaker dependent. Index Terms: whispered speech, speech analysis

Physical task stress is known to affect the fundamental frequency of speech. This study of two American English vowels /IY/ and /AH/ investigates whether physical task stress affects the center frequencies of formants F1 and F2, and whether it affects the glottal open quotient, and whether these effects are different for different speakers, the different vowels, and two different vowel contexts. Formant center frequencies are measured from the acousticwaveform, and the glottal openquotient is measured from the electroglottograph signal. The study finds in general that the production of vowels is affected by physical task stress. In particular, the study finds that F1, F2, and the glottal open quotient are affected by physicaltask stress. It also finds that the effects of stress on F1 vary for different speakers, and that the effects of stress on the glottal open quotient vary for different combinations of speakerand vowel. Index Terms: physical task stress, open quotient, electroglottograph

Intra-session and inter-session variability in the Multi-session Audio Research Project (MARP) corpus are contrasted in two experiments that exploit the long-term nature of the corpus. In the first experiment, Gaussian Mixture Models (GMMs) model 30-second session chunks, clustering chunks using the Kullback-Leibler (KL) divergence. Cross-session relationships are found to dominate the clusters. Secondly, session detection with 3 variations in training subsets is performed. Results showed that small changes in long-term characteristics are observed throughout the sessions. These results enhance understanding of the relationship between long-term and short-term variability in speech and will find application in speaker and speech recognition systems. Index Terms: speaker identification, session variability

It is known that speech under physical task stress degrades speech system performance. Therefore, an analysis of speech under physical task stress is performed across several parame-ters to identify acoustic correlates. Formal listener tests are also performed to determine the relationship between acoustic corre-lates and perception. To verify the statistical significance of all results, student-t statistical tests are applied. It was found that fundamental frequency decreases for many speakers, that ut-terance duration increases for some speakers and decreases for others, and that the glottal waveform is quantifiably different for many speakers. Perturbation of two speech features, funda-mental frequency and the glottal waveform, is applied in listener tests to quantify the degree to which these features convey phys-ical stress content in speech. Finally, the enhanced understand-ing of physical task stress speech provided here is discussed in the context of speech systems. Index Terms: physical task stress, stress analysis 1.