Signal processing based on spherical harmonic representations requires an efficient and accurate modeling of the radial functions whose spectrum is commonly described by spherical Bessel and Hankel functions. In this paper, the time-domain radial functions for spherical waves are realized as FIR filters. The filter design exploits the continuous-time representations of the radial functions which have a finite temporal support and are described by the Legendre polynomials. Since the discontinuities occurring in the time domain exhibit infinite temporal bandwidth, a direct time-domain sampling results in frequency-domain aliasing. In order to reduce the aliasing artifacts, the time-domain discontinuities of different orders are replaced with band-limited functions that are analytically derived from the Lagrange interpolation kernel. The FIR coefficients are then obtained by sampling the band-limited representations which leads to a considerable reduction in spectral aliasing. The coefficients are given in closed form expressed solely by polynomials which makes the proposed method suited for real-time applications.

This paper presents a discrete-time model of the spherical harmonics expansion describing a sound field. The so-called radial functions are realized as digital filters, which characterize the spatial impulse responses of the individual harmonic orders. The filter coefficients are derived from the analytical expressions of the time-domain radial functions, which have a finite extent in time. Due to the varying degrees of discontinuities occurring at their edges, a time-domain sampling of the radial functions gives rise to aliasing. In order to reduce the aliasing distortion, the discontinuities are replaced with the higher-order anti-derivatives of a band-limited step function. The improved spectral accuracy is demonstrated by numerical evaluation. The proposed discrete-time sound field model is applicable in broadband applications such as spatial sound reproduction and active noise control.

Theoretic assumptions for sound field synthesis include an anechoic environment for reproduction. As in practice loudspeaker arrays are usually installed inside listening rooms, the synthesised sound field is altered by reflections. These physical changes potentially influence human perception of the sound field. This paper analyses the localisation properties of 2.5D Wave Field Synthesis in free field and in listening rooms with different reflection factors for varying spacings between the secondary sources, in comparison to the localisation of real sources in a listening experiment. Only the directional aspect of localisation in the horizontal plane, not distance or elevation, is investigated. Stimuli were presented using dynamic binaural synthesis. Binaural room impulse responses of the listening rooms were simulated with the image source method. The results show that accurate localisation in Wave Field Synthesis in a reflective environment is only possible for small secondary source spacings as has already been known for free field conditions from the literature. Furthermore, the localisation ability of listeners for both real and virtual sources is slightly degraded by reflections leading to a larger localisation error with a greater localisation blur.

The avoidance of spatial aliasing is a major challenge in the practical implementation of sound field synthesis. Such methods aim at a physically accurate reconstruction of a desired sound field inside a target region using a finite ensemble of loudspeakers. In the past, different theoretical treatises of the inherent spatial sampling process led to anti-aliasing criteria for simple loudspeaker array arrangements, e.g., lines and circles, and fundamental sound fields, e.g., plane and spherical waves. Many criteria were independent of the listener's position inside the target region. Within this paper, a geometrical framework based on ray-approximation of the underlying synthesis problem is proposed. Unlike former approaches, this model predicts spatial aliasing artefacts for arbitrary convex loudspeaker arrays and as a function of the listening position and the desired sound field. Anti-aliasing criteria for distinct listening positions and extended listening areas are formulated based on the established predictions. For validation, the model is applied to different analytical sound field synthesis approaches: The predicted spatial structure of the spatial aliasing agrees with numerical simulation of the synthesised sound fields. Moreover, it is shown within this framework, that the active prioritization of a control region using so-called local sound field synthesis approaches does indeed reduce spatial aliasing artefacts. For the scenario under investigation, a method for local wave field synthesis achieves an artefact-free synthesis up to a frequency which is between 2.9 and 17.3 times as high as for conventional wave field synthesis.

Sound field synthesis aims at the reproduction of an arbitrary target sound field over an extended listening area applying a densely spaced loudspeaker ensemble. Two basic analytic methodologies-the explicit and the implicit-exist in order to derive the required loudspeaker driving functions. The explicit solution aims at the direct solution of the involved integral equation describing the general sound field synthesis problem, resulting in driving functions in the form of a spectral integral. The implicit solution extracts the driving function from an appropriate boundary integral representation of the target sound field. So far the relationship between two approaches was investigated for target field specific synthesis scenarios. For linear arrays this paper introduces a high-frequency approximation for the explicit solution resulting in a novel, purely spatial domain formulation of the direct approach. The presented driving functions allow the synthesis of an arbitrary virtual sound field, optimizing the reproduction on an arbitrary reference line. It is furthermore shown that for an arbitrary virtual sound field, the implicit solution constitutes a high-frequency approximation of the explicit method.

Sound field synthesis techniques including wave field synthesis and near-field-compensated higher order ambisonics aim at a physically accurate reproduction of a desired sound field inside an extended listening area. This area is surrounded by loudspeakers individually driven by their respective driving signals. The latter have to be chosen such that the superposition of all emitted sound fields coincides with the desired one. Due to practical limitations, artefacts impair the synthesis accuracy resulting in a perceivable change in timbre. Recently, two approaches to so-called local wave field synthesis were published that enhance the reproduction accuracy in a limited region while allowing stronger artefacts outside. This paper reports on two listening experiments comparing conventional techniques for sound field synthesis with the mentioned approaches. Furthermore, the influence of different parametrizations for local wave field synthesis is investigated. The results show that the enhanced reproduction accuracy in local wave field synthesis leads to a reduction of perceived colouration, if a suitable parametrization is chosen.

Wave Field Synthesis allows the reconstruction of an arbitrary target sound field within a listening area by using a secondary source contour of spherical monopoles. While phase correct synthesis is ensured over the whole listening area, amplitude deviations are present besides a predefined reference curve. So far, the existence and potential shapes of this reference curve was not extensively discussed in the Wave Field Synthesis literature. This paper introduces improved driving functions for 2.5D Wave Field Synthesis. The novel driving functions allow for the control of the locations of amplitude correct synthesis for arbitrarily shaped-possibly curved-secondary source distributions. This is achieved by deriving an expressive physical interpretation of the stationary phase approximation leading to the presented unified Wave Field Synthesis framework. The improved solutions are better suited for practical applications. Additionally, a consistent classification of existing implicit and explicit 2.5D sound field synthesis solutions as special cases of the unified framework is given.

Local sound field synthesis allows for synthesizing a given desired sound field inside a limited target region such that the field is free of considerable spatial aliasing artifacts. Spatial aliasing artifacts are a consequence of overlaps due to unavoidable repetitions of the space-spectral coefficients of the secondary source driving function. We analyze various conceivable analytic ways of restricting the bandwidth of the spatial spectrum of the driving function such that considerable overlapping is prevented: local spatial bandlimitation (A), spectral windowing (B), and local spatial band-limitation plus spectral windowing (C). While solution B is computationally significantly more efficient than A and C, it provides only limited control over the spatial location around which the aliasing-free region evolves. Solutions A and C provide more flexibility and higher accuracy whereby both achieve largely identical results so that the spectral windowing after the local spatial bandlimitation may be skipped. We present a detailed analysis of the properties of the spatial aliasing artifacts arising in the synthesis of a virtual plane wave. We establish a procedure for predicting the maximum possible size of the aliasing-free target region depending on its location and on the propagation direction of the desired sound field. The results can help reducing regularization in numerical solutions as they represent physical limitations that can be considered in the choice of parameters.

This paper introduces a binaural model for the localisation and tracking of a moving sound source’s azimuth in the horizontal plane. The model uses a nonlinear state space representation of the sound source dynamics including the current position of the listener’s head. The state is estimated via an unscented Kalman Filter by comparing the interaural level and time differences of the binaural signal with semi-analytically derived localisation cues from a spherical head model. The localisation performance of the model is evaluated in combination with two different head movement approaches based on openand closed-loop control strategies. The results show that adaptive strategies outperform non-adaptive ones and are able to compensate systematic deviations between the spherical head model and human heads.

In this paper we present a system that allows to "touch", grab and manipulate sounds in mid-air. Further, arbitrary objects can seem to emit sound. We use spatial sound reproduction for sound rendering and computer vision for tracking. Using our approach, sounds can be heard from anywhere in the room and always appear to originate from the same (possibly moving) position, regardless of the listener's position. We demonstrate that direct "touch" interaction with sound is an interesting alternative to indirect interaction mediated through controllers or visual interfaces. We show that sound localization is surprisingly accurate (11.5 cm), even in the presence of distractors. We propose to leverage the ventriloquist effect to further increase localization accuracy. Finally, we demonstrate how affordances of real objects can create synergies of auditory and visual feedback. As an application of the system, we built a spatial music mixing room.

This paper reviews the current state of loudspeaker-based spatial sound reproduction methods from technical perspective as well as perceptual perspective. A nomenclature is developed that allows for a strict separation between these two perspectives. The physical fundamentals, practical realization, and results from perceptual studies are discussed for a number of well-established and emerging reproduction techniques. Further, the paper outlines novel approaches to spatial sound evaluation in terms of perceived quality and provides a comparison of current approaches.

The theory of sound field synthesis methods like Wave Field Synthesis (WFS) and Near-field Compensated Higher Order Ambisonics (NFC-HOA) may be formulated based on the assumption of continuous distributions of secondary sources that enclose the receiver area in the general case. In practice, a finite number of discrete loudspeakers is used, which constitutes a fundamental departure from the theoretical requirements. In this paper, we present a detailed analysis of the consequences of this spatial discretization on the synthesized sound field via an analytical frequency-dependent modal decomposition of the latter for the case of Gaussian sampling. It is shown that the underlying mechanisms are essentially similar to those in the discretization of circular secondary source distributions so that the results obtained in the latter context may be directly applied. The outstanding parallel in the discretization of spherical and circular distributions is the fact that in both cases, repetitions in a given space frequency domain occur. Therefore, the spatial bandwidth of the desired sound field has essential influence on the properties of the evolving artifacts. We propose to categorize sound field synthesis methods into spatially narrowband, wideband, and fullband approaches and show that NFC-HOA constitutes a spatially narrowband method and that WFS constitutes a spatially fullband method.

Methods like Wave Field Synthesis aim at the synthesis of a given desired sound field over a large receiver area. Practical limitations lead to considerable artifacts commonly referred to as spatial aliasing. Above a given frequency these artifacts are apparent anywhere in the receiver area when linear arrays of secondary sources are considered. This paper presents an analytical approach based on the Spectral Division Method which achieves an accuracy of the synthesized sound field which is significantly higher than in conventional approaches in a limited target zone. This local increase in accuracy is achieved via a manipulation of the spatial bandwidth of the secondary source driving function.

Hands-free full-duplex communication systems require acoustic echo cancelers to reduce echoes. Multichannel sound reproduction enhances realism in virtual reality and multimedia communication systems. However, in the case of multichannel systems the acoustic echo cancellation problem is challenging because of the non-uniqueness of the solution in the least squares sense. There fore, preprocessing techniques are required. Known preprocessing approaches act in the temporal domain. In this paper we propose an acoustic echo cancellation preprocessing stage which bases on the spatial diversity offered by massive multichannel reproduction systems.

Head-related impulse responses (HRIRs), which are measured from an acoustic source to the left and right ear, characterize the acoustic properties of the outer ear. Due to the involved measurement effort, HRIRs are typically available only for source positions on a circle or surface of a sphere. Range extrapolation techniques aim at calculating HRIRs with a different source distance. A number of techniques have been proposed in the past which are based on expansions of HRIRs in terms of surface spherical harmonics. This paper presents an alternative approach which bases on the interpretation of a BRIR dataset as virtual loudspeaker array which is driven by Wave Field Synthesis in order to achieve range extrapolation.

We present an analytical approach targeting the physical reproduction of sound fields by means of linear distributions of loudspeakers. Unlike with conventional analytical approaches like wave field synthesis, the employed loudspeakers are not required to be omnidirectional. The approach does not compensate for loudspeaker properties which deviate from certain assumptions but it rather allows for the explicit consideration of these properties within some limits which are outlined.

Combining the channels of a multiple input/multiple output (MIMO) system into suitably chosen modes by a domain transformation offers great improvements of adaptive filtering algorithms. In this paper we present an algorithm for adaptive MIMO filtering, called source-domain adaptive filtering (SDAF), with application to multichannel acoustic echo cancellation operating in an optimally adjusted transform domain without requiring a-priori knowledge about the system. Experimental results show a significant performance improvement compared to fixed transformation bases.

In this paper, we consider physical reproduction of sound fields via planar and linear distributions of secondary sources (i.e., loudspeakers). The presented approach employs a formulation of the reproduction equation in spatial frequency domain which is explicitly solved for the secondary source driving signals. Wave field synthesis (WFS), the alternative formulation, can be shown to be equivalent under equal assumptions. Unlike the WFS formulation, the presented approach does not employ a far-field approximation when linear secondary source distributions are considered but provides exact results. We focus on the investigation of the spatial truncation and discretization of the secondary source distribution occurring in real-world implementations and present a rigorous analysis of evanescent and propagating components in the reproduced sound field.

Sound field reproduction methods like higher order Ambisonics which are based on orthogonal expansions always introduce a limitation of the spatial bandwidth of the secondary source driving function. This spatial truncation creates a sweet spot in the center of the secondary source distribution. This spot, or rather area, is ldquosweetrdquo both in terms of spatial aliasing artifacts as well as in terms of accuracy of the desired component of the reproduced wave field. The higher the temporal frequency of the reproduced wave field the smaller is the sweet spot. In this paper we show that the location sweet spot can be moved freely inside the secondary source distribution. The accuracy of the actual reproduced wave field is then significantly higher in the sweet spot than in the same region in the conventional approach.

In this paper, we introduce a direct manipulation tabletop multi-touch user interface for spatial audio scenes. Although spatial audio rendering exists for several decades now, mass market applications have not been developed and the user interfaces still address a small group of expert users. We implemented an easy-to-use direct manipulation interface for multiple users, taking full advantage of the object-based audio rendering mode. Two versions of the user interface have been developed to explore variations in information architecture and will be evaluated in user tests.

In this paper, we present the derivation and investigation of analytical expressions for the loudspeaker driving signals for higher order Ambisonics. The approach relies on the assumption of a continuous distribution of secondary sources on which sampling is performed to yield the actual loudspeaker signals for real-world implementations. For source-free volumes enclosed by the secondary source distribution, this formulation of Ambisonics leads to what is known as simple source approach. Since the simple source approach is theoretically well documented, we will depart from it and concentrate on the special case of a circular distribution of secondary point sources and derive analytical expressions for the driving signals. We furthermore derive a closed-form expression for the actual reproduced wave field for the circular secondary source distribution.

This paper introduces a new algorithm to render virtual sound sources with spatial properties in immersive environments. The algorithm, referred to as multipole-matched rendering, uses the method-of-moments and singular-value decomposition to optimally match a spherical-multipole expansion of the virtual source to the field resulting from a spatially distributed speaker set. The flexibility of this method over other approaches, such as wavefield synthesis, allows for complex speaker geometries, and requires a smaller number of speakers to achieve a similar spatial rendering performance for listeners in immersive environments. The trade-off for the enhanced performance is a smaller area of faithful reproduction. This limited area, however, can be focused around listener locations for a sweet-spot solution. Experimental results are presented from perceptual tests comparing multipole-matched rendering to both wavefield synthesis and stereo rendering using a linear speaker array. The experiments included 13 subjects and demonstrated that the perceived direction of a virtual sound source for the new method is comparable to that of wavefield synthesis (no significant difference). The results demonstrate the potential of multipole-matched rendering as an efficient technique for rendering virtual sound sources in immersive environments.

In multi-party teleconferencing, the transport of separate speech streams to a particular user and the subsequent spatial rendering of the different streams enables a more efficient communication. A simple means of spatial presentation at client side is that of binaural rendering and headphone presentation. For downwardcompatibility, e.g. when the transport mechanism does not support multiple parallel downlink streams, a system is proposed that combines an automatic speaker classification mechanism with a spatial rendering of the segregated streams. The combined system aims at a better separability of the speakers than conventional systems. The paper details the two basic components, namely automatic speaker classification, and binaural rendering. Based on a first evaluation of the approach, a proof of concept is provided, and directions for further improvement are discussed.

This paper presents an audio-visual database that can be used as a reference database for testing and evaluation of video, audio or joint audio-visual person tracking algorithms, as well as speaker localization methods. Additional possible uses include the testing of face detection and pose estimation algorithms. A number of different scenes are included in the database, ranging from simple to complex scenes that can challenge existing algorithms. They include different subjects, with appearances that can cause problems to video tracking algorithms, (e.g. facial features such as beards, glasses, etc.), optimal and artificially created sub-optimal lighting conditions, subject movement based on simple as well as random motion trajectories, different distances from the camera/microphones and occlusion. The database incorporates ground truth data (3D position in time) originating from a commercially available 4-camera infrared (IR) tracking system. Examples of how the database can be used to evaluate video and audio tracking algorithms are also provided.

For high-quality multimedia communication systems such as teleconferencing or virtual reality applications, multichannel sound reproduction is highly desirable. While progress has been made in stereo and multichannel acoustic echo cancellation (MC AEC) in recent years, the corresponding sound reproduction systems still imply a restrained listening area ('sweet spot'). A volume solution for spatial sound in a large listening area is offered by wave field synthesis (WFS) or by ambisonics, where arrays of loudspeakers generate a prespecified sound field. However, before this new technique can be utilized for full-duplex systems, an efficient solution to the MC AEC problem has to be found. This paper presents a novel approach that extends the current state of the art of MC AEC and transform-domain adaptive filtering by reconciling the flexibility of adaptive filtering and the underlying physics of acoustic waves in a systematic and efficient way. To achieve this, the new framework of wave-domain adaptive filtering (WDAF) exploits the spatial information provided by densely sampled contours for both recording and reproduction. Experimental results with a 32-channel AEC verify the concept for both simulated and actually measured room acoustics.

Wave field synthesis is an auralization technique which allows the entire wave field to be controlled within the entire listening area. However, reflections in the listening room interfere with the auralized wave field and may impair the spatial reproduction. Active listening room compensation aims at reducing these impairments by using the playback system. Due to the high number of playback channels used for wave field synthesis, the existing approaches to room compensation are not applicable. A novel approach to active room compensation overcomes these problems by a transformation from the space-time to the wave domain and application of wave-domain adaptive filtering.

In this combustion, we embed full-duplex multichannel communication interfaces for tele-presence systems into a general framework. On the reproduction side, we consider a wide range of multichannel acoustic rendering techniques including traditional stereophony, '5.1' systems, and wave field synthesis using loud speaker arrays for sound immersion. On the recording side, microphone arrays are discussed for capturing clean desired signals with spatial information. Based on this general framework, real-time implementations of such full-duplex multichannel communication systems are then described. We combine wave field synthesis with multichannel acoustic echo cancellation and adaptive beamforming and discuss a real-time implementation on standard desktop and laptop PCs.

Common room compensation algorithms are capable of dereverberating the listening room at some discrete points only. Outside these equalization points the sound quality is often even worse compared to the unequalized case. As wave field synthesis in principle allows to control the wave field within the listening area it can also be used to compensate for the reflections caused by the listening room in the complete listening area. We present a novel approach to listening room compensation which is based upon the theory of wave field synthesis and that results in a large compensated area.

For high-quality multimedia communication systems, such as teleconferencing, or tele-teaching (especially of music), multichannel sound reproduction is highly desirable. While current approaches still rely on a restrained listening area, the sweet spot, a volume solution for a large listening space is offered by the wave field synthesis (WFS) method, where arrays of loudspeakers generate a prespecified sound field. On the recording side of the two-way systems, the use of microphone arrays is an effective approach to cope with undesired signal components in the receiving room. However, before full-duplex communication can be deployed, efficient approaches to the acoustic echo cancellation (AEC) problem in this challenging scenario have to be found. We investigate different options for system integration, after a brief discussion of the current state of the art. We then present a first real-time solution on a regular PC platform, based on an efficient AEC for MIMO (multi-input and multi-output) systems in the frequency-domain.

This paper presents a face localization and tracking algorithm which is based upon skin color detection and principal component analysis (PCA) based eye localization. Skin color segmentation is performed using statistical models for human skin color. The skin color segmentation task results in a mask marking the skin color regions in the actual frame, which is further used to compute the position and size of the dominant facial region utilizing a robust statistics-based localization method. To improve the results of skin color segmentation, a foreground/background segmentation and an adaptive background update scheme are added. Additionally, the derived face position is tracked with a Kalman filter. To overcome the problem of skin color ambiguity, an eye detection algorithm based on PCA is presented.

This paper presents an object localization and tracking algorithm integrating audio and video-based object localization results. A face tracking algorithm and a microphone array are used to compute two single-modality speaker position estimates. These position estimates are then combined into a global position estimate using a decentralized Kalman filter. Experiments with a model railway show that such an approach yields more robust results for audio-visual object tracking than either modality by itself.

Object localization based on audio and video information is important for the analysis of dynamic scenes, such as video conferences or traffic situations. In this paper, we view the the dynamic audio-video object localization problem as a joint recursive estimation problem. It is solved using a decentralized Kalman filter fusing both audio and video position estimates. To better take into account different object maneuvers, multiple state-space equations are also incorporated. The result is a recursive multi-state multi-sensor estimator. Experiments show that it yields significantly improved joint position estimates compared to results achieved by using either an audio or a video system only.