Higher-Order Stereophony is a new spatial audio approach which extends classic stereo to higher order soundfield reproduction and generalised loudspeaker arrays, in a similar manner to Higher-Order Ambisonics. Higher-Order Stereophony reproduces the soundfield accurately across a line only, which must be orientated to align with the listener's interaural axis. This work introduces Dynamic Higher-Order Stereophony, which expands the technique to include listener tracking using dynamic amplitude panning. This means the soundfield is reproduced across a dynamically moving line dependent on the listener's orientation, to ensure correct reproduction of a desired set of binaural signals. A number of classic stereo approaches are shown to be special cases of first order Stereophony, and decoders to reproduce Higher-Order Ambisonics content using the new technique presented. A listening test comparing Higher-Order Stereophony and Ambisonics reproduction, low-passed at 4 kHz to reduce spatial aliasing artefacts, shows that the former technique can perform equally well as Ambisonics to the same truncation order with respect to positional properties of a virtual sound source, while using a smaller number of loudspeakers. The approach can also produce rear virtual sources using only loudspeakers positioned in the front of the listener, however with the requirement of listener tracking.

This work introduces a new theory for spatial audio recording and reproduction named Higher Order Stereophony. Through the use of the Taylor expansion, the technique accurately reproduces a sound field across a line that is orientated as the interaural axis of a listener, to attempt to recreate a set of desired binaural signals. The technique utilises loudspeaker amplitude panning, and is shown to encompass in its framework traditional Stereophony approaches such as the stereo sine law. Therefore, the technique expands Stereophony to higher orders and more loudspeakers, leading to a greater frequency range of accurate reproduction, in a similar manner to Higher Order Ambisonics. Higher Order Stereophony is shown to exhibit many similarities to Higher Order Ambisonics, and decoders to transition between the different sound field representations are derived. Higher Order Stereophony is also re-derived through a mode matching approach using a subset of spherical harmonics, those with degree index equal to zero only. The theoretical results are then validated through experimental measurements using various microphone arrays, considering the reproduced sound field across a single line and the reproduced spherical harmonic coefficients of the sound field.

A method for reproducing binaural audio over an array of loudspeakers is presented. The proposed method is based on spatially-matched filters and is shown to be, in general, more stable than the conventional inverse filter approach. The non-weighted formulation of the method introduces reductions in the reproduced pressure magnitude response at each control point and is therefore inherently lossy. On the other hand, far less error is introduced for frequencies where the Hermitian angle between the row vectors of the plant matrix is close to ninety degrees. For sufficiently dense arrays, little error is introduced at mid-low and high frequencies. The trade-off between reductions in the reproduced target signals and crosstalk cancellation is controlled by introducing frequency-dependent weights to the left and right channel filters. The weights that minimise the error are derived. It is shown that the optimal weight solution emphasises achieved crosstalk cancellation over lossless focusing.

Sound field reproduction is a promising technology for reproducing a desired sound field over a specific area. However, several practical challenges exist, such as the requirement for a large number of loudspeakers and the degradation of spatial reproduction accuracy due to room reflections. This work proposes a sound field reproduction method based on a cylindrical loudspeaker array that is used to both radiate sound directly towards the listener and take advantage of controlled first-order reflections from the room’s walls. The sound field corresponding to one or more virtual sound sources is created by the combination of the direct and reflected sound at the listener position. As long as the lateral walls generate welldefined specular reflections, the simple model of the reverberant field with the proposed formulation enables this method to work without elaborate acoustical measurements. Two approaches with different concepts of controlling the array are compared and their mathematical equivalence is shown. To evaluate the performance, both simulations and experiments were conducted, showing a good agreement with theory.

Conventional channel-based room equalisation can reduce overall colouration caused by the room response, however it cannot separately correct the colouration caused by the late and early parts of the response, or consider the reverberance in the source signal. A room compensation method is developed here for a source signal in which the dry source sound and the associated target reverberant response are encoded separately, which is possible in an object-based audio framework. The target response is modified using the reproduction room response. Subject to some conditions the combined response approximates the target, with accurate early and late equalisations, reverberant balance, and decay timing. Stochastic assumptions are used to simplify the processing, enabling efficient real-time processing of the encoded audio.

A novel Direction-of-Arrival (DOA) estimation method based on a plane wave sound field model was recently proposed for circular microphone arrays [1]. This article presents a detailed theoretical analysis of the method that relies on a Finite Rate of Innovation (FRI) assumption, investigating the impact of different theoretical acoustic source models. The method's estimation accuracy, and robustness against noisy measurement data and deviations from the model are investigated. The estimation performance is validated and assessed on the basis of results obtained from both simulations and experimental data.

The low frequency behaviour of the source strength solution of the two loudspeaker crosstalk cancellation system is analysed. Using the rigid sphere head model, an accurate 1storder approximation of the source strength solution is derived, assuming a centred listener position. It is shown that the 1storder approximation of the source strength solution corresponds to the well-known stereo sine law. The validity of this approximation for real systems is supported by a comparison of simulated low frequency complex source strengths and the real stereo panning functions. To obtain a realistic analysis, the simulated source strengths are created using KEMAR head-related transfer functions. It is shown that the simulated source strengths converge to the stereo sine law in the low frequency limit under the expected conditions.

Compensated amplitude panning (CAP) is a spatial audio reproduction method for loudspeakers that takes the listener head orientation into account. It can produce stable images in any direction using only two loudspeakers. In its original form, CAP is inherently an object-based method, with each image produced separately. An exact and efficient method is presented here for dynamically decoding a first order Ambisonic encoding, which is equivalent to using CAP to separately reproduce the constituents of the encoding. Both the stereo and multichannel cases are considered.

The process of understanding acoustic properties of environments is important for several applications, such as spatial audio, augmented reality and source separation. In this paper, multichannel room impulse responses are recorded and transformed into their direction of arrival (DOA)-time domain, by employing a superdirective beamformer. This domain can be represented as a 2D image. Hence, a novel image processing method is proposed to analyze the DOA-time domain, and estimate the reflection times of arrival and DOAs. The main acoustically reflective objects are then localized. Recent studies in acoustic reflector localization usually assume the room to be free from furniture. Here, by analyzing the scattered reflections, an algorithm is also proposed to binary classify reflectors into room boundaries and interior furniture. Experiments were conducted in four rooms. The classification algorithm showed high quality performance, also improving the localization accuracy, for non-static listener scenarios.

Conventional amplitude panning can be used to produce images of distant objects. An extended panning method is presented that can also produce image cues for the near-field region, by the control of Inter-aural Level Difference cues in the low frequency range below ~1000 Hz. The approach has grown from an adaptive panning method that corrects for the dependence of image direction on head orientation. Stereo panning functions are derived from a general formulation. A single first order filter is required for each image. The method is tested by simulating a range of configurations using measured Head Related Transfer Functions, and also with listening tests. The results confirm the ability of the method to control near-field cues, while also compensating image direction for head rotation.

Amplitude panning produces interaural time difference (ITD) cues that help localize images in directions between loudspeakers. However, if the panning gains are static, then the ITD cues produced in this way vary inconsistently as the listener's head rotates, compared with a real source, and so the dynamic ITD cues are inaccurate. This effect destabilizes the perception of the image and overall scene, and is worse for loudspeakers that are more widely spaced relative to the listener. Based on a simple head model that is accurate in the low-frequency ITD regime, the ITD is calculated for a general field, including those produced by panning. A simple formula is derived relating head orientation, image direction, and a field description vector. Panning functions are then found that compensate for head orientation and are valid for any image direction. For the special case when the listener is facing the image, the functions are equivalent to vector base amplitude panning. The performance is first assessed objectively using measured binaural responses, rather than the simple head model. Subjective comparison is then made with pre-existing listening tests and new listening tests in which the listener's head is tracked to control the panning gains in real-time. These show that images can be stabilized as predicted, and, furthermore, that with the same panning functions, images can be produced in all directions using two loudspeakers placed in front.

Object-based audio is an emerging representation for audio content, where content is represented in a reproduction-format-agnostic way and, thus, produced once for consumption on many different kinds of devices. This affords new opportunities for immersive, personalized, and interactive listening experiences. This paper introduces an end-to-end object-based spatial audio pipeline, from sound recording to listening. A high-level system architecture is proposed, which includes novel audio-visual interfaces to support object-based capture and listener-tracked rendering, and incorporates a proposed component for objectification, that is, recording content directly into an object-based form. Text-based and extensible metadata enable communication between the system components. An open architecture for object rendering is also proposed. The system's capabilities are evaluated in two parts. First, listener-tracked reproduction of metadata automatically estimated from two moving talkers is evaluated using an objective binaural localization model. Second, object-based scene capture with audio extracted using blind source separation (to remix between two talkers) and beamforming (to remix a recording of a jazz group) is evaluated with perceptually motivated objective and subjective experiments. These experiments demonstrate that the novel components of the system add capabilities beyond the state of the art. Finally, we discuss challenges and future perspectives for object-based audio workflows.

In this paper we propose a cuboid-based air-tight indoor room geometry estimation method using combination of audio-visual sensors. Existing vision-based 3D reconstruction methods are not applicable for scenes with transparent or reflective objects such as windows and mirrors. In this work we fuse multi-modal sensory information to overcome the limitations of purely visual reconstruction for reconstruction of complex scenes including transparent and mirror surfaces. A full scene is captured by 360$^{\circ}$ cameras and acoustic room impulse responses (RIRs) recorded by a loudspeaker and compact microphone array. Depth information of the scene is recovered by stereo matching from the captured images and estimation of major acoustic reflector locations from the sound. The coordinate systems for audio-visual sensors are aligned into a unified reference frame and plane elements are reconstructed from audio-visual data. Finally cuboid proxies are fitted to the planes to generate a complete room model. Experimental results show that the proposed system generates complete representations of the room structures regardless of transparent windows, featureless walls and shiny surfaces.

Panning techniques, such as vector base amplitude panning (VBAP), are a widely used practical approach for spatial sound reproduction using multiple loudspeakers. Although limited to a relatively small listening area, they are very efficient and offer good localization accuracy, timbral quality, as well as a graceful degradation of quality outside the sweet spot. The aim of this paper is to investigate optimal sound reproduction techniques that adopt some of the advantageous properties of VBAP, such as the sparsity and the locality of the active loudspeakers for the reproduction of a single audio object. To this end, we state the task of multiloudspeaker panning as an ℓ1optimization problem. We demonstrate and prove that the resulting solutions are exactly sparse. Moreover, we show the effect of adding a nonnegativity constraint on the loudspeaker gains in order to preserve the locality of the panning solution. Adding this constraint, ℓ1-optimal panning can be formulated as a linear program. Using this representation, we prove that unique ℓ1-optimal panning solutions incorporating a nonnegativity constraint are identical to VBAP using a Delaunay triangulation for the loudspeaker setup. Using results from linear programming and duality theory, we describe properties and special cases, such as solution ambiguity, of the VBAP solution.

Sound fields can be encoded with a fixed number of signals, using microphones or panning functions. The sound field may later be reproduced approximately by decoding the signals to a loudspeaker array. The Stereo and Ambisonic systems provide examples. A framework is presented for addressing general questions about such encodings. The first problem considered is the conversion between encodings. The solution is applied to the decoding of scene encodings to a loudspeaker array. This is generalized to the decoding of subscenes where the resolution is focused in an angular window. Within an object-based audio framework such subscenes are useful for representing complex objects without using all the channels required for a full scene. The second problem considered is the compression of a scene encoding to a smaller encoding, from which the original can be reconstructed. The spatial distribution of compression error can be controlled.

In this paper, we propose a system for private sound which is based on the weighted pressure matching method (WPMM). The aim is to design the input signals to an array of loudspeakers that allow for the synthesis of a target field defined with large amplitude variations between the so-called dark points and the listener's position. The system enables listeners to control the trade-off between directivity performance and the accuracy of reproduction of the target field at the listening position when the input energy to the array is limited. This is achieved by calculating the WPMM weight in the dark zone based on a constraint on the characteristics of the sound field in the listening zone. The system is validated for a number of predefined use-case scenarios. The results of the experiments in an anechoic environment with a circular array prototype show that listeners can control the performance trade-off in a wide frequency range. In the second part of the paper, algorithms are presented for the fast update of the input signals when the user selects a new value of the performance constraint.

Microphone arrays as a means of sound field acquisition have been the topic of extensive research for more than eight decades now. A number of designs have been suggested, each trying to overcome difficulties that are inherent to either the decomposition of the sound field, the transducers in use or the presence of the array itself. This work presents a theoretical analysis of circular microphone arrays that do not measure the sound pressure but the component of its gradient that is tangential to a given boundary. Its performance is compared to that of a conventional pressure sensor array as a benchmark. The focus of the analysis and subsequent assessment lies on spatial aliasing and performance in the presence of noise.

Three dimensional surround sound reproduction over large areas is a prevailing challenge due to the enormous numbers of loudspeakers required. In this paper, we propose an array of higher order loudspeakers which provide a mode matching solution to the problem based on 3D wavefield translation. It is shown that for a given bandwidth, the use of Lthorder sources significantly brings down the minimum loudspeaker requirement by a factor of 1=(L + 1)2. Furthermore, the array is shown to be capable of exterior field cancellation, increasing its performance in echoing environments. Design examples are given for interior field, exterior field and interior and exterior combined field reproduction.