This paper introduces high-quality audio coding using psychoacoustic models. This technology is now abundant, with gadgets named after a standard (mp3 players) and the ability to play high-quality audio from literally billions of devices. The usual paradigm for these systems is based on filterbanks, followed by quantization and coding, controlled by a model of human hearing. The paper describes the basic technology, theoretical framework to apply to check for optimality, and the most prominent standards built on the basic ideas and newer work.

We consider the problem of multiple-loudspeaker low-frequency room equalization for a wide listening area, where the equalized loudspeaker is helped using the remaining ones. Using a spatial discretization of the listening area, we formulate the problem as a multipoint error minimization between desired and synthesized magnitude frequency responses. The desired response and cost function are formulated with a goal of capturing the room's spectral power profile, and penalizing strong resonances. Considering physical and psychoacoustical observations, we argue for the use of gain-limited, short, and well-localized equalization filters, with an additional delay for loudspeakers that help the equalized one. We propose a convex optimization framework for computing room equalization filters, where the mentioned filter requirements are incorporated as convex constraints. We verify the effectiveness of our equalization approach through simulations.

Measuring binaural room impulse responses (BRIRs) for different rooms and different persons is a costly and time-consuming task. In this paper, we propose a method that allows to compute BRIRs from a B-format room impulse response (B-format RIR) and a set of head-related transfer functions (HRTFs). This enables to measure the room-related properties and head-related properties of BRIRs separately, reducing the amount of measurements necessary for obtaining BRIRs for different rooms and different persons to one B-format RIR measurement per room and one HRTF set per person. The BRIRs are modeled by applying an HRTF to the direct sound part of the B-format RIR and using a linear combination of the reflections part of the B-format RIR. The linear combination is determined such that the spectral and frequency-dependent interaural coherence cues match those of corresponding directly measured BRIRs. A subjective test indicates that the computed BRIRs are perceptually very similar to corresponding directly measured BRIRs.

Super-directional loudspeaker arrays can be used to achieve high directivity in a limited low-frequency range. As opposed to microphone arrays, the distance between the loudspeakers has to be relatively large, resulting in aliasing starting at relatively low frequencies. On the other hand, mounting a loudspeaker on a rigid baffle (e.g., a rigid cylinder or sphere) increases its directivity with frequency. Using super-directional array techniques at low frequencies and leveraging loudspeakers' increased directivity at high frequencies enables achieving high directivity both at low and high frequencies. The design of baffled circular loudspeaker arrays and an improved beamforming procedure for achieving high directivity in a broad frequency range is described.

In handsfree tele or video conferencing acoustic echoes arise due to the coupling between the loudspeaker and microphone. Usually an acoustic echo canceler (AEC) is used for eliminating the undesired echoes. The weaknesses of AEC are that it is relatively complex and that it is not robust against non-linearities occurring when using low-end components such as small loudspeakers. We are describing an algorithm which models the acoustic echo path by means of an overall delay and a coloration effect filter. The echo is removed by means of short-time spectral modification. The algorithm design aims at low computational complexity and high robustness. The effect of echo path changes and non-linearities on the performance of this algorithm are investigated.

Parametric multichannel audio coding represents an audio signal as one single audio channel plus side information. The side information contains estimates of perceptually relevant differences between the original audio channels. Usually, time difference, level difference, and coherence cues are considered. These cues determine, to a large degree, the auditory spatial image that is perceived when playing back multichannel audio signals. Level difference and time difference synthesis is simple: Different gain factors and delays are applied to the sum signal in subbands for generating the different decoder output channels. However, it is not as obvious how coherence cues can be synthesized. Several heuristic methods for coherence synthesis were proposed previously. In this paper, we are proposing a systematic approach for coherence synthesis. The coherence that is measured in the encoder between a pair of channels is reproduced in the decoder. For that purpose, de-correlation filters modeling late reverberation with impulse responses of a length of several hundred milliseconds are used, resulting in the ability of the scheme to generate naturally sounding diffuse sound. A method for reducing the computational complexity of the scheme is presented. The results of a subjective test indicate that the proposed scheme achieves good audio quality. Furthermore, the scheme was compared to a previous scheme without multichannel coherence synthesis and performs significantly better for all items tested.

This paper studies the quality of multimedia content at very low bitrates. We carried out subjective experiments for assessing audiovisual, audio-only, and video-only quality. We selected content and encoding parameters that are typical of mobile applications. Our focus were the MPEG-4 AVC (a.k.a. H.264) and AAC coding standards. Based on these data, we first analyze the influence of video and audio coding parameters on quality. We investigate the optimal trade-off between bits allocated to audio and to video under global bitrate constraints. Finally, we explore models for the interactions between audio and video in terms of perceived audiovisual quality

Full-duplex hands-free telecommunication systems employ an acoustic echo canceler (AEC) to remove the undesired echoes that result from the coupling between a loudspeaker and a microphone. Traditionally, the removal is achieved by modeling the echo path impulse response with an adaptive finite impulse response (FIR) filter and subtracting an echo estimate from the microphone signal. It is not uncommon that an adaptive filter with a length of 50-300 ms needs to be considered, which makes an AEC highly computationally expensive. In this paper, we propose an echo suppression algorithm to eliminate the echo effect. Instead of identifying the echo path impulse response, the proposed method estimates the spectral envelope of the echo signal. The suppression is done by spectral modification-a technique originally proposed for noise reduction. It is shown that this new approach has several advantages over the traditional AEC. Properties of human auditory perception are considered, by estimating spectral envelopes according to the frequency selectivity of the auditory system, resulting in improved perceptual quality. A conventional AEC is often combined with a post-processor to reduce the residual echoes due to minor echo path changes. It is shown that the proposed algorithm is insensitive to such changes. Therefore, no post-processor is necessary. Furthermore, the new scheme is computationally much more efficient than a conventional AEC.

Acoustic echoes arise whenever there is acoustic coupling between a loudspeaker and a microphone. A traditional solution for eliminating the undesired echo signal is an acoustic echo canceler (AEC), which identifies the echo path between a loudspeaker and a microphone by means of an adaptive filter. The echo signal can be canceled successfully when the modeling filter approaches the true echo path. In practice, however, a modeling filter often differs from the true echo path due to complicated reasons such as environment changes, the lack of knowledge about the length of the echo path, and so on, resulting in residual echo signals. Another way to mitigate the echo effect is through echo suppression. Unlike an AEC, an acoustic echo suppressor (AES) achieves echo attenuation by means of spectral modification. This approach usually has a much lower complexity, and is robust against minor echo path changes. However, it sometimes introduces audible distortions to the processed signal. Many practical systems combine an AEC and an AES in a sequential way such that the former achieves major echo cancellation, and the latter attenuates the residual echoes. In this paper, we propose a novel hybrid approach for addressing the echo problem. The full-band signal is split into two frequency bands, with AEC being applied to the lower band, and an AES to the upper band, respectively. Through subjective tests we demonstrate the superior quality and robustness of this new method, compared to a full-band AEC or a full-band AES.

Binaural Cue Coding (BCC) is a method for multichannel spatial rendering based on one down-mixed audio channel and BCC side information. The BCC side information has a low data rate and it is derived from the multichannel encoder input signal. A natural application of BCC is multichannel audio data rate reduction since only a single down-mixed audio channel needs to be transmitted. An alternative BCC scheme for efficient joint transmission of independent source signals supports flexible spatial rendering at the decoder. This paper (Part I) discusses the most relevant binaural perception phenomena exploited by BCC. Based on that, it presents a psychoacoustically motivated approach for designing a BCC analyzer and synthesizer. This leads to a reference implementation for analysis and synthesis of stereophonic audio signals based on a Cochlear Filter Bank. BCC synthesizer implementations based on the FFT are presented as low-complexity alternatives. A subjective audio quality assessment of these implementations shows the robust performance of BCC for critical speech and audio material. Moreover, the results suggest that the performance given by the reference synthesizer is not significantly compromised when using a low-complexity FFT-based synthesizer. The companion paper (Part II) generalizes BCC analysis and synthesis for multichannel audio and proposes complete BCC schemes including quantization and coding. Part II also describes an alternative BCC scheme with flexible rendering capability at the decoder and proposes several applications for both BCC schemes.

Binaural Cue Coding (BCC) is a method for multichannel spatial rendering based on one down-mixed audio channel and side information. The companion paper (Part I) covers the psychoacoustic fundamentals of this method and outlines principles for the design of BCC schemes. The BCC analysis and synthesis methods of Part I are motivated and presented in the framework of stereophonic audio coding. This paper, Part II, generalizes the basic BCC schemes presented in Part I. It includes BCC for multichannel signals and employs an enhanced set of perceptual spatial cues for BCC synthesis. A scheme for multichannel audio coding is presented. Moreover, a modified scheme is derived that allows flexible rendering of the spatial image at the receiver supporting dynamic control. All aspects of complete BCC encoder and decoder implementations are discussed, such as down-mixing of the input signals, low complexity estimation of the spatial cues, and quantization and coding of the side information. Application examples are given and the performance of the coder implementations are evaluated and discussed based on subjective listening test results.

Binaural Cue Coding (BCC) offers a compact parametric representation of auditory spatial information. This representation can be applied to stereophonic or multi-channel audio compression. It allows to reconstruct the spatial image given a mono audio signal and spatial cues that require only a low data volume. This paper focuses on the extraction of the spatial cues from a stereophonic signal with a BCC analyzer. Results of a first subjective quality evaluation confirm that this technique is able to approximate the spatial image of critical reference signals. Thus BCC has the potential to offer highly efficient stereophonic or multi-channel coding, even at very low bit rates that currently permit coding of only one mono channel.

We present a novel concept for representing multi-channel audio signals: Binaural Cue Coding (BCC). BCC aims at separating the basic audio content and the information relevant for spatial perception. A multi-channel audio signal is represented as a mono signal and BCC parameters. We present two types of applications of BCC. Firstly, a number of separate sound source signals are reduced to a mono signal and BCC parameters. In this case, the decoder has control over the location of each source in auditory space. In other words, the decoder can render spatial images as if the separate source signals were given. Secondly, a multi-channel audio signal is reduced to a mono signal and BCC parameters. In this case the decoder generates a multi-channel signal with a spatial image similar to the spatial image of the input signal of the encoder. Results from a subjective test suggest that BCC, combined with existing mono audio coders, offers better quality than conventional stereo and multi-channel perceptual transform audio coders for a wide range of bitrates.

This paper presents a new approach to the adaptation of a wavelet filterbank based on perceptual and rate-distortion criteria. The system makes use of a wavelet-packet transform where each subband can have an individual time-segmentation. Boundary effects can be avoided by using overlapping blocks of samples and therefore switching bases is possible at every tree-level without affecting other subbands. A modified psychoacoustic model using perceptual entropy can control the switching of the wavelet filterbank and the individual time-segmentation of every subband allows one to take advantage of temporal masking. Additionally a rate-distortion measure can control the filterbank for lossless audio coding applications or in cases where large coding gains can be achieved without using perceptual criteria. The weight of the perceptual measure as well as the weight of the rate-distortion measure can be selected individually, enabling to trade lossless coding versus perceptual coding.