Singing voice separation based on deep learning relies on the usage of time-frequency masking. In many cases the masking process is not a learnable function or is not encapsulated into the deep learning optimization. Consequently, most of the existing methods rely on a post processing step using the generalized Wiener filtering. This work proposes a method that learns and optimizes (during training) a source-dependent mask and does not need the aforementioned post processing step. We introduce a recurrent inference algorithm, a sparse transformation step to improve the mask generation process, and a learned denoising filter. Obtained results show an increase of 0.49 dB for the signal to distortion ratio and 0.30 dB for the signal to interference ratio, compared to previous state-of-the-art approaches for monaural singing voice separation.

Monaural singing voice separation task focuses on the prediction of the singing voice from a single channel music mixture signal. Current state of the art (SOTA) results in monaural singing voice separation are obtained with deep learning based methods. In this work we present a novel recurrent neural approach that learns long-term temporal patterns and structures of a musical piece. We build upon the recently proposed Masker-Denoiser (MaD) architecture and we enhance it with the Twin Networks, a technique to regularize a recurrent generative network using a backward running copy of the network. We evaluate our method using the Demixing Secret Dataset and we obtain an increment to signal-to-distortion ratio (SDR) of 0.37 dB and to signal-to-interference ratio (SIR) of 0.23 dB, compared to previous SOTA results.

This paper deals with the automatic transcription of solo bass guitar recordings with an additional estimation of playing techniques and fretboard positions used by the musician. Our goal is to first develop a system for a robust estimation of the note parameters pitch, onset, and duration (score-level parameters). As a second step, we aim to automatically detect the applied plucking and expression style as well as the fret and string positions for each note (instrument-level parameters). Our approach is to first apply a note onset detection followed by a tracking of the fundamental frequency contours based on a reassigned magnitude spectrogram. Then, we model the spectral envelope of each note and derive various timbre-related audio features. Using a support vector machine classifier, we automatically classify the instrument-level parameters for each detected note event. Our results show that the proposed system achieves accuracy values above 0.88 for the estimation of the plucking style, expression style, and string number for isolated note samples. As an additional contribution, we analyze the influence of the note duration characteristics in the classification performance. In a score-level evaluation on a novel public dataset of solo bass guitar tracks, our method outperforms three existing transcription algorithms for bass transcription in polyphonic music as well as a melody transcription algorithm for monophonic music.

We present a system to realistically model the sound of bass guitars, and how to estimate the corresponding parameters from the sound of a bass guitar alone, without other physical measurements. Our model includes plucking and expression styles of the musician, like vibrato or bending, and the string number for a realistic modeling and reproduction of the sound. We show that we can estimate the playing techniques and the string number with relatively high accuracy.

Modern audio coding technologies apply methods of bandwidth extension (BWE) to efficiently represent audio data at low bitrates. An established method is the well-known spectral band replication (SBR) that is part of MPEG High Efficiency Advanced Audio Coding (HE-AAC). However, if the signal features a distinct harmonic spectral structure, the use of these methods tends to result in audible artifacts, because the harmonic structure is not reconstructed correctly. In this paper a bandwidth extension method is proposed which eliminates the undesirable effects and allows for an efficient implementation in the Modified Discrete Cosine Transform (MDCT) domain. The proposed Harmonic Spectral Bandwidth Extension (HSBE) method uses arbitrary frequency shifts for modulating the replicated spectrum in a way that the harmonic structure of the signal is preserved. A listening test demonstrates the advantage of the proposed method compared to the state of the art.

In this paper, we present a novel audio synthesis model that allows us to simulate bass guitar tones with 11 different playing techniques to choose from. In contrast, previous approaches focussing on bass guitar synthesis only implemented the two slap techniques. We apply a digital waveguide model extended by different modular parts to imitate the sound production process on this instrument. The results of aMUSHRA listening test reveal that an audio coding scheme based on the presented algorithm offers a high perceived sound quality in comparison to conventional low bit-rate coding schemes while requiring a much lower bit-rate.

In this paper, we present the results of a pre-study on music performance analysis of ensemble music. Our aim is to implement a music classification system for the description of live recordings, for instance to help musicologist and musicians to analyze improvised ensemble performances. The main problem we deal with is the extraction of a suitable set of audio features from the recorded instrument tracks. Our approach is to extract rhythm-related audio features and to apply them for regression-based modeling of eight more general musical attributes. The model based on Partial Least-Squares Regression without preceding Principal Component Analysis performed best for all of the eight attributes.

In this paper,we present a feature-based approach for the classification of different playing techniques in bass guitar recordings. The applied audio features are chosen to capture typical instrument sounds induced by 10 different playing techniques. A novel database that consists of approx. 4300 isolated bass notes was assembled for the purpose of evaluation. The usage of domain-specific features in a combination of feature selection and feature space transformation techniques improved the classification accuracy by over 27% points in comparison to a state-of-the-art baseline system. Classification accuracy reached 93.25% and 95.61% for the recognition of plucking and expression styles respectively.

In the field of audio coding a technique called Spectral Band Replication (SBR) is used in several codecs to reduce the data rate. We already developed an SBR tool with very little algorithmic delay for use in low delay applications like teleconferencing or live music performance but it produces a relatively high amount of side information. Further it only supports an SBR crossover frequency at half the signal bandwidth. This paper presents an enhanced SBR tool for low delay applications which uses a method known from speech coding called Codebook Mapping (CBM) to get a reduced side information data rate and an optional 3 band decomposition to reach low crossover frequencies. This leads to a low delay SBR tool with a reduced amount of side information and flexible crossover frequencies.

Traditionally, speech coding and audio coding were separate worlds. Based on different technical approaches and different assumptions about the source signal, neither of the two coding schemes could efficiently represent both speech and music at low bitrates. This paper presents a unified speech and audio codec, which efficiently combines techniques from both worlds. This results in a codec that exhibits consistently high quality for speech, music and mixed audio content. The paper gives an overview of the codec architecture and presents results of formal listening tests comparing this new codec with HE-AAC(v2) and AMR-WB+. This new codec forms the basis of the reference model in the ongoing MPEG standardization activity for Unified Speech and Audio Coding.

This paper compares two prediction structures for predictive perceptual audio coding in the context of the ultra low delay (ULD) coding scheme. One structure is based on the commonly used AR signal model, leading to an IIR predictor in the decoder. The other structure is based on an MA signal model, leading to an FIR predictor in the decoder. We find that the AR-based predictor has a slightly better performance in case of an undisturbed transmission channel, but the MA-based predictor has a much better performance in case of transmission errors. For a bit error rate (BER) of 1.0e-5, the perceptual quality of the proposed MA model predictor achieves a mean objective difference grade (ODG) of -0.66 ODG whereas the AR model predictor only reaches -3.42 ODG.

We describe an efficient conversion method, which directly converts a desired spectral representation from compressed audio material. The conversion method provides a feature extraction algorithm with a suitable complex frequency representation of an audio signal. The presented conversion allows us to trade off computational complexity with accuracy. We then test several operating points with an MPEG audio feature extraction system. That leads, in general, to a reduction of the computational complexity from O (N log N) to O(N), compared to the conventional method of first decoding and then applying the DFT to the resulting time domain audio signal.

This paper describes high data-rate audio data hiding using the IntMDCT. The IntMDCT is an integer approximation of the MDCT with perfect reconstruction. Based on this transform, we describe a straight-forward way to embed data and extract it in a bit-exact manner while perceptual transparency is maintained. Since the IntMDCT spectrum can be used to obtain a closer approximation of the masking threshold compared to time domain approaches, it is possible to achieve higher data rates. In a simple experimental implementation, we found we could embed data at rates up to about 140 kb/s without introducing audible distortions

The paper presents a comparison of a previous and a new approach to shape quantization noise in low bit rate predictive audio coding. The previous approach uses an adaptation of the step size of a uniform quantizer, the new approach uses a quantizer with clipping. Both approaches are evaluated using a predictive audio coding scheme. The presented results of a listening test show the improved performance of the new approach

In this paper, lossless audio coding using the integer modified discrete cosine transform (IntMDCT) is discussed. The IntMDCT is constructed as an integer approximation of the MDCT using the lifting scheme and is reversible. The rounding error shape of the IntMDCT is derived. When the spectral energy of the input audio signal is concentrated at the low frequencies, the rounding error spectrum limits the lossless coding performance. A method for shaping the rounding error in the transform domain is presented. This rounding error shaping scheme manipulates the error so that it is below the spectral envelope of the signal at the high frequencies in order to improve the lossless coding performance for the signal. Examples of an error shaping filter design are presented and verified by simulations. An IntMDCT-based lossless coding implementation is carried out to illustrate the use of the error shaping filters

The goal of the presented schemes is to obtain an ultra low delay audio coder with a good performance even at low bit rates (around 64 kb/s). The problem to be solved is to gain sufficient frequency resolution at low frequencies for precise low frequency psycho-acoustics and quantization noise shaping, because the ear has a higher frequency resolution at lower frequencies. Our approach is to use warped linear noise shaping pre- and post-filters, and a short DFT for the psycho-acoustic model (length 256), but with frequency warping. We compare four different psychoacoustic versions: DFT with no warping; DFF without warping using warped pre- and post-filters; warping with the so-called NDFT (WDFT); DFT with all-pass delay chain pre-processing. Listening tests show that the best performance is obtained using WDFT.

In this contribution we propose a new way to state and solve the channel equalization problem for an OFDM (orthogonal frequency division multiplexing) system, without the use of a guard interval. Therefore, the spectral efficiency is increased. The approach is based on the channel representation as a polynomial matrix. We develop a new decomposition of this matrix, which diagonalizes the channel matrix independently of the channel realization. It leads to a simple inversion of the channel matrix. The decomposition consists of forward and inverse DFTs and polynomial diagonal matrices. Further on, there is no restriction on the length of the channel impulse response.

This paper proposes an encoding method for high-quality, low-delay audio communication that is robust to losses in packetized transmission. Robustness is provided by a multiple description vector quantization (MDVQ) technique that is designed to minimize the mean-squared error (MSE). The key to applying this technique effectively is the use of psycho-acoustically controlled preand post-filters that make the mean-squared quantization error perceptually relevant. Experiments show that the MDVQ-based encoder yields better results-in both MSE and subjective audio quality-than simple alternative coders with the same low delay.

The integer modified discrete cosine transform (IntMDCT), an integer approximation of the MDCT, is a reversible transform realized by the lifting scheme and thus is a useful transform for lossless audio coding. Because of the integer approximation, however, the approximation error appears as a "noise floor" in the transform domain and limits the lossless coding efficiency. In this paper, a theoretical analysis of the approximation error of the IntMDCT is discussed. The result is then used to design a simple test filter applied to each rounding operation of the IntMDCT in such a way that the error spectrum is shaped towards the low frequencies. As a result, especially when the spectral energy of an input signal is concentrated in the low frequency domain, the lossless coding efficiency is improved.

Recently lifting-based integer transforms have received much attention, especially in the area of lossless audio and image coding. The usual approach is to apply the lifting scheme to each Givens rotation. Especially in the case of long transform sizes in audio coding applications, this leads to a considerable approximation error in the frequency domain. This paper presents a multidimensional lifting approach for reducing this approximation error. In this approach, large parts of the transform are calculated without rounding operations, only the output is rounded and added. The new approach is applied and evaluated for both the integer modified discrete cosine transform (IntMDCT) and the integer fast Fourier transform (IntFFT).

This papers presents an embedded fine grain scalable perceptual and lossless audio coding scheme. The enabling technology for this combined perceptual and lossless audio coding approach is the integer modified discrete cosine transform (IntMDCT), which is an integer approximation of the MDCT based on the lifting scheme. It maintains the perfect reconstruction property and therefore enables efficient lossless coding in the frequency domain. The close approximation of the MDCT also allows us to build a perceptual coding scheme based on the IntMDCT. In this paper a bitsliced arithmetic coding technique is applied to the IntMDCT values. Together with the encoded shape of the masking threshold a perceptually hierarchical bitstream is obtained, containing several stages of perceptual quality and extending to lossless operation when transmitted completely. A concept of encoding subslices is presented in order to obtain a fine adaptation to the masking threshold especially in the range of perceptually transparent quality.

A low delay audio coding scheme for communications applications is proposed. Its compression ratio is comparable to current state-of-the-art audio coding schemes, but with a much lower delay. The source of delay in conventional audio coding are the filters for the subband coding, and the block switching of the filter bank. The block switching leads to high peaks in bit-rate which necessitates a large bit rate buffer to smooth the bit rate for a transmission channel. To avoid or reduce these delays, we replace the subband coding by predictive coding, and the hard switching of the filter bank by soft switching of the predictors. The overall delay becomes 6 ms at 32 kHz sampling rate. A subjective listening test with bit-rates around 64 kb/s for mono signals shows that the new scheme has a comparable quality to a conventional state-of-the-art coder (PAC).

This paper proposes a versatile perceptual audio coding method that achieves high compression ratios and is capable of low encoding/decoding delay. It accommodates a variety of source signals (including both music and speech) with different sampling rates. It is based on separating irrelevance and redundancy reductions into independent functional units. This contrasts traditional audio coding where both are integrated within the same subband decomposition. The separation allows for the independent optimization of the irrelevance and redundancy reduction units. For both reductions, we rely on adaptive filtering and predictive coding as much as possible to minimize the delay. A psycho-acoustically controlled adaptive linear filter is used for the irrelevance reduction, and the redundancy reduction is carried out by a predictive lossless coding scheme, which is termed weighted cascaded least mean squared (WCLMS) method. Experiments are carried out on a database of moderate size which contains mono-signals of different sampling rates and varying nature (music, speech, or mixed). They show that the proposed WCLMS lossless coder outperforms other competing lossless coders in terms of compression ratios and delay, as applied to the pre-filtered signal. Moreover, a subjective listening test of the combined pre-filter/lossless coder and a state-of-the-art perceptual audio coder (PAC) shows that the new method achieves a comparable compression ratio and audio quality with a lower delay.

A novel predictive lossless coding scheme is proposed. The prediction is based on a new weighted cascaded least mean squared (WCLMS) method. To obtain both a high compression ratio and a very low encoding and decoding delay, the residuals from the prediction are encoded using either a variant of adaptive Huffman coding or a version of adaptive arithmetic coding. WCLMS is especially designed for music/speech signals. It can be used either in combination with psycho-acoustically pre-filtered signals to obtain perceptually lossless coding, or as a stand-alone lossless coder. Experiments on a database of moderate size and a variety of pre-filtered mono-signals show that the proposed lossless coder (which needs about 2 bit/sample for pre-filtered signals) outperforms competing lossless coders, such as ppmz, bzip2, Shorten, and LPAC, in terms of compression ratios. The combination of WCLMS with either of the adaptive coding schemes is also shown to achieve better compression ratios and lower delay than an earlier scheme combining WCLMS with Huffman coding over blocks of 4096 samples.

A design method for low latency multicarrier transmission is presented. It can be considered as a generalization of the trailing-zeros transmitter approach (Scaglione et al., 1999). The generalization mainly consists of using FIR redundant filter banks for the transmitter and receiver instead of pure block transforms and choosing the guard interval independently of the channel impulse response length. Thanks to the latter, we can design a multicarrier transmission system with a low latency time, which is a critical parameter for online applications, even for the case that the channel has a long impulse response, as eg, a twisted-pair copper wire line of several miles length. The design of the transmitter and receiver is based on a Smith decomposition of the channel. The advantages as well as limitations of the new algorithm are discussed.

A novel predictive lossless coding scheme is proposed. The prediction is based on a new weighted cascaded least mean squared (WCLMS) method. WCLMS is especially designed for music/speech signals. It can be used either in combination with psycho-acoustically pre-filtered signals to obtain perceptually lossless coding, or as a stand-alone lossless coder. Experiments on a database of moderate size and a variety of pre-filtered mono-signals show that the proposed lossless coder (which needs about 2 bit/sample for pre-filtered signals) outperforms competing lossless coders, WaveZip, Shorten, LTAC and LPAC, in terms of compression ratios.

High quality multimedia streaming applications over the Internet require very low packet loss rates. The Internet is characterized by long bursts of packet losses and delays. A large receive buffer can be used to mitigate the effects of packet losses and delays. However, a large receive buffer introduces a large delay in the playback of a packet. This large delay could be annoying at the start of a program or during switch over to another channel in a multi-channel broadcast. We introduce a separate low-delay tuning stream to address this start-up problem. In the steady state, this tuning stream is synchronized appropriately with the high-delay steady state stream to give an enhanced composite signal.

A novel concept for perceptual audio coding is presented which is based on the combination of a pre- and post-filter, controlled by a psychoacoustic model, with a transform coding scheme. This paradigm allows modeling of the temporal and spectral shape of the masked threshold with a resolution independent of the used transform. By using frequency warping techniques the maximum possible detail for a given filter order can be made frequency-dependent and thus better adapted to the human auditory system. The filter coefficients are represented efficiently by LSF parameters which can be adaptively interpolated over time. First experiments with a system obtained by extending an existing transform codec showed that this approach can significantly improve the performance for speech signals, while the performance for other signals remained the same.

A new filter structure and design method for time-varying cosine modulated FIR filter banks with critical sampling, perfect reconstruction, and an efficient implementation is presented. The proposed filter banks have an arbitrary system delay which can be chosen in the design process and is independent of the arbitrary filter length, hence making a low system delay possible. The time variation includes changing the number of bands and/or filters during signal processing while maintaining critical sampling and perfect reconstruction. The transition windows can be overlapping, which improves the frequency responses. It is based on a factorization of the polyphase matrices into a cascade of 2 types of simple matrices.

Historically, exact reconstruction FIR filter banks have had system delays of L-1, where L is the length of the analysis and synthesis filters. Recently it was shown that the system delay could be made less than L-1, which is attractive in applications like speech coding where excessive delays are annoying. In this paper, a formulation and new design algorithm are introduced for two-band low-delay filter banks. The formulation is related to that of two-band lattice filter banks and provides a broad range of design flexibility within a compact framework. Both exact reconstruction and specified system delay are guaranteed by the structure of the framework.