The perceived quality of a sound played in a closed room is often degraded by the added reverberation. In order to combat these effects, the methods of room impulse response equalization may be used. Typically, properties of the human auditory system, such as temporal masking, are used for a better control of the late echoes. There, a prefilter is used to modify the played signal, which renders the echoes inaudible for a given position. When targeting a human listener who is always performing small movement, the method fails due to the spatial mismatch. In such a case, a whole volume needs to be equalized, which requires a huge amount of measurements. In this work, we propose to reduce this burden by measuring only a random subset of the required room impulse responses and reconstruct a grid employing sparse methods. By further interpolation an equalization of the whole target area is achieved.

For conventional sampling of sound-fields, the measurement in space by use of stationary microphones is impractical for high audio frequencies. Satisfying the Nyquist-Shannon sampling theorem requires a huge number of sampling points and entails other difficulties, such as the need for exact calibration and spatial positioning of a large number of microphones. Dynamic sound-field measurements involving tracked microphones may weaken this spatial sampling problem. However, for aliasing-free reconstruction, there is still the need of sampling a huge number of unknown sound-field variables. Thus in real-world applications, the trajectories may be expected to lead to underdetermined sampling problems. In this paper, we present a compressed sensing framework that allows for stable and robust sub-Nyquist sampling of sound fields by use of moving microphones.

The conventional sampling of sound fields by use of stationary microphones is impractical for large bandwidths. Satisfying the Nyquist-Shannon sampling theorem in three-dimensional space requires a huge number of sampling positions. Dynamic sound-field measurements with moving microphones together with a compressed-sensing recovery allow for weakening the spatial sampling problem. For bandlimited signals, the dynamic samples taken along the microphone trajectory may be related to the room impulse responses on a virtual grid in space via spatial interpolation. The tracking of the microphone positions and the knowledge of the excitation sequence allow for setting up a linear system of equations that can be solved for the room impulse responses on the modeled virtual grid. Nevertheless, there is still the necessity for recovering a huge number of sound-field variables, in order to ensure aliasing-free reconstruction. Thus, for practical applications, random or suboptimally chosen trajectories may be expected to lead to underdetermined sampling problems for a given volume of interest. In this paper, we present a compressed sensing framework that enables us to uniquely solve the dynamic sampling problem despite having underdetermined variables. The spatio-temporal sampling problem is integrated into compressed sensing models that allow for stable and robust sub-Nyquist sampling given incoherent measurements. For a modeled equidistant grid and sparse Fourier representations, the influence of the microphone trajectories on the compressed sensing problem is investigated and a simple expression is derived for evaluating trajectories with regard to compressed-sensing based recovery.

The sampling of sound fields involves the measurement of spatially dependent room impulse responses, where the Nyquist-Shannon sampling theorem applies in both the temporal and spatial domains. Therefore, sampling inside a volume of interest requires a huge number of sampling points in space, which comes along with further difficulties such as exact microphone positioning and calibration of multiple microphones. In this paper, we present a method for measuring sound fields using moving microphones whose trajectories are known to the algorithm. At that, the number of microphones is customizable by trading measurement effort against sampling time. Through spatial interpolation of the dynamic measurements, a system of linear equations is set up which allows for the reconstruction of the entire sound field inside the volume of interest.

We present in this work an approach for audio scene classification. Firstly, given the label set of the scenes, a label tree is automatically constructed where the labels are grouped into meta-classes. This category taxonomy is then used in the feature extraction step in which an audio scene instance is transformed into a label tree embedding image. Elements of the image indicate the likelihoods that the scene instances belong to different meta-classes. A class of simple 1-X (i.e. 1-max, 1-mean, and 1-mix) pooling convolutional neural networks, which are tailored for the task at hand, are finally learned on top of the image features for scene recognition. Experimental results on the DCASE 2013 and DCASE 2016 datasets demonstrate the efficiency of the proposed method.

We introduce in this work an efficient approach for audio scene classification using deep recurrent neural networks. An audio scene is firstly transformed into a sequence of high-level label tree embedding feature vectors. The vector sequence is then divided into multiple subsequences on which a deep GRU-based recurrent neural network is trained for sequence-to-label classification. The global predicted label for the entire sequence is finally obtained via aggregation of subsequence classification outputs. We will show that our approach obtains an F1-score of 97.7% on the LITIS Rouen dataset, which is the largest dataset publicly available for the task. Compared to the best previously reported result on the dataset, our approach is able to reduce the relative classification error by 35.3%.

In this paper, we present an efficient approach for audio scene classification. We aim at learning representations for scene examples by exploring the structure of their class labels. A category taxonomy is automatically learned by collectively optimizing a tree-structured clustering of the given labels into multiple metaclasses. A scene recording is then transformed into a label-tree embedding image. Elements of the image represent the likelihoods that the scene instance belongs to the metaclasses. We investigate classification with label-tree embedding features learned from different low-level features as well as their fusion. We show that the combination of multiple features is essential to obtain good performance. While averaging label-tree embedding images over time yields good performance, we argue that average pooling possesses an intrinsic shortcoming. We alternatively propose an improved classification scheme to bypass this limitation. We aim at automatically learning common templates that are useful for the classification task from these images using simple but tailored convolutional neural networks. The trained networks are then employed as a feature extractor that matches the learned templates across a label-tree embedding image and produce the maximum matching scores as features for classification. Since audio scenes exhibit rich content, template learning and matching on low-level features would be inefficient. With label-tree embedding features, we have quantized and reduced the low-level features into the likelihoods of the metaclasses, on which the template learning and matching are efficient. We study both training convolutional neural networks on stacked label-tree embedding images and multistream networks. Experimental results on the DCASE2016 and LITIS Rouen datasets demonstrate the efficiency of the proposed methods.

We introduce a new learned descriptor for audio signals which is efficient for event representation. The entries of the descriptor are produced by evaluating a set of regressors on the input signal. The regressors are class-specific and trained using the random regression forests framework. Given an input signal, each regressor estimates the onset and offset positions of the target event. The estimation confidence scores output by a regressor are then used to quantify how the target event aligns with the temporal structure of the corresponding category. Our proposed descriptor has two advantages. First, it is compact, i.e. the dimensionality of the descriptor is equal to the number of event classes. Second, we show that even simple linear classification models, trained on our descriptor, yield better accuracies on audio event classification task than not only the nonlinear baselines but also the state-of-the-art results.

The human auditory system is very well matched to both human speech and environmental sounds. Therefore, the question arises whether human speech material may provide useful information for training systems for analyzing nonspeech audio signals, e.g., in a classification task. In order to answer this question, we consider speech patterns as basic acoustic concepts, which embody and represent the target nonspeech signal. To find out how similar the nonspeech signal is to speech, we classify it with a classifier trained on the speech patterns and use the classification posteriors to represent the closeness to the speech bases. The speech similarities are finally employed as a descriptor to represent the target signal. We further show that a better descriptor can be obtained by learning to organize the speech categories hierarchically with a tree structure. Furthermore, these descriptors are generic. That is, once the speech classifier has been learned, it can be employed as a feature extractor for different datasets without retraining. Lastly, we propose an algorithm to select a sufficient subset, which provides an approximate representation capability of the entire set of available speech patterns. We conduct experiments for the application of audio event analysis. Phone triplets from the TIMIT dataset were used as speech patterns to learn the descriptors for audio events of three different datasets with different complexity, including UPC-TALP, Freiburg-106, and NAR. The experimental results on the event classification task show that a good performance can be easily obtained even if a simple linear classifier is used. Furthermore, fusion of the learned descriptors as an additional source leads to state-of-the-art performance on all the three target datasets.

In listening room compensation, the aim is to compensate for the degradations that are rendered to an audio signal by transmission in a closed room. Due to multiple reflections of the soundwaves, the listener receives a superposition of delayed and attenuated versions of the source signal. A filter is designed so that the convolution of the room impulse response and the equalizer contains better acoustic properties than the original acoustic channel. For dereverberation, the filter is usually designed by optimizing a solely time-domain based cost function and, therefore, may introduce spectral distortions. Recent methods also consider the frequency-domain representation of the overall system and aim at yielding a flat overall frequency response. However, in some cases, it may be desirable to follow a predefined frequency response rather than obtaining a flat one. Hearing impaired persons, for example, may prefer or even need an amplification of certain frequency ranges. In this work, we propose a new method to jointly shape the time- and the frequency-domain representations of the overall acoustic channel according to prescribed curves. Furthermore, we integrate the concept of auditory scales into the filter design.

Audio event detection has been an active field of research in recent years. However, most of the proposed methods, if not all, analyze and detect complete events and little attention has been paid for early detection. In this paper, we present a system which enables early audio event detection in continuous audio recordings in which an event can be reliably recognized when only a partial duration is observed. Our evaluation on the ITC-Irst database, one of the standard database of the CLEAR 2006 evaluation, shows that: on one hand, the proposed system outperforms the best baseline system by 16% and 8% in terms of detection error rate and detection accuracy respectively; on the other hand, even partial events are enough to achieve the performance that is obtainable when the whole events are observed.

The human auditory system is very well matched to both human speech and environmental sounds. Therefore, the question arises whether human speech material may provide useful information for training systems for analyzing nonspeech audio signals, such as in a recognition task. To find out how similar nonspeech signals are to speech, we measure the closeness between target nonspeech signals and different basis speech categories via a speech classification model. The speech similarities are finally employed as a descriptor to represent the target signal. We further show that a better descriptor can be obtained by learning to organize the speech categories hierarchically with a tree structure. We conduct experiments for the audio event analysis application by using speech words from the TIMIT dataset to learn the descriptors for the audio events of the Freiburg-106 dataset. Our results on the event recognition task outperform those achieved by the best system even though a simple linear classifier is used. Furthermore, integrating the learned descriptors as an additional source leads to improved performance.

Despite the success of the automatic speech recognition framework in its own application field, its adaptation to the problem of acoustic event detection has resulted in limited success. In this paper, instead of treating the problem similar to the segmentation and classification tasks in speech recognition, we pose it as a regression task and propose an approach based on random forest regression. Furthermore, event localization in time can be efficiently handled as a joint problem. We first decompose the training audio signals into multiple interleaved superframes which are annotated with the corresponding event class labels and their displacements to the temporal onsets and offsets of the events. For a specific event category, a random-forest regression model is learned using the displacement information. Given an unseen superframe, the learned regressor will output the continuous estimates of the onset and offset locations of the events. To deal with multiple event categories, prior to the category-specific regression phase, a superframe-wise recognition phase is performed to reject the background superframes and to classify the event superframes into different event categories. While jointly posing event detection and localization as a regression problem is novel, the superior performance on two databases ITC-Irst and UPC-TALP demonstrates the efficiency and potential of the proposed approach.

In listening room compensation, the aim is to compensate for the degradations that are rendered to an audio signal by transmission in a closed room. Due to multiple reflections of the soundwaves, the listener receives a superposition of delayed and attenuated versions of the source signal. A filter is designed so that the convolution of the room impulse response and the equalizer contains better acoustic properties than the original acoustic channel. Common approaches for derever-beration optimize only the time-domain representation of the overall impulse response and may introduce distortions in the frequency domain. Equalization of the frequency response, on the other hand, often does not consider the time-domain behavior in an ideal way. In this paper, we propose a novel method to jointly consider both the time- and frequency-domain behavior. It outperforms the methods known from literature in terms of dereverberation and equalization performance. Results are presented for a room impulse response measured in a real living room.

This paper proposes an approach for the efficient automatic joint detection and localization of single-channel acoustic events us-ing random forest regression. The audio signals are decom-posed into multiple densely overlapping superframes annotated with event class labels and their displacements to the temporal starting and ending points of the events. Using the displacement information, a multivariate random forest regression model is learned for each event category to map each superframe to con-tinuous estimates of onset and offset locations of the events. In addition, two classifiers are trained using random forest clas-sification to classify superframes of background and different event categories. On testing, based on the detection of category-specific superframes using the classifiers, the learned regressor provides the estimates of onset and offset locations in time of the corresponding event. While posing event detection and lo-calization as a regression problem is novel, the quantitative eval-uation on ITC-Irst database of highly variable acoustic events shows the efficiency and potential of the proposed approach. Index Terms: acoustic event detection, regression forest, ran-dom forest, superframe

The purpose of room impulse response reshaping is to reduce reverberation and thus to improve the perceived quality of the received signal by prefiltering the source signal before it is played with a loudspeaker. The filter design is usually carried out by solving an optimization problem. There are, in general, two possibilities to improve the robustness of the equalizers against small movements of the listener and/or receiver; namely multi-position approaches or the utilization of a regularization term. Multi-position approaches suffer from the extensive effort of measuring multiple room impulse responses. Stochastic models may describe the average system error due to spatial mismatch, but only quadratic penalty terms have been considered so far. In this contribution we propose a third method to improve robustness against spatial misalignment. We combine the two approaches by generating multiple realizations of distorted room impulse responses and feeding them into the multiposition algorithm. Based on our previous work, we propose a model to capture the perturbations with respect to the assumed displacement.

By using room impulse response shortening and reshaping it is possible to reduce the reverberation effects and therefore improve the perceived quality. This may be achieved by a prefilter that modifies the overall impulse response to have a faster decay. The traditional filter shortening approach using least-squares methods is fast and directly computable, but it suffers from late echoes. Newer approaches using the p-norm overcome this drawback but are computationally very demanding, as the optimization process uses a gradient-descent approach with slow convergence. In this work we propose a modification to this approach that results in a significantly faster convergence. With this modification, the algorithm is less likely to be trapped in a local minimum and therefore also leads to a better convergence point. The method will be demonstrated on simulated and real-world room impulse responses.

Virtual 3-D sound can be easily delivered to a listener by binaural audio signals that are reproduced via headphones, which guarantees that only the correct signals reach the corresponding ears. Reproducing the binaural audio signal by two or more loudspeakers introduces the problems of crosstalk on the one hand, and, of reverberation on the other hand. In crosstalk cancellation, the audio signals are fed through a network of prefilters prior to loudspeaker reproduction to ensure that only the designated signal reaches the corresponding ear of the listener. Since room impulse responses are very sensitive to spatial mismatch, and since listeners might slightly move while listening, robust designs are needed. In this paper, we present a method that jointly handles the three problems of crosstalk, reverberation reduction, and spatial robustness with respect to varying listening positions for one or more binaural source signals and multiple listeners. The proposed method is based on a multichannel room impulse response reshaping approach by optimizing a -norm based criterion. Replacing the well-known least-squares technique by a -norm based method employing a large value for allows us to explicitly control the amount of crosstalk and to shape the remaining reverberation effects according to a desired decay.

In this paper, we present a new algorithm for solving the permutation ambiguity in convolutive blind source separation. A common approach for separation of convolutive mixtures is the transformation to the time-frequency domain, where the convolution becomes a multiplication. This allows for the use of well-known instantaneous ICA algorithms independently in each frequency bin. However, this simplification leads to the problem of correctly aligning these single bins previously to the transformation to the time domain. Here, we propose a new criterion for solving this ambiguity. The new approach is based on the sparsity of the speech signals and yields a robust depermutation algorithm. The results will be shown on real world examples.

In this paper, we propose to use the scaling ambiguity of convolutive blind source separation for shortening the unmixing filters. An often used approach for separating convolutive mixtures is the transformation to the time-frequency domain where an instantaneous ICA algorithm can be applied for each frequency separately. This approach leads to the so called permutation and scaling ambiguity. While different methods for the permutation problem have been widely studied, the solution for the scaling problem is usually based on the minimal distortion principle. We propose an alternative approach that allows the unmixing filters to be as short as possible. Shorter unmixing filters will suffer less from circular-convolution effects that are inherent to unmixing approaches based on bin-wise ICA followed by permutation and scaling correction. The results for the new algorithm will be shown on a real-world example.

In this paper, we present a new algorithm for solving the permutation ambiguity in convolutive blind source separation. Transformed to the frequency domain, existing algorithms can efficiently solve the reduction of the source separation problem into independent instantaneous separation in each frequency bin. However, this independency leads to the problem of correctly aligning these single bins. The new algorithm models the frequency-domain separated signals by means of the generalized Gaussian distribution and employs the small deviation of the parameters between neighboring bins for the detection of correct permutations. The performance of the algorithm will be demonstrated on synthetic and real-world data.