This work investigates pretrained audio representations for few shot Sound Event Detection. We specifically address the task of few shot detection of novel acoustic sequences, or sound events with semantically meaningful temporal structure, without assuming access to non-target audio. We develop procedures for pretraining suitable representations, and methods which transfer them to our few shot learning scenario. Our experiments evaluate the general purpose utility of our pretrained representations on AudioSet, and the utility of proposed few shot methods via tasks constructed from real-world acoustic sequences. Our pretrained embeddings are suitable to the proposed task, and enable multiple aspects of our few shot framework.

Pre-trained model representations have demonstrated state-of-the-art performance in speech recognition, natural language processing, and other applications. Speech models, such as Bidirectional Encoder Representations from Transformers (BERT) and Hidden units BERT (HuBERT), have enabled generating lexical and acoustic representations to benefit speech recognition applications. We investigated the use of pre-trained model representations for estimating dimensional emotions, such as activation, valence, and dominance, from speech. We observed that while valence may rely heavily on lexical representations, activation and dominance rely mostly on acoustic information. In this work, we used multi-modal fusion representations from pre-trained models to generate state-of-the-art speech emotion estimation, and we showed a 100% and 30% relative improvement in concordance correlation coefficient (CCC) on valence estimation compared to standard acoustic and lexical baselines. Finally, we investigated the robustness of pre-trained model representations against noise and reverberation degradation and noticed that lexical and acoustic representations are impacted differently. We discovered that lexical representations are more robust to distortions compared to acoustic representations, and demonstrated that knowledge distillation from a multi-modal model helps to improve the noise-robustness of acoustic-based models.

In this paper we propose a frequency-domain technique to identify and extract the ambience information in stereo audio signals. The method is based on the computation of an inter-channel coherence index and a non-linear mapping function that allow us to determine time-frequency regions that consist mostly of ambience components in the two-channel signal. Ambience signals are then synthesized and used to feed the surround channels of a multichannel playback system. Simulation results demonstrate the effectiveness of the technique in extracting ambience information and up-mix tests on real audio reveal the various advantages and disadvantages of the system compared to previous up-mix strategies.

In this paper we describe a system that suppresses the acoustic interference due to the coupling between the microphone and the loudspeakers of a hands-free multi-channel desktop audio system. The proposed system operates in the short-time Fourier transform domain and uses spectral subtraction to suppress the unwanted interference, which consists of the local audio and the remote speech signal (echo). The interference estimate is obtained with a sub-band RLS-based adaptive multi-channel echo canceller. Test results show that under some adverse conditions and with low complexity constraints the system can achieve better and more consistent speech quality than a time-domain acoustic echo canceller.

We describe a new method for blind system identification that uses the cross relation properties between two or more sensor signals to estimate the impulse responses of the channels. The method performs as well or better than other similar blind identification techniques under noisy and ill-conditioned channel conditions, and is computationally simpler to implement.

Experimentally measured head-related transfer functions reveal that the interaural time delay varies from person to person. Furthermore, it is not constant around a cone of confusion, but can vary by as much as 18% of the maximum interaural delay. The major sources for this variation are shown to be the shape of the head and the displacement of the ears from the center of the head. A simple ellipsoidal head model is presented that can accurately account for this ITD variation and can be adapted to individual listeners.

In this paper we propose a new method of processing speech degraded by reverberation. The method is based on analysis of short (2 ms) segments of data to enhance the regions in the speech signal having high signal to reverberant component ratio (SRR). The short segment analysis shows that SRR is different in different segments of speech. The processing method involves identifying and manipulating the linear prediction residual in three different regions of the speech signal, namely, the high SRR region, the low SRR region and only the reverberation component region. A weighting function is derived to modify the LP residual. The weighted residual samples are used to excite the time-varying LP all-pole filter to obtain perceptually enhanced speech.

To overcome the problems related with the long impulse responses produced by reverberation, we use a long time window (high frequency resolution) analysis during the channel normalization steps of the feature extraction process in automatic speech recognition (ASR). After normalization, a trade between frequency and time resolution is used to increase the rate at which the time information is sampled (short-time domain), yielding an appropriate domain to derive ASR features. Experiments on data with reverberation times of about 0:5 s show that the new technique achieves significant performance improvement of a speech recognizer under reverberation, with only some performance degradation on clean speech. 1. INTRODUCTION In spite of the efforts by many researches during the last 50 years, there has been very little success in reducing the effects of reverberation in speech communications. The effects of reverberation on the details of the speech signal are complicated and difficult to ...

In this paper we propose a method for enhancement of speech in the presence of additive noise. The objective is to selectively enhance the high SNR regions in the noisy speech in the temporal and spectral domains, without causing significant distortion in the resulting enhanced speech. This is proposed to be done at three different levels: (a) At the gross level, by identifying the regions of speech and noise in the temporal domain, (b) At the finer level, by identifying the regions of high and low SNR portions in the noisy speech, and (c) At the short--time spectrum level, by enhancing the spectral peaks over spectral valleys. Processing of noisy speech for enhancement involves mostly weighting the LP residual samples. The weighted residual samples are used to excite the time-- varying LP filter to produce enhanced speech. 1. INTRODUCTION Speech signal collected under normal environmental conditions is usually degraded due to noise and distortions. Performance of speech systems depe...

We present a simple analysis showing that channel normalization techniques are less effective when applied to spectral energies obtained by (weighted) summation of components of the short-time Fourier power spectrum of speech. We show that applying channel normalization processing prior to critical band integration or linear predictive all-pole modeling improves the effectiveness of the techniques.

In this paper we explore speech dereverberation techniques whose principle is the recovery of the envelope modulations of the original (anechoic) speech [ 3 ], [4].Based on our previous experience with such kind of processing for additive noise reduction applications, we apply a data designed lterbank technique [2] to the reverberant speech.Comparing our results with other works we discuss the eectiveness and limitations of this type of approaches.

Finite impulse response (FIR) Wiener-like filters are applied to time trajectories of the cubic-root compressed short-term power spectrum of noisy speech recorded over cellular telephone communications. Informal listenings indicate that the technique brings a noticeable improvement to the quality of processed noisy speech while not causing any significant degradation to clean speech. Alternative filter structures are being investigated as well as other potential applications in cellular channel compensation and narrowband to wideband speech mapping.

A new technique is presented which improves the subjective quality of band-limited speech. The approach is based on a linear model of speech production, in which we independently estimate the spectral envelope and excitation function for a broad-bandwidth speech signal to reconstruct missing frequency components in narrow-bandwidth speech.