In this paper we present the differentiable log-Mel spectrogram (DMEL) for audio classification. DMEL uses a Gaussian window, with a window length that can be jointly optimized with the neural network. DMEL is used as the input layer in different neural networks and evaluated on standard audio datasets. We show that DMEL achieves a higher average test accuracy for sub-optimal initial choices of the window length when compared to a baseline with a fixed window length. In addition, we analyse the computational cost of DMEL and compare to a standard hyperparameter search over different window lengths, showing favorable results for DMEL. Finally, an empirical evaluation on a carefully designed dataset is performed to investigate if the differentiable spectrogram actually learns the optimal window length. The design of the dataset relies on the theory of spectrogram resolution. We also empirically evaluate the convergence rate to the optimal window length.

Coherence estimation between speech envelope and electroencephalography (EEG) is a proven method in neural speech tracking. This paper proposes an improved coherence estimation algorithm which utilises phase sensitive multitaper cross-spectral estimation. Estimated EEG coherence differences between attended and ignored speech envelopes for a hearing impaired (HI) population are evaluated and compared. Testing was made on 31 HI subjects and showed significant coherence differences for grand averages over the delta, theta, and alpha EEG bands. Significance of increased coherence for attended speech was stronger for the new method compared to the traditional method. The new method of estimating EEG coherence, improves statistical detection performance and enables more rigorous data-based hypothesis-testing results.

Joint time-frequency representations are important tools when estimating the instantaneous frequency. The widely used spectrogram is known to have poor energy localisation, which the reassignment method improves. However, the reassignment method is sensitive to noise. In this paper we present a multitaper reassigned spectrogram (MTRS) that is robust to noise and tailored to short duration transient signals. The MTRS is designed for oscillating transients with Gaussian envelopes and uses the Hermite functions as windows. The novel reassignment coordinates needed for the MTRS are derived in this paper. The MTRS localises energy at the instantaneous frequencies of transients, while white Gaussian noise is made flatter by the use of multiple windows. It is shown that the MTRS is more noise robust compared to reassignment with just one window, and that it improves on the energy localisation of the spectrogram. Thus, our MTRS is sparse, easy to interpret, and well suited for instantaneous frequency estimation, even for signals with overlapping transients and low signal-to-noise ratios.

The traditional parametric approach to Granger causality (GC), based on linear vector autoregressive modeling, suffers from difficulties related to the inaccurate modeling of the generative process. These limits can be solved by using nonparametric spectral estimates in the frequency-domain formulation of GC, also known as spectral GC. In a simulation study, we compare the mean square error of the estimated spectral GC using different multitaper spectral estimators, finding that the Peak Matched multitapers are preferable for estimating spectral GC characterized by peaks. As an illustrative example, we apply the non-parametric approach to the analysis of brain functional connectivity in steady-state visually evoked potentials.

In this paper, the matched reassigned spectrogram is expanded into a novel matched phase reassignment (MPR) method based on the reassigned cross-spectrogram. It is shown that for two phase synchronized oscillating transient signals, the method gives perfect time-frequency (TF) localization. The MPR is thereby a TF local measure of phase synchronization and can also be used to measure phase difference between short transient signals. We explore the technique to measure the phase difference, and compare to commonly used phase estimators, for different types of disturbances. We also investigate the methods ability to classify phase synchronized and non-synchronized signals. Simulations show that the MPR method outperforms the state-of-the-art methods. An illustrative example of phase difference estimates of the electrical signals measured from the brain is included.

The reassignment vectors of the matched reassigned spectrogram (MRS) have shown to be sensitive to noise, with resulting degraded precision in the time-frequency localization. In this paper we propose a multitaper reassignment (mtRS) method for estimation of time- and frequency locations of Gaussian envelope transients. We present novel expressions for the reassignment vectors with use of a non-matched window. These reassignment vectors are then averaged with the usual MRS reassignment vectors forming a multitaper reassignment with use of orthogonal windows. The mtRS is shown to give a higher precision in the time- and frequency estimates in low SNR compared to the usual MRS and also in comparison to other state-of-the-art methods.

Previous studies have shown that measurements of beluga whale (Delphinapterus leucas) and bottlenose dolphin (Tur-siops truncatus) echolocation signals at off-axis angles may be well modeled using a Gaussian bell function or a chirped Gaussian function as envelope. In this paper, we apply a novel technique, the matched window reassignment, for investigation of a range of models of dolphin echolocation transients. Using a set of parameterized skewed envelopes and a dictionary-based estimation algorithm, it is shown that the Rényi entropy can be used as an evaluation measure of the model fit. Acoustic recordings from an array of hydrophones of a beluga's echolocation signal are investigated, where the Gaussian envelope frequency modulation function resulted in the lowest Rényi entropy mean and the two-parametric Gum-bel model showed the best observed fit.

In this paper optimal spectral analysis window shapes, using weighted discrete prolate spheroidal sequences as basis functions, are proposed. These windows are not typically positive or even. The windows are time-shifted, combining the computational efficiency of the Welch method and the appealing property of predefined frequency resolution of the Thomson spectral estimator. The parameters of the optimal windows are found by minimising the resulting spectral covariances and optimising the window overlap, for the predetermined frequency resolution and number of windows. The windows are found to have low side lobes, giving small spectral leakage, and the final spectral estimate gives close to optimal variance reduction, i.e. the covariance between different sub-spectra is very small.

High-resolution time-frequency (TF) images of multi-component signals are of great interest for visualization, feature extraction and estimation. The matched Gaussian multitaper spectrogram has been proposed to optimally resolve multi-component transient functions of Gaussian shape. Hermite functions are used as multitapers and the weights of the different spectrogram functions are optimized. For a fixed number of multitapers, the optimization gives the approximate Wigner distribution of the Gaussian shaped function. Increasing the number of multitapers gives a better approximation, i.e. a better resolution, but the cross-terms also become more prominent for close TF components. In this submission, we evaluate a number of different concentration measures to automatically estimate the number of multitapers resulting in the optimal spectrogram for TF images of dolphin echolocation signals. The measures are evaluated for different multi-component signals and noise levels and a suggestion of an automatic procedure for optimal TF analysis is given. The results are compared to other well known TF estimation algorithms and examples of real data measurements of echolocation signals from a beluga whale (Delphinapterus leucas) are presented.

The reassignment technique is used to increase localization for signal components in the time-frequency representation. The technique gives perfect localization for infinite linear chirp-signals, impulses and constant frequency signals but not for short non-stationary signals. In this paper, a scaled reassignment is proposed, based on the spectrogram using a Gaussian window. The resulting reassignment gives perfect localization for a Gaussian function when the window length matches the function length. Based on the scaled reassignment, an algorithm that estimates the Gaussian function length is also proposed.

In this paper, we introduce a time-frequency spectral estimator for smooth spectra, allowing for irregularly sampled measurements. A non-parametric representation of the time dependent (TD) covariance matrix is formed by assuming that the spectrum is piecewise linear. Using this representation, the time-frequency spectrum is then estimated by solving a convex covariance fitting problem, which also, as a byproduct, provides an enhanced estimation of the TD covariance matrix. Numerical examples using simulated non-stationary processes show the preferable performance of the proposed method as compared to the classical Wigner-Ville distribution and a smoothed spectrogram.

Different short-term spectrum estimators for speaker verification under additive noise are considered. Conventionally, mel-frequency cepstral coefficients (MFCCs) are computed from discrete Fourier transform (DFT) spectra of windowed speech frames. Recently, linear prediction (LP) and its temporally weighted variants have been substituted as the spectrum analysis method in speech and speaker recognition. In this paper, 12 different short-term spectrum estimation methods are compared for speaker verification under additive noise contamination. Experimental results conducted on NIST 2002 SRE show that the spectrum estimation method has a large effect on recognition performance and stabilized weighted LP (SWLP) and minimum variance distortionless response (MVDR) methods yield approximately 7 % and 8 % relative improvements over the standard DFT method at −10 dB SNR level of factory and babble noises, respectively in terms of equal error rate (EER).

In speech and audio applications, short-term signal spectrum is often represented using mel-frequency cepstral coefficients (MFCCs) computed from a windowed discrete Fourier transform (DFT). Windowing reduces spectral leakage but variance of the spectrum estimate remains high. An elegant extension to windowed DFT is the so-called multitaper method which uses multiple time-domain windows (tapers) with frequency-domain averaging. Multitapers have received little attention in speech processing even though they produce low-variance features. In this paper, we propose the multitaper method for MFCC extraction with a practical focus. We provide, first, detailed statistical analysis of MFCC bias and variance using autoregressive process simulations on the TIMIT corpus. For speaker verification experiments on the NIST 2002 and 2008 SRE corpora, we consider three Gaussian mixture model based classifiers with universal background model (GMM-UBM), support vector machine (GMM-SVM) and joint factor analysis (GMM-JFA). Multitapers improve MinDCF over the baseline windowed DFT by relative 20.4% (GMM-SVM) and 13.7% (GMM-JFA) on the interview-interview condition in NIST 2008. The GMM-JFA system further reduces MinDCF by 18.7% on the telephone data. With these improvements and generally noncritical parameter selection, multitaper MFCCs are a viable candidate for replacing the conventional MFCCs.

An idea is presented where multitapers are extracted from a known cross-spectrum and applied for estimation of the coherence function. An important property of the extracted windows are the time-delay between left and right tapers. Therefore, time-delayed Thomson multitapers are also proposed as an approximation. Simulations are performed using different linear filters as well as disturbing white noise and comparisons are made with the results of the usual Thomson and Welch methods. An example of estimation of EEG coherence is also shown.

Usually the mel-frequency cepstral coefficients (MFCCs) are derived via Hamming windowed DFT spectrum. In this paper, we advocate to use a so-called multitaper method instead. Mul-titaper methods form a spectrum estimate using multiple win-dow functions and frequency-domain averaging. Multitapers provide a robust spectrum estimate but have not received much attention in speech processing. Our speaker recognition exper-iment on NIST 2002 yields equal error rates (EERs) of 9.66 % (clean data) and 16.41 % (-10 dB SNR) for the conventional Hamming method and 8.13 % (clean data) and 14.63 % (-10 dB SNR) using multitapers. Multitapering is a simple and robust alternative to the Hamming window method. Index Terms: speaker verification, multiple window method 1.

Usually the mel-frequency cepstral coefficients are estimated either from a periodogram or from a windowed periodogram. We state a general estimator which also includes multitaper estimators. We propose approximations of the variance and bias of the estimate of each coefficient. By using Monte Carlo computations, we demonstrate that the approximations are accurate. Using the proposed formulas, the peak matched multitaper estimator is shown to have low mean square error (squared bias + variance) on speech-like processes. It is also shown to perform slightly better in the NIST 2006 speaker verification task as compared to the Hamming window conventionally used in this context.

The aim of this paper is to find a multiple window estimator that is mean square error optimal for cepstrum estimation. The estimator is compared with some known multiple window methods as well as with the parametric AR-estimator. The results show that the new estimator has high performance, especially for data with large spectral dynamics, and that it is also robust against parameter choices. Simulated speech data is used for the evaluation. It is also shown that the windows of the estimator can be approximated with the sinusoidal multiple windows and that the weighting factors of the different periodograms can be analytically computed.

This paper presents multiple windows with suppressed sidelobes that approximate a time-frequency kernel. The multiple windows are given from the eigenvalue decomposition of the time-lag kernel. By using a penalty matrix both in time- and frequency domain and solving a generalized eigenvalue problem, the multiple windows are achieved. Examples are given and the resulting sidelobe suppression and the number of windows needed are analysed

Peak matched multiple windows are found as the Karhunen-Loeve basis functions of a predefined peaked spectrum. With a penalty function, an optimization procedure can be constrained with resulting control of sidelobes. Weighting factors, included in the averaging of the periodograms, can be designed to fulfill certain constraints. Desirable properties are low variance and small bias. This paper presents a discussion of minimization of variance at the peak compared with the optimization that also include the neighbourhood of the peak.

We address the problem of detecting double-talk in a full duplex transmission line. A new double-talk detector (DTD) based on measuring the similarity between the far- and near-end speech signals is proposed. The detector is block oriented and operates in the frequency domain where the similarity between the signals is measured by means of the coherence function. The coherence is estimated with a short sequence of data by exploiting the multiple window spectrum estimation technique. Theoretical evaluation and examples of its performance are presented. The proposed DTD operates accurately in a wide range of situations, i.e., a difference in speech levels and hybrid attenuations ranging from 0 to 20 dB.

The paper proposes a new multiple window method for estimating a peaked spectrum. The multiple windows are adapted to the signal, giving a less biased estimate for estimation of peaks than does the Thomson multiple window method. Still the result from estimation of a flat spectrum shows comparable results in variance reduction. The method is based on solution of an eigenvalue problem where the eigenvectors of a special correlation matrix are used as multiple windows. The correlation matrix corresponds to a low frequency dominant spectrum with limited bandwidth. The design results in windows that are further improved by a penalty function to reduce leakage from nearby frequencies. This gives a better estimate when the process contains of large spectrum dynamics.