Neuromorphic computing, inspired by nervous systems, revolutionizes information processing with its focus on efficiency and low power consumption. Using sparse coding, this paradigm enhances processing efficiency, which is crucial for edge devices with power constraints. The Locally Competitive Algorithm (LCA), adapted for audio with Gammatone and Gammachirp filter banks, provides an efficient sparse coding method for neuromorphic speech processing. Adaptive LCA (ALCA) further refines this method by dynamically adjusting modulation parameters, thereby improving reconstruction quality and sparsity. This paper introduces an enhanced ALCA version, the ALCA Central Frequency (ALCA-CF), which dynamically adapts both modulation parameters and central frequencies, optimizing the speech representation. Evaluations show that this approach improves reconstruction quality and sparsity while significantly reducing the power consumption of speech classification, without compromising classification accuracy, particularly on Intel’s Loihi 2 neuromorphic chip.

Single-channel speech enhancement algorithms have seen great improvements over the past few years. Despite these improvements, they still lack the efficiency of the auditory system in extracting attended auditory information in the presence of competing speakers. Recently, it has been shown that the attended auditory information can be decoded from the brain activity of the listener. In this paper, we propose two novel end-to-end deep learning methods referred to as the Brain Enhanced Speech Denoiser (BESD) and the U-shaped Brain Enhanced Speech Denoiser (U-BESD) respectively, that take advantage of this fact to denoise a multi-talker speech mixture without considering further background noises or reverberations. We use a Feature-wise Linear Modulation (FiLM) between the brain activity and the sound mixture, to better extract the features of the attended speaker to perform speech enhancement. We show, using electroencephalography (EEG) signals recorded from the listener, that both BESD and U-BESD successfully extract the attended speaker without any prior information about this speaker. Moreover, U-BESD also outperforms a current state-of-the-art approach that also uses brain activity to perform enhancement. The proposed neural network-based methods would thus make great candidates for realistic applications where no prior information about the attended speaker is available, such as hearing aids, cellphones, or noise cancelling headphones.

The auditory system is extremely efficient in extracting attended auditory information in the presence of competing speakers. Single-channel speech enhancement algorithms, however, greatly lack this efficacy. In this paper, we propose a novel deep learning method referred to as the Brain Enhanced Speech Denoiser (BESD), that takes advantage of the attended auditory information present in the brain activity of the listener to denoise a multi-talker speech. We use this information to modulate the features learned from the sound and the brain activity, in order to perform speech enhancement. We show that our method successfully enhances a speech mixture, without prior information about the attended speaker, using electroencephalography (EEG) signals recorded from the listener. This makes it a great candidate for realistic applications where no prior information about the attended speaker is available, such as hearing aids or cell phones.

In this paper, we introduce a discriminative training algorithm of the non-negative matrix factorization (NMF) model for single-channel enhancement of convolutive noisy speech. The basis vectors for the clean speech and noises are estimated simultaneously during the training stage by incorporating the concept of classification from machine learning. Specifically, we employ the probabilistic generative model (PGM) of classification, specified by an inverse Gaussian distribution, as a priori structure for the basis vectors. Both the NMF and classification parameters are obtained by using the expectation-maximization (EM) algorithm, which guarantees convergence to a stationary point. Experimental results show that the proposed algorithm provides better enhancement performance than the benchmark algorithms.

In this paper, we propose a basis compensation algorithm for non-negative matrix factorization (NMF) models as applied to supervised single-channel speech enhancement. In the proposed framework, we use extra free basis vectors for both the clean speech and noise during the enhancement stage in order to capture the features which are not included in the training data. Specifically, the free basis vectors of the clean speech are obtained by exploiting a priori knowledge based on a Gamma distribution. The free bases of the noise are estimated using a regularization approach, which enforces them to be orthogonal to the clean speech and noise basis vectors estimated during the training stage. Experimental results show that the proposed NMF algorithm with basis compensation provides better performance in speech enhancement than the benchmark algorithms.

In this letter, we introduce a discriminative training algorithm of the basis vectors in the nonnegative matrix factorization (NMF) model for single-channel speech enhancement. The basis vectors for the clean speech and noises are estimated simultaneously during the training stage by incorporating the concept of classification from machine learning. Specifically, we consider the probabilistic generative model (PGM) of classification, which is specified by class-conditional densities, along with the NMF model. The update rules of the NMF are jointly obtained with the parameters of the class-conditional densities using the expectation-maximization (EM) algorithm, which guarantees convergence. Experimental results show that the proposed algorithm provides better performance in speech enhancement than the benchmark algorithms.

A flexible and multipurpose bio-inspired hierarchical model for analyzing musical timbre is presented in this paper. Inspired by findings in the fields of neuroscience, computational neuroscience, and psychoacoustics, not only does the model extract spectral and temporal characteristics of a signal, but it also analyzes amplitude modulations on different timescales. It uses a cochlear filter bank to resolve the spectral components of a sound, lateral inhibition to enhance spectral resolution, and a modulation filter bank to extract the global temporal envelope and roughness of the sound from amplitude modulations. The model was evaluated in three applications. First, it was used to simulate subjective data from two roughness experiments. Second, it was used for musical instrument classification using the k-NN algorithm and a Bayesian network. Third, it was applied to find the features that characterize sounds whose timbres were labeled in an audiovisual experiment. The successful application of the proposed model in these diverse tasks revealed its potential in capturing timbral information.

In Bayesian short-time spectral amplitude (STSA) estimation for speech enhancement, the spectral components are traditionally assumed uncorrelated. However, this assumption is inexact since some correlation is present in practice. We thus investigate a multi-dimensional STSA estimator that assumes correlated frequency components. Since the closed-form solution of this optimum estimator is not readily available, we previously derived closed-form expressions for an upper and a lower bound on the desired estimator. In this paper, we study the proximity between the upper and the lower bounds and propose a new family of estimators that are derived from these bounds and characterized by a scalar parameter 0 ≤ γ ≤ 1, with γ = 0 corresponding to the lower bound and γ = 1 to the upper bound. Experimental results show that the proposed estimators achieve a better performance than existing estimators, especially at high SNR.

In this letter, we show that many existing short-time spectral amplitude (STSA) Bayesian estimators for speech enhancement all have a similarly structured cost function. On this basis, we propose a new cost function that generalizes those of existent Bayesian STSA estimators and then obtain the corresponding closed-form solution for the optimal clean speech STSA. The resulting family of estimators, which we will term the generalized weighted family of STSA estimators (GWSA), features a new parameter that acts only on the estimated clean speech STSA. It is found that this new parameter yields an added flexibility in terms of achievable gain curves when compared to those of existing estimators. Moreover, we show that the new estimator family tends to a Wiener filter for high instantaneous signal-to-noise ratios.

In this paper, we first propose a new family of Bayesian estimators for speech enhancement where the cost function includes both a power law and a weighting factor. Secondly, we set the parameters of the estimator based on perceptual considerations by taking into account the masking properties of the ear and the perceived loudness of sound. Our results show that the new estimator achieves better overall performance than existing Bayesian estimators both in terms of objective and subjective measures. Specifically, it shows a segmental SNR improvement of up to 0.65 dB while it obtains the best scores in a MUSHRA test for both white and aircraft cockpit noises.

We propose a new family of Bayesian estimators for speech enhancement where the cost function includes both a power law and a weighting factor. The parameters of the cost function, and therefore of the corresponding estimator gain, are chosen based on characteristics of the human auditory system, namely, the compressive nonlinearities of the cochlea, the perceived loudness and the ear's masking properties. It is found that choosing the parameters in this way results in a decrease of the estimator gain at high frequencies. This frequency dependence of the gain improves the noise reduction while limiting the speech distortion. Experimental results show that the new estimators achieve better enhancement performance than existing Bayesian estimators such as those based on the minimum mean-square error (MMSE) of the short-time spectral amplitude (STSA), the MMSE of the logarithm of the STSA (LSA) or the weighted euclidien (WE) error, both in terms of objective and subjective measures.